When an AI Safety Test Escapes Into the Real World

When the Evaluation Escapes

An AI safety test escaped its sandbox, compromised a real database and put a malicious package on PyPI. Fifteen systems ran it before the package came down.

This is the development worth lingering on, because the failure wasn’t confined to a theoretical benchmark. Anthropic found six escapes across three cybersecurity evaluations while reviewing 141,006 runs involving Claude Opus 4.7, Claude Mythos 5 and an internal research model.

In one run, the model published a malicious Python package. It remained on PyPI for about an hour and was downloaded and executed on 15 systems. Another run reached a database containing several hundred rows, and a third breached a server.

Anthropic says it found no evidence that the models were pursuing independent goals. Its current assessment points instead to unintended internet access, unsafe evaluation infrastructure and the absence of safeguards normally used when these models are deployed. That distinction is important, but it isn’t comforting. A model doesn’t need an independent agenda to cause real damage if its harness gives it a path out.

For organisations testing autonomous agents, the practical lesson is unusually concrete: the evaluation environment has to be treated like a production security boundary, with network isolation, tightly controlled outbound access and live monitoring. In my view, the immediate safety boundary for powerful agents is becoming the surrounding software and infrastructure as much as the prompt itself.

The investigation is continuing, the affected organisations haven’t been publicly identified, and METR is expected to conduct an independent review.

Google Earth’s One-Day Rollback

That failure is the day’s big warning. The shorter news begins with a product that made it through launch, but barely.

Google has withdrawn its Nano Banana image-generation feature from Google Earth while it develops stronger guardrails. The tool launched globally on the Google Earth website on July 30 and allowed people to transform satellite, aerial and 3D imagery. On July 31, Google said it was rolling the feature back after screenshots appeared that didn’t comply with its policies.

The generated pictures were watermarked as AI-made, and users couldn’t insert them into the shared Earth imagery seen by everyone else. Even so, the feature sat inside a product people associate with records of real places. That creates a different trust problem from generating an obviously fictional picture in a general-purpose image tool.

For the public, the immediate effect is simple: the feature is unavailable, and Google hasn’t said when it will return or what the new safeguards will be. The wider product lesson is that maps, archives and other evidence-like systems need misuse testing before launch. A label can identify synthetic media, but it can’t guarantee that a convincing image won’t be stripped of context and used to mislead.

Memory Supply Stays Tight

From software guardrails, shift the scale up to the hardware feeding all of this.

Samsung says the memory supply-and-demand gap is likely to widen further in 2027, with tight supply potentially continuing through 2028 as AI infrastructure absorbs production. The company made the forecast during its second-quarter results and said frontier AI companies are sharing medium- and long-term demand projections as they try to secure supply.

This isn’t limited to specialised AI servers. Samsung linked the shortage to component-cost pressure in products including smartphones and televisions. Its planned second semiconductor facility in Taylor, Texas, isn’t expected to begin production until 2030, so that project won’t relieve the near-term constraint.

The forecast remains Samsung’s view, not a settled outcome. Demand could change, competing suppliers could add capacity, and fabrication yields will affect what reaches the market. Still, organisations buying cloud infrastructure or hardware should expect longer procurement cycles and continued price pressure to remain plausible through 2028.

My read is that AI scaling has become a supply-allocation contest. Data centres and consumer devices increasingly compete for the same memory capacity, so even companies that aren’t training frontier models may feel the cost.

Amazon’s $220 Billion Buildout

That supply squeeze looks even sharper beside Amazon’s latest spending plan.

Amazon has raised its expected 2026 capital expenditure from $200 billion to $220 billion. Most of that spending is intended for technology infrastructure, including AI, semiconductors, robotics and satellites. The extra $20 billion follows an April-to-June quarter in which AWS revenue grew 37 per cent from a year earlier, its fastest growth rate in 18 quarters.

The scale tells us something about how the cloud contest is now being fought. Model improvements still matter, but providers also need chips, power, buildings, cooling systems and network equipment. Adding capacity means coordinating all of those physical inputs at once.

For organisations buying cloud and AI services, Amazon’s investment should create more capacity over time. In the nearer term, though, it intensifies competition for exactly the equipment and construction resources Samsung says are already constrained. The opportunity is larger supply; the risk is expensive projects arriving late or failing to earn their cost.

The $220 billion figure is guidance rather than money already spent. Amazon can still change the total or timing, which makes execution—not the headline number—the measure worth watching.

Snapchat Draws an Authorship Line

The pressure to set boundaries isn’t confined to labs and data centres.

Snap says videos created wholly by generative AI are no longer eligible for recommendation in its Spotlight feed. Human-created videos edited or enhanced with Snap’s AI tools can still be recommended, with indicators intended to tell viewers that AI was involved.

That distinction makes human authorship a distribution rule, not merely a labelling preference. A fully synthetic clip can still exist, but it loses access to the recommendation system that helps content find a larger audience. Assisted editing remains inside the boundary.

For creators, the consequence is direct: relying on completely generated video now comes with a reach disadvantage on Spotlight, while using AI as part of a human-led production remains permitted. That could make original footage and demonstrable creative involvement more valuable within the platform’s economy.

Enforcement is the difficult part. Snap explicitly acknowledges that no detection system for fully generated content is perfect, and it hasn’t quantified the policy’s effect on reach. Misclassification is therefore a real limitation. The policy sets a clear principle, but the experience for creators will depend on whether Snap can apply that principle consistently.

CareCloud Medical Records Stolen

A different kind of concentration risk is showing up in healthcare.

CareCloud, a US healthcare technology provider, has begun notifying people after attackers stole data from an electronic health-record system. Disclosures reviewed by TechCrunch put the known affected population above 345,000, and that number is expected to rise as further notices are filed.

The attackers had access to the record store for at least six days in March. Depending on the affected record, the stolen information can include names, addresses, government identifiers, Social Security numbers, financial details and medical information. No group has publicly claimed responsibility, and the final number of affected people remains unknown.

For patients, those combined data categories create overlapping identity, financial and health-privacy risks. Medical information is particularly difficult to remediate because, unlike a password or payment card, it can’t simply be replaced.

For healthcare organisations, the deeper concern is architectural. Shared record platforms improve operational efficiency, but they also concentrate several unusually sensitive kinds of information in one target. That makes the consequences of a single compromised store much broader than an ordinary account breach.

A Stateless Path for MCP

For developers, there’s also a quieter infrastructure change worth watching.

Cloudflare has released Agents SDK 0.20 with client and server support for the MCP 2026-07-28 release candidate. MCP, or Model Context Protocol, is a standard for connecting AI systems with tools and other resources. The new design supports stateless servers that don’t require a transport session or a Cloudflare Durable Object.

A Worker can expose tools, prompts, resources and elicitation through an ordinary server route. Updated clients first probe for the new protocol, then fall back to the older initialisation flow on the same connection. That compatibility behaviour should make a gradual transition possible.

For developers operating simple MCP endpoints, stateless handling can remove session infrastructure and make horizontal scaling easier. But applications that genuinely depend on session state still need a migration plan or a parallel legacy route. Cloudflare has feature-frozen the older McpAgent API without naming a precise removal version.

There’s also timing risk: the specification is still a release candidate. Adopting it now may reduce deployment complexity, but teams should preserve compatibility rather than assume the final protocol will remain unchanged.

What Changes for You

And if you build robots, one part of Google’s latest system is now within reach.

Google DeepMind has released Gemini Robotics-ER 2 through the Gemini API and Google AI Studio. It’s the reasoning and orchestration layer: according to Google, the model can process continuous video, use tools, plan multi-step tasks, monitor execution and revise the plan as conditions change.

That means robotics developers can now prototype higher-level behaviour through a public API instead of relying entirely on a closed demonstration or early-access programme. The practical shift is access to the layer that decides what a robot should do next and whether its plan is working.

The limitation is just as important. Google’s motor-control Gemini Robotics 2 and On-Device models remain restricted to early-access partners, while its Enterprise Agent Platform access is in private preview. Fine multi-finger manipulation and movement speed also remain difficult, by Google’s own account, and the company’s broader reliability claims haven’t been independently established across uncontrolled environments.

So the newly accessible part can make orchestration experiments easier, but it doesn’t remove the hard work of hardware integration, physical safety and dependable control. For working robotics developers, this is a useful bridge from demonstrations to prototypes—not a general solution for deploying autonomous machines.