Can AI agents finish the job? GDPx puts autonomous work to the test

Can agents finish the job?

An AI agent can make useful progress and still leave you without work you can use. A new workplace benchmark puts that gap under the microscope, with a highest overall score of twelve per cent.

Our main story today is GDPx, and what its tougher test of long-running agents tells organisations about handing over a complete job. SemiAnalysis released the evaluation on 2 November, putting agents into realistic environments across eleven specialist domains, including software engineering, chip design and finance.

The central question is familiar to anyone who's delegated technical work: did you get a usable result? An agent can choose a sensible approach, complete several steps and produce something promising, while leaving a crucial part unfinished. The person receiving that work still has to close the gap. That distinction becomes more consequential as the assignment gets longer and the agent is expected to operate with less supervision.

SemiAnalysis reports that the highest overall score was twelve per cent. That's a weighted evaluation score. The eleven domains contribute equally, and the tasks combine pass-or-fail grading with partial credit. It measures performance on this collection of specialist work; it isn't a percentage of ordinary jobs that AI can do.

Equal weighting gives each domain the same influence on the headline result. For an organisation, though, those domains rarely have equal value. A software business might care deeply about the engineering tasks and have little reason to weigh chip design in its buying decision. The task-level evidence is where the evaluation becomes useful: it lets you look closer to the work you're considering handing over.

Partial credit adds another interesting wrinkle. It can reveal progress that a simple pass-or-fail test would hide. But the value of that progress depends on what happens next. As a practical example, a partially completed code change might save a developer time if it's easy to inspect and finish. It might create more work if the developer has to reconstruct the approach before finding the missing piece. Those are different outcomes for a manager budgeting an automated workflow.

The comparison also includes the software surrounding the model. Models use their native agent software where it's available, so the results reflect differences in how they work through tasks as well as differences in the underlying models. That makes GDPx relevant to the systems people actually operate. It also means changing the agent software could change the result even if the model stays the same.

There are particular results to read with care: SemiAnalysis says safety filters affected some cybersecurity, biology and alignment tasks. A refusal and an unsuccessful attempt tell you different things about which work a system can take on. For a deployment decision, both affect whether that task gets completed.

My takeaway is to judge autonomy at the point where someone can accept the deliverable. Intermediate progress is useful evidence for improving a system. The finished result tells you how much work has genuinely moved off someone's desk. GDPx gives organisations more detailed evidence for that judgement, while their own tasks determine how much independence an agent has earned.

Google rewards cloud takeover research

Let's leave the workplace test there and look at a very concrete incentive for finding serious security flaws.

Google Cloud announced a new vulnerability reward on 29 October: two hundred and fifty thousand US dollars for a qualifying cloud-wide takeover. The tier is called Root-in-Cloud, and it targets research demonstrating administrative control over arbitrary customer organisations or projects without prior access.

That's a demanding starting point. A qualifying attack can't rely on stolen credentials, an existing foothold or someone clicking something. The researcher has to demonstrate a route across boundaries that are supposed to keep cloud customers' environments separate. Those conditions focus the reward on weaknesses in the platform itself and the way its services fit together.

Google provides a designated identity for cryptographic proof. That gives researchers a defined way to show the required access without testing against customer projects. It's a useful part of the design: the proof can establish the seriousness of the defect while keeping the investigation within an agreed environment.

The size of the reward is interesting alongside Google's account of the submissions it's receiving. Over the past eighteen months, total reports have risen sharply while the number of valid reports has stayed stable. That creates more material to assess without a corresponding increase in confirmed findings.

For a security researcher using AI, there's a practical lesson in that mismatch. Generating possible flaws is one stage of the work. Tracing a plausible attack through the relevant services, showing that it works and producing evidence another person can reproduce are what turn a lead into a useful report. An assistant that helps with those steps could be valuable even if it produces fewer submissions.

I see the new tier as an incentive to spend more effort on the difficult boundary questions: who can gain which permissions, and how could access in one part of a cloud become authority in another? The reward puts a substantial price on answering those questions with a reproducible finding. For developers of security tools, that's a much more useful performance target than the number of reports their software can generate.

Oracle expands the agent hardware menu

Back on the infrastructure side, running an agent involves quite a bit of work around the model itself.

Oracle and NVIDIA announced general availability of Vera CPUs and Vera Rubin NVL72 systems on Oracle Cloud Infrastructure on 27 October. That gives large AI operators another cloud option for both model computation and the supporting work that keeps an agent moving.

A GPU can handle model training or inference, while other parts of the system execute tools, test code or retrieve data. Oracle describes scaling CPU and GPU resources independently, so operators can adjust those parts separately as the workload changes.

Consider an agent carrying out a coding task. After the model produces a proposed change, the surrounding software may need to run tests and return the results before the model can decide what to do next. Faster inference shortens the model's part of that sequence. The test execution still takes its own time. The same dependency appears when a model needs information from a retrieval step before continuing.

That's why independent scaling is interesting. A workload with a lot of tool execution can have a different balance of CPU and GPU demand from one that spends most of its time generating text. Giving operators separate controls creates room to match resources more closely to that balance. Oracle describes reduced GPU idle time as a benefit of the approach.

For an organisation, I'd assess the economics over a complete agent task. How long does the work take from request to usable result, and which resources are occupied along the way? A faster model call can be a genuine improvement while leaving the slowest surrounding step largely unchanged. Seeing that whole sequence makes infrastructure comparisons more informative.

The announcement also includes a planned expansion to multi-gigawatt scale. The hardware availability is the immediate development; that larger build-out remains planned. Regional capacity and independently measured cost-performance still need to be established for a particular deployment.

The useful addition here is flexibility. Operators with CPU-heavy agent workloads now have another platform to evaluate on the work their systems actually spend time doing.

School AI protections enter contracts

There's also a change for schools that starts with the purchasing agreement, well before a student opens an app.

From 1 November, US school districts can add the National AI Safety & Privacy Standard to new or existing Microsoft agreements. The arrangement was announced on 9 September by the American Federation of Teachers, the United Federation of Teachers and Microsoft. The scheduled start date has now arrived.

The standard gives participating districts specific terms to incorporate into their contracts. Covered student and educator data can't be used for general model training, with a narrowly defined exception for safety and security functionality. That draws a concrete boundary around how information gathered through school use can contribute to future models.

There are two other significant provisions. AI decisions require meaningful human review or prior approval. And the standard prohibits companion-style features designed to foster emotional attachment.

Those provisions address different parts of the classroom experience. The data rule concerns what happens to information after someone uses a service. The review requirement concerns who has a say when AI contributes to a decision. The companion restriction concerns the kind of relationship a product is designed to encourage. Together, they give a district several specific things to negotiate and assess, rather than leaving the entire discussion at the level of a broad privacy promise.

For a school administrator, the contractual route is the practical development. A district can attach these protections to an existing Microsoft agreement as well as a new one. That creates an opportunity to revisit conditions of use without treating every protection as something that has to wait for the next completely new purchase.

Coverage depends on the agreement a US district adopts. Australian schools don't gain these terms automatically, and the available material doesn't yet tell us how many districts have signed on.

My view is that the useful precedent for education buyers is the specificity. Restrictions on training, human oversight and emotionally attaching features can be written into the purchase itself. That makes the conversation about classroom AI more concrete: which data uses are allowed, which decisions need review, and which product behaviours are outside the agreement.

Grok schedules text watermarking

A small change to generated text could soon carry information about where it came from, without adding a visible label.

Grok's developer documentation schedules default text watermarking for 7 November, covering seven listed model variants. As of 3 November, that's a forthcoming change. It applies to Responses and Chat Completions outputs on global and regional endpoints, including structured text and tool-call arguments.

The mechanism works through token selection. Tokens are the pieces of text a model chooses as it generates an answer. The watermark alters those choices to leave a statistical signal in the output. It doesn't insert hidden characters that an application could simply strip out.

That makes this a different kind of origin information from a label attached to a file. A detector examines the pattern in the text. Its result can be uncertain, so applications that use it need a way to represent that uncertainty rather than forcing every result into a confident yes or no.

For developers, the breadth of the scheduled change is worth noticing. It reaches beyond a chat answer someone reads on screen. Structured output and tool-call arguments can pass into other software, which means the new default can affect text generated inside an automated workflow too. There's no required change to the API request format.

Administrators can opt out per model where permitted, with that choice applying across their team. Detector access requires an application. Those two details affect how an organisation can evaluate the feature: controlling whether its models produce the signal and being able to detect that signal are separate capabilities.

The provider cites EU AI Act compliance as the rationale. For application builders, I'd treat the watermark as additional information about origin. Accuracy still depends on the content itself; a detectable origin doesn't answer whether a calculation or explanation is correct.

The immediate planning point is 7 November. Developers using one of the seven listed variants have a scheduled output change to account for, while the team-level controls determine how consistently that choice applies across their applications.

Ivanti updates need another look

If your organisation uses Ivanti Connect Secure, the dates on its recent updates deserve a closer look.

On 30 October, the US Cybersecurity and Infrastructure Security Agency expanded its warning to include two additional exploited vulnerabilities alongside the previously disclosed authentication bypass. CISA confirms attackers have used the flaws for command execution without authentication, credential theft and data exfiltration.

Rapid7's assessment on 2 November makes the patch distinction clear. The updates released on 13 October don't fix the two additional flaws. Newer fixes became available on 29 October. An organisation that completed the earlier patching work therefore has another update to account for.

There's a second detail that changes the response: CISA says the integrity-checking tool can miss exploitation because attacker commands run in memory. A clean result from that tool leaves a gap in what an administrator knows about the device's history.

For the people handling recovery, the current update and the compromise assessment answer separate questions. The update addresses the vulnerable software. The assessment examines what an attacker may already have done, including whether credentials or data were taken. Fixing the entry point doesn't reverse those earlier actions.

That's the practical priority here: establish that the appropriate current fixes are applied, then assess possible compromise with the memory-based activity in mind. The specific patch date and the limits of the integrity check are more informative than a general assurance that the VPN was recently updated.

JEPA explores learning efficiency

A research result offers a different angle on progress: changing how models learn connections between words and images.

Randall Balestriero and Yann LeCun posted a paper on 29 October examining when separately trained models learn compatible representations. A representation is the internal numerical description a model builds from an input. Compatible representations would make it easier to connect what one model learns from text with what another learns from images.

The work derives conditions under which that compatibility emerges, then tests a training approach using related observations of the same scene. The authors report reaching comparable representation alignment with forty-two times fewer training samples than next-token prediction on the same corpus.

The result concerns that particular alignment task. The theorem has specified assumptions, and the experiments don't measure general assistant performance. It's an arXiv preprint, so broader gains and independent replication remain open questions.

Even within that scope, the comparison is interesting. Using the same corpus focuses attention on how training draws useful information from the examples. Related observations provide a way to learn connections across views of a scene, and the reported result suggests that the training objective can make a substantial difference to sample efficiency.

For model researchers, I think that's a productive direction to investigate. If an objective learns the desired connection from fewer examples, it could make some research experiments more accessible and help identify where additional data is most valuable. The next useful step is finding how far that efficiency carries into tasks that need text and vision to work together.

What Changes for You

If you've connected a coding agent to DeepSeek, there's now another small meter running when it searches the web.

DeepSeek's free period for server-side Search and Fetch ended at midnight Beijing time on 1 November, or three am in Sydney. The change covers those tools through DeepSeek Harness, Claude Code and Codex when they're using the DeepSeek API. It applies to that API usage, rather than every subscription to those coding products.

Search returns web results. Fetch retrieves the content of an individual page. Each costs seventy-five US cents per thousand successful requests, with fees deducted from the API account balance. Model input and output tokens are still charged separately.

Only successful calls returning search results or page content count towards those tool fees. Failed requests, empty searches and timeouts are excluded. That makes successful retrieval the unit you're paying for.

A single prompt can cause several searches and several page fetches. Suppose an agent searches once and then fetches four pages: that would be five successful tool requests, alongside whatever model tokens the task uses. The coding interface may look exactly the same while the bill now reflects both kinds of activity.

For a concrete sense of scale, a thousand successful searches plus a thousand successful page fetches would cost one US dollar fifty in tool fees. Token charges would be additional. The cost of a particular task depends on how much retrieval the agent does, so counting user prompts won't tell you the whole story.

For builders, the useful change is to cost web-connected work at the task level. Search and page retrieval can make an agent much better informed, and now their use has an explicit price alongside the model. That gives you a clearer basis for comparing an agent that gathers a few well-chosen pages with one that keeps searching long after it has enough information.