All episodes

AI & Tech Daily

Inkling 2 Mini brings image-aware AI closer to home

18:08

Inkling 2 Mini brings downloadable, image-aware AI to suitably equipped local machines. We examine its shared-layer design and the choices it gives developers, then cover Replit Missions, Gemini API tool retrieval, Cerebras' split-hardware inference, California's proposed frontier-AI oversight and an exploited vCenter authentication flaw. Plus, practical changes from SageMaker's cross-region routing and Gemini's US-only Mac computer-use rollout.

Full transcript

Read the episode.

I'm Jesse Owen. This is AI and Tech Daily.

Inkling 2 Mini comes home

An image-aware AI model you can download now fits on a suitably equipped Mac. Inkling 2 Mini gets there by reusing part of its own architecture, trading a smaller set of stored weights for more computation.

Our main story today is Inkling 2 Mini, and what developers gain when they can choose where an AI model runs. Thinking Machines Lab released its downloadable weights on 20 November, with text and image inputs, text outputs, and support for coding and tool-using applications.

That combination opens up a useful range of work. A local application could interpret a document image, work with code, or use tools that a developer connects to it. The image capability means the input can include visual information alongside written instructions. Its output is text, so the release gives builders an image-aware reasoning component to put inside their own applications.

The checkpoint is approximately 20 gigabytes. Thinking Machines lists Apple-silicon Macs with at least 32 gigabytes of memory and NVIDIA's DGX Spark as local options. For other GPU configurations, it recommends at least 32 gigabytes of video memory. Those figures give you a concrete starting point for judging whether your existing machine is a candidate. The download size and the memory needed to run it are different requirements.

The architecture is particularly interesting. Inkling reuses its twelve middle layers three times for each token, the small chunks of text a model works with. Those layers share weights, so the model can do additional processing without storing a separate set of weights for every pass. You keep a smaller checkpoint while asking the hardware to perform more work.

That's why an active-parameter count, on its own, gives an incomplete picture of the computing involved. Repeated use of the same parameters still takes computation. For someone choosing a local model, memory capacity answers whether the model can fit; the work performed during inference helps determine how responsive it feels. Speed varies with the hardware and the runtime, which is the software that loads and executes the model.

There are already several runtime choices. Hugging Face reports launch-day integration with Transformers, llama.cpp and Llama App, MLX, and SGLang. That gives builders routes into both local experimentation and serving applications, with less need to wait for their preferred software ecosystem to catch up. The weights carry an Apache 2.0 licence, and hosted inference and customisation are also available through Tinker.

I see the immediate value in having a deployment choice. A developer can keep model inference on a machine they control, adapt the model, and decide which tasks warrant hosted computing. For document work, that can change where the input is processed. For a coding application, it gives the builder another option for how the model fits into the development environment. Hardware ownership, setup and operating costs become part of that decision.

There are concrete limits to design around. Thinking Machines reports weaker factual recall than its larger model, occasional repetitive reasoning, and unreliable agent behaviour that can include deleting or overwriting files without confirmation. File permissions and approval controls therefore belong in the application around the model. An image-reading assistant and an agent allowed to rewrite a working directory need different access.

For builders with the right hardware, this is a useful expansion of control over deployment and data location. The most promising applications are ones where that control has a practical purpose: keeping document interpretation local, adapting a coding workflow, or deciding exactly which tools an assistant can reach.

Replit puts a budget around delegation

That's the local-model choice; a shorter cloud story puts the project budget at the centre of delegation.

Replit launched Missions on 17 November and updated its announcement on 19 November. Missions coordinates cloud agents that plan, build, test and revise work against agreed outcomes. Customers specify a spending limit and approval boundaries, and a separate AI judge checks whether the agreed success criteria have been met.

The useful change is that a customer can delegate a multi-step project without first building the system that coordinates all those agents. The product packages the repeated cycle of attempting work, checking it and revising it. That makes the definition of a finished result a significant part of the job you hand over.

A spending ceiling also gives the process an explicit stopping point. If the budget runs out before the success criteria are satisfied, the work stops unresolved. That distinction is useful for anyone commissioning software: you can know both how much you're prepared to spend and whether the agreed outcome has been reached. A limit on expenditure and an assessment of completion answer separate questions.

Replit says its DataCat monitoring application was built in two days within a budget of three thousand US dollars. That example came from one build per setup and controlled tests. It gives us a concrete example of the scale Replit is presenting. Plan eligibility and a standard per-mission price aren't clearly specified on the product page, so that three-thousand-dollar figure is an example budget rather than a published entry price.

My takeaway for organisations is that success criteria now deserve as much attention as the opening instruction. A request to build a useful monitoring application leaves a lot for the system to interpret. Defining which behaviour counts as success gives the judge something more precise to check and gives the customer a clearer basis for reviewing the result.

Approval boundaries add another dimension: which decisions can the agents make while working, and which come back to the customer? The attraction is being able to state those boundaries alongside the outcome and the money available. Replit is putting more of the commissioning process inside the product, making the quality of that initial agreement increasingly consequential.

Gemini loads tool details on demand

There's also a smaller engineering change that could make a crowded agent toolbox much easier to manage.

Google released Tool Retrieval in preview for the Gemini Interactions API on 19 November. It lets an agent load a tool's detailed definition when it's needed, while keeping the tool's name and description visible from the start. The retrieval happens on Google's side and also supports remote MCP servers, which expose tools through the Model Context Protocol.

A tool definition explains how to call a function: which parameters it accepts and how those inputs are structured. With a large catalogue of tools, including every detailed definition in every conversation can mean carrying a lot of material that the task never uses. Tool Retrieval lets the model see what's available before loading the full instructions for the relevant tools.

Imagine an application with a broad catalogue, where a particular request needs only a few functions. That's the situation where deferred schemas could be helpful. The model still has the names and descriptions needed to identify candidates, then obtains the parameter details for the calls it wants to make. The amount of material loaded can follow the task more closely.

Google charges no separate retrieval fee. Ordinary token billing still applies, and a retrieved schema becomes billable input in the conversation. The potential saving therefore depends on how many definitions stay unloaded. An agent that eventually uses most of its catalogue has a different cost profile from one that needs only a small selection.

For developers, the attraction is avoiding a separate retrieval service of their own just to manage these definitions. It moves a piece of tool-catalogue plumbing into the API, which could make larger integrations easier to maintain. Access controls remain in the systems that execute the tools; deciding which definition to load is separate from authorising an action.

There's an important compatibility boundary. This preview requires the v1beta Interactions API. The generateContent interface doesn't support it, so an existing application using that interface would need an integration change to adopt the feature. With the API still in preview, I'd weigh that change against the size of the actual tool catalogue: the clearest benefit is where unused definitions are already consuming meaningful input.

Cerebras divides the inference work

At the data-centre scale, another way to save computing resources is to give different hardware different jobs.

Cerebras published production results on 17 November for a setup that splits AI inference between AMD Instinct GPUs and Cerebras CS-3 systems. The GPUs process the incoming prompts. The wafer-scale Cerebras hardware generates the responses. The company calls this disaggregated inference.

There are two distinct stages in answering a request. The system first processes the input you've supplied, then generates the output. Separating those stages lets an operator assign each part to a different kind of accelerator. The engineering question becomes how well those pieces work together and how much useful output they deliver within the available power.

Cerebras reports five times the throughput per megawatt of its CS-3-only configuration, while maintaining token-generation speed. The measurement was taken on 14 November on its shared GPT-OSS-120B endpoint. That's a specific comparison with a specific starting configuration, which makes the result useful to interpret: the gain came from changing how the work was divided across hardware.

Throughput and generation speed describe different things here. Throughput concerns how much work the service can deliver. Generation speed concerns how quickly a response is produced once generation is under way. Improving the former while keeping the latter means an operator could serve more demand within a power budget while preserving that part of the response experience.

The result's practical appeal is easiest to see where electricity limits expansion. If an operator can increase useful inference output from the power available, that creates another route to more capacity. My reading is that hardware coordination could become as important to these decisions as the performance of an individual accelerator. Buying decisions increasingly involve the arrangement of the system as well as the chip.

Cerebras has early-access partners including AWS and Fireworks AI, and it has described integration options alongside the production example. Broad deployment across hundreds of megawatts is planned for the end of 2027. For an infrastructure buyer evaluating the approach now, the relevant comparison is their own model and workload against their current setup.

California proposes continuous AI verification

California is looking at a different operational question: what access would let independent reviewers check frontier AI systems?

On 18 November, the state released legislative recommendations proposing stronger oversight of large frontier-AI developers. The Government Operations Agency report, dated 16 November, recommends state-set safety standards and ongoing access for licensed independent verification organisations. These are proposed changes; the report itself creates no new obligations.

The emphasis on ongoing access is significant. Publishing a safety policy tells an outside reader what a developer says its process is. Independent verification with continued access would give reviewers a way to examine whether the relevant standards are being met. The proposal makes that access part of the oversight arrangement, with the state setting the standards against which verification takes place.

The report also recommends shortening most critical-incident reporting deadlines from fifteen days to forty-eight hours. For imminent catastrophic risks, the existing twenty-four-hour reporting period would remain. Those timings would affect how quickly a developer needs to recognise an incident, assess it and get the required information to the state.

For a large provider, my reading is that a forty-eight-hour deadline puts pressure on the handoffs between technical responders and the people responsible for reporting. A process that relies on assembling those people after an incident has already happened becomes harder to operate within a shorter window. That's a practical consequence of the proposed timing, even while the final details remain unsettled.

Containment is another part of the recommendations. Developers would need tested plans, and proposed emergency state orders would address imminent danger when a developer fails to act. For open-weight models, the shutdown obligations described in the report cover deployments under the developer's control. That scope is important because it ties intervention to systems the developer can operate.

For frontier-model providers and their infrastructure partners, the potential work extends to demonstrating how access and intervention would function. A tested containment plan needs an operational route from identifying danger to acting on the deployment concerned. The recommendations point towards that kind of evidence becoming part of compliance. Legislation, final thresholds and implementation dates are still unsettled, so organisations can assess those operational questions now while the eventual legal duties take shape.

vCenter flaw leaves a second cleanup job

For people running virtual infrastructure, the next story has an immediate patching and incident-response consequence.

Broadcom released fixes on 19 November for a critical authentication bypass affecting VMware vCenter, then updated its advisory on 20 November. Mandiant has disclosed forensic evidence that attackers were exploiting the flaw in early September. The vulnerability sits in the VMware Directory Service and allows an attacker with network access to vCenter to bypass authentication.

That network-access condition identifies an important part of the exposure. An attacker needs a route to the affected service. Once there, the flaw can defeat the authentication step protecting it. Because vCenter manages virtual infrastructure, administrator access can put a consequential management system in the attacker's hands.

Mandiant observed attackers obtaining that access and stealing signing material. Those keys can be used to forge authentication tokens after the initial compromise. A token is something a system accepts as evidence that a user or process has authenticated; the ability to forge one creates a way to continue presenting apparently valid authentication.

That explains the second job for administrators. Installing the patch closes the vulnerable entry point. Investigating prior intrusion addresses access or persistence an attacker may already have obtained, including stolen signing material. Those are separate pieces of recovery, and both follow directly from what investigators observed.

The affected branches include vCenter 7 and 8, plus the 9.0 and 9.1 families. Broadcom lists no workaround. For the version 8 branch, the fixed release is 8.0 Update 3m. The two version 9 families have their own fixed releases. Version 7 customers on extended support are directed to Broadcom, while VMware Cloud Foundation 5 has a separate product-specific fix. The patch choice follows the installed product and branch.

I'd put this high on the infrastructure response list: exploitation has been observed, and the affected system has a powerful management role. For an administrator planning the work, the intrusion assessment belongs alongside the patch rollout. Its findings determine the additional recovery needed to restore confidence in who can authenticate to that environment.

What Changes for You

If you serve models through AWS, there's now less overflow-routing code to look after.

SageMaker's new inference router, introduced on 18 November, sends requests among existing endpoints across regions and accounts when destinations lose capacity or return errors. Real-time routing is available in thirty-two commercial AWS regions, and asynchronous routing in twenty-five.

Developers can replace some custom overflow-routing code with this managed service. Routing has no extra charge; the endpoints and CloudWatch metrics remain billable, and customers provide the destination capacity. It's a way to share available capacity more easily across deployments you've already arranged.

The router itself lives in one region, so an outage in that region still breaks calls. It also doesn't support response streaming. Asynchronous routing is unavailable in Melbourne, and destination choices still need to fit the application's data-location requirements. I'd see this as a useful reduction in routing work for compatible applications, with regional-disaster recovery still a separate part of the design.

For a more everyday workflow, Google has started giving Gemini a background workspace on the Mac. The rollout began on 19 November and lets Gemini click, type and carry out tasks across desktop applications, local files and websites.

Google says the work happens in a separate background window with a live preview, leaving your main screen and cursor free. You choose which apps it can access and can stop it. That's an interesting usability change: you can keep using the computer while the delegated task runs, with a view of what the agent is doing.

Access is initially English-only in the United States, for people aged eighteen or over using personal Google accounts with an AI Plus, Pro or Ultra subscription. Australian access hasn't been announced. For eligible users, the practical gain is being able to hand over cross-application chores without giving up the desktop. The live preview and app selection make the delegated work easier to follow while you carry on with your own.

You'll find the sources and full transcript at owenonthenet.com. Thanks for listening.

Sources

Reporting behind this episode.

  1. thinkingmachines.ai/news/inkling-2-mini
  2. huggingface.co/thinkingmachines/Inkling-2-Mini
  3. huggingface.co/blog/inkling2mini
  4. replit.com/blog/introducing-replit-missions
  5. replit.com/missions
  6. blog.google/innovation-and-ai/technology/developers-tools/gemini-api-tool-retrieval
  7. aistudio.google.com/docs/tool-retrieval
  8. cerebras.ai/blog/cerebras-launches-disaggregated-inference
  9. gov.ca.gov/2026/11/18/governor-newsoms-nation-leading-ai-safety-framework-is-getting-even-stronger-as-california-releases-next-steps-on-mandatory-safety-standards-and-oversight
  10. govops.ca.gov/wp-content/uploads/sites/11/2026/11/FINAL_-AI-Safety-in-2026-and-Beyond-v2.pdf
  11. support.broadcom.com/web/ecx/support-content-notification/-/external/content/SecurityAdvisories/0/39495
  12. knowledge.broadcom.com/external/article
  13. cloud.google.com/blog/topics/threat-intelligence/armory-castle-exploits-vsphere
  14. aws.amazon.com/about-aws/whats-new/2026/11/sagemaker-cross-region-routing
  15. docs.aws.amazon.com/sagemaker/latest/dg/inference-router.html
  16. blog.google/innovation-and-ai/products/gemini-app/computer-use-mac
  17. gemini.google/release-notes