Articles
Hold the model constant when evaluating geospatial agents A new Google study separates specialist geospatial tools from the orchestration layer around them. Buyers should compare both against expert-defined workflows and full operating cost before funding an agent as a finished product. Use repository rules and a hold-out benchmark to review agent patches A change can pass the public test suite and still break a repository-specific compatibility contract. Google’s ADK case shows how maintainers can combine repository rules, a workload withheld from the agent and code review to decide which performance proposals are worth shipping. Check the plan around an agent’s repair Diffusion models can replace a selected section of a structured agent plan while conditioning on the steps around it. The operational test is whether those retained steps still fit the changed task state. Vary the practice world while preserving its rules AgentGarten separates an executable simulation from the AI-generated view an agent sees. That opens a route to more varied practice environments, provided the visual interface preserves the facts the task depends on. Make model routing earn its own cost Cheaper small models change the economics of deciding which model to call. Compare direct calls, upfront routing and sequential escalation, including the cost of selecting and checking each answer. Give agents a stop condition at the effect boundary A recent report of unintended model actions shows why an agent needs a defined, testable way to stop at a tool boundary. Permission checks and monitoring must decide the outcome when the task cannot be completed safely. Trace an agent evaluation back to the run The UK AI Security Institute’s open-source Transect makes long agent evaluations easier to inspect. Its source-linked labels can help reviewers find episodes worth checking, but do not validate the labels or the final score. Set the boundary for multimodal search EmbeddingGemma 2 brings text, images, video and audio into one local search space. The architecture decision is which content and derived vectors may stay on a device, and which may enter a shared index. Plan the evaluation route before healthcare AI scales The Government has accepted all 44 recommendations on artificial intelligence (AI) in healthcare, but much of the implementation detail is still forthcoming. Providers can prepare by deciding how local evaluations will be classified, governed and connected to clinical safety evidence. Design a content record around the watermark signal OpenAI’s textGrain adds a detectable statistical pattern to selected model output. Publishing systems should treat a positive result as one content signal and retain source, editing and approval history separately. Give long-running agents a working account of the task New research on explicit belief states gives agents a way to track what remains unknown and recognise stalled progress. The opportunity is a more useful investigation workflow, with a measurable cost for maintaining that state. Choose analysis work with an independent check BootLoops gives AI agents a toolkit for exact and checkable quantitative work. Its useful design lesson is to choose analyses where an independent computation can verify the result, while a domain expert still decides whether the question matters.
Bring us the question
Something here already on your desk?
Bring the question, the systems involved and the decision you need to make. We will confirm whether a scoped engagement is appropriate.