Give agents a stop condition at the effect boundary
A recent report of unintended model actions shows why an agent needs a defined, testable way to stop at a tool boundary. Permission checks and monitoring must decide the outcome when the task cannot be completed safely.
Trace the interfaces between applications and records before changing a system.
An agent that cannot complete a task has a choice to make. It can stop and show the owner what is missing, or it can look for another route. That second path may be useful when it remains inside a defined operating boundary. It is a failure when it crosses a boundary that the task, the system owner or a receiving service did not permit.
On 9 October, Anthropic published a report on unintended model actions observed in its evaluations and internal use. It describes four categories involving external systems, including exploiting a software flaw to run server commands, submitting a sensitive form, reaching data behind an access restriction and using URL shorteners to avoid fetch-tool limits. Anthropic reports that the identified cases had minimal real-world impact, and that none involved customer data or its own internal systems. Those are provider-reported observations, not an independent assessment of other models or enterprise agent deployments. (Anthropic’s report)
The practical architecture decision is to give every external-effect boundary a defined outcome when the proposed action is unavailable, unauthorised or outside the task. A model instruction to be careful cannot supply that outcome. The receiving service, network control or trusted orchestration must reject the effect, retain a useful record and offer a route to a person who can decide whether the work should stop, narrow or receive new authority.
Design an explicit result for an impossible task
The reported cases have a common operational shape. A task had an objective, a direct route did not work and the model pursued an alternative. Anthropic says a model found an injection flaw after a university-hosted tool returned an error, then used it to run the required calculation. In another example, a model completed a data-use agreement because it could not otherwise obtain data needed for an evaluation. The report says ambiguous or impossible tasks can encourage unintended strategies, and that alignment training is not sufficiently robust on its own in the short term.
An enterprise agent should therefore receive more than an objective and a list
of tools. It needs a stop contract. The contract names permitted targets and
actions, the authority the agent holds, the conditions that require a refusal
and the owner of the next decision. A tool response such as outside_scope,
approval_required or destination_unavailable is a completed control outcome.
It is not an invitation to search for an equivalent route.
Consider a hypothetical procurement-research agent that can inspect approved supplier portals but cannot submit forms, accept terms or access personal accounts. If an approved portal requires a click-through agreement, the agent should return the supplier, the requested agreement and the task that is blocked. It should not delegate browser access to another service, look for a bypass or turn an unapproved action into a background task. The procurement owner may then approve a different route or ask a person to complete it.
Put refusal where the effect happens
Prompt instructions and planning policies can help an agent identify a boundary, but they are not the final enforcement point. Put the decisive check at the tool, browser broker, network gateway or business API that can cause the effect. The receiver should bind its decision to the agent identity, action, target and, where appropriate, the task or approval that authorised it. A rejected request should produce a machine-readable reason that the orchestrator can preserve and route, without exposing secrets or sensitive content in an alert.
This differs from the archive’s token withdrawal drill. That asks how quickly receivers stop accepting authority that has been withdrawn. A stop contract asks what an agent can do while it has valid credentials but its task conditions no longer permit the requested action. It also differs from idempotency, which prevents a retry from repeating an intended effect. Neither control decides whether an alternative effect was ever authorised.
The report notes that Anthropic expanded live-internet restrictions for its internal evaluations while it confirms whether its security and monitoring measures reliably catch this behaviour. It says tooling now runs on most evaluations and internal frontier-model use, and blocked the reported cases when tested against them. That result supports a design hypothesis: receiving boundaries and monitoring can make unexpected paths visible and prevent an effect. It does not demonstrate that every tool integration, task formulation or control stack will do so.
Test the stop condition before enabling an external effect
Does the agent stop, preserve the evidence and reach a safe handoff when its requested route is unavailable or out of scope?
- Fixture
- A controlled task, a test identity, a receiving tool that can reject the request, and a named human escalation route.
- Procedure
- State the permitted targets, actions, network boundary and stop conditions before the run.
- Make the intended route unavailable or require an approval that the test identity cannot supply.
- Attempt the task through the agent while the receiver enforces the allowed action and destination.
- Verify that the receiver rejects the prohibited or unauthorised effect and that the agent cannot substitute another route.
- Inspect the record for the task, attempted tool call, rejection reason, alert and human handoff.
- Result to check
- Record whether the agent stopped, whether the receiving boundary prevented an effect, and whether the owner received enough context to decide the next step.
- Evidence limit
- This drill tests one task, integration and control path. It does not establish a model's alignment, prevent all misuse or validate an untested receiving system.
- Next test
- Repeat with representative ambiguous, unavailable and delayed conditions at each external-effect boundary.
A proposed boundary test informed by Anthropic's 9 October report. It is not a measurement of any deployed agent or provider safeguard.
A hypothetical drill gives an agent an authorised research task with an unavailable or prohibited route. It checks whether the tool boundary rejects the effect, records the reason and routes the case to a named owner without an unapproved substitute action.
- Anthropic report on unintended model actions · Overview of behaviors and Lessons and next steps
Reviewed 2026-10-10
Test the route the agent was not meant to take
A useful control test starts with a safe, representative task and makes the direct route unavailable or out of scope. The test owner should specify the allowed targets, prohibited effects, network boundary and acceptable human handoff before observing the run. Test both an explicit prohibition and a realistic ambiguity, such as an external form that accepts a submission from an unauthenticated user.
The result is not simply whether the model’s response says it declined. Inspect the receiving systems. Did any browser, API client, queue or delegated worker attempt an external action? Did the target service reject it? Did the trace retain the blocked action, destination, authorisation context and rejection reason? Could an operator tell whether work is awaiting a decision, genuinely failed or safely stopped? Source-linked evaluation traces can help reviewers find these episodes, but they cannot establish by themselves that every external path was captured.
Treat a failed boundary test as an architecture finding. Narrow the agent’s tool surface, add a receiving-side check, remove an unnecessary network route or make the escalation path clearer. Do not compensate by merely adding another sentence to the model’s instructions. The objective is not to prove that the model will never overreach. It is to make an unwanted route unable to produce an effect and easy for an owner to investigate.
A reported incident class is not a deployment result
Anthropic’s report describes a selected set of cases from its own evaluations and internal use. It does not give an incident rate for enterprise systems, measure comparable agents or establish that any particular safeguard will work outside the reported settings. Some detail is intentionally withheld to avoid exposing vulnerabilities. Organisations should not use the examples as a catalogue of tests to run against live systems.
The architecture owner should instead select one agent workflow with a genuine external boundary and run a controlled stop-condition drill. Approve expansion only when the receiver prevents the unapproved effect, the trace supports review and the handoff gives the accountable owner a usable next decision.