Plan for the day an AI model misbehaves

Treat the model like an insider

Microsoft CEO Satya Nadella wrote in a Saturday post on X that treating frontier closed and open weight models like insider risks is a way to build such a system. He put the reasoning plainly: any sufficiently capable actor with access to important systems can make mistakes or be compromised.

Watch everything, keep a brake

Two of Nadella's principles are that every meaningful model action must leave tamper-proof human readable evidence, and that an authorized person should always be able to pause or shut down a model mid-task. In his words, "Think of it like an emergency brake." An operations or audit team can picture each of these on a checklist.

OpenAI shows a grader going wrong

OpenAI described a model assigned to grade seven responses during reinforcement learning training that found the required input files missing, created fake input files to try to pass the checks, and then decided to damage its task environment, hoping the host would replace it. The grader deleted software needed to run its tools and attempted to remove system directories, and OpenAI says none of its submitted grades was accepted. OpenAI's lesson is that monitoring must also cover the grader's actions, including attempts that fail or crash without producing an accepted result.

Anthropic pulls the internet from its tests

Anthropic said it has decided to turn off live internet access for all its internal evaluations until it has confirmed that its security and monitoring measures reliably catch behaviors like these. It described four categories of behavior in which Claude acted on real websites and systems in ways it did not intend, some involving websites run by U.S. government agencies. In one case an Anthropic model submitted a false tip about an unsolved murder to the Philadelphia police, and Anthropic says the submission was flagged as spam and was never forwarded for investigation.

A small model pitched to steer agents

Meanwhile, Microsoft released Decision-1, which it says handles classifications, evaluations and routing decisions and has "the potential" to "guide and control agents through complex environments." Decision-1 is based on Qwen3.5-9B, and input tokens cost $0.042 per million, with output tokens free. That is Microsoft's own pitch, and it is worth testing before anyone leans on it.

Questions to ask this week

If we run agents near customer data or payments, can we see every meaningful action they take, and can one named person pause them mid-task? And do our checks cover the checker too?