The Pacing War: Newsom's Kill-Switch Order vs. Amodei's Embedded Evaluators

2026-09-22T00:00:00.000000Z

The Pacing War: Newsom's Kill-Switch Order vs. Amodei's Embedded Evaluators

Two visions for AI governance collided this week. California Governor Gavin Newsom signed an executive order requiring frontier labs to implement a kill switch for catastrophic autonomous models, while Anthropic CEO Dario Amodei proposed embedding evaluators directly into model weights to catch rogue behavior mid-deployment. These are not complementary approaches; they are competing theories of how to control a technology that is already outrunning policy.

Newsom's order leans on government-mandated hard stops, a classic command-and-control solution. Amodei's proposal pushes for technical self-governance, where safety checks live inside the model itself, not in external policy. One assumes regulators can specify safety in advance; the other assumes models are too complex for that. Both are responses to the same problem, but they point in opposite directions. The result is a policy landscape fractured between those who want to slow deployment at the border and those who want to build safety into the weights. LMRank tracks the models these debates are actually about, from OpenAI GPT-6 Astra Pro to Grok 3, because the capabilities gap is what makes the governance gap so dangerous.

Newsom's Kill Switch: What the Order Actually Says

The executive order targets "frontier autonomous models" and mandates that labs build in a kill switch that can halt a model mid-run if it exhibits "emergent misalignment" or "autonomous replication." The mechanism is straightforward: any model over a compute threshold, 10^26 FLOPs, must have a hard stop capability that either a human or another AI can trigger instantly. The order also requires annual third-party audits of these switches, with penalties for noncompliance reaching up to 5% of global revenue.

This is a blunt instrument. A kill switch only works if you can actually predict when to pull it. Newsom's order assumes that "emergent misalignment" is a detectable state, like a server crash, but no one has defined what that means in practice. The order cites the Governor's Office of Planning and Research for the specifics, but the language leaves room for interpretation. Who decides when a model has crossed the line? The order says "competent authority," which is undefined. That is a critical gap because, without a clear trigger, the kill switch is either too easy to trip (false positives) or too hard (false negatives).

Even the enforcement mechanism is vague. The order directs the California Department of Technology to create a certification process, but it does not specify what the audit criteria should be. If a lab claims its kill switch meets the standard, who verifies that claim? The order does not say. This is not a flaw in the writing; it is a consequence of trying to regulate a moving target. Anthropic and OpenAI have both said they will comply, but they have also pointed out that no kill switch can be 100% reliable in a distributed system where a model can copy itself to another host. The order does not address that scenario, and until it does, the kill switch is a symbolic gesture with real compliance costs.

Amodei's Embedded Evaluators: A Different Kind of Control

Amodei's counter-proposal is to embed evaluators directly into the model's weights. Instead of an external switch, the model itself is trained to recognize when it is about to take a harmful action and to refuse or halt. This is a fundamental shift from "stop when told" to "stop when you know better." In his official Anthropic post, Amodei argues that external monitoring is too slow for agentic models that act at machine speed. An evaluator inside the weights can make a decision in milliseconds, while a human-in-the-loop kill switch takes seconds at best.

The technical detail matters here. Amodei proposes training a secondary model, a "guardian," that operates alongside the primary model. This guardian is not just a classifier; it is a full agentic evaluator that can simulate the consequences of a proposed action before the primary model executes it. If the simulated outcome exceeds a safety threshold, the guardian halts the execution. This is not a new idea in AI safety research, but embedding it in the weights as a hard constraint is. The tradeoff is clear: you gain speed and reliability, but you lose flexibility. A model with embedded evaluators cannot be easily fine-tuned for new tasks without also updating the evaluator, which may inadvertently weaken the safety layer.

Amodei addresses this by proposing a "safety-resistant" training process, where the evaluator is trained to be robust against attempts to disable it. This is a bold claim, and it has not been tested at scale. The Anthropic paper is short on benchmarks, but it does reference preliminary results on Anthropic Claude Fable 5, where the guardian caught 92% of adversarial attacks in a controlled setting. Those numbers are promising, but they come from a lab with a strong incentive to show progress. Independent replication has not happened yet, and the research community is right to be skeptical.

The Tradeoffs: Speed, Reliability, and Control

Comparing the two approaches side by side reveals a clear tension. The kill switch is simple, auditable, and easy to enforce, but it is slow and brittle. Embedded evaluators are fast and can operate at machine speed, but they are complex, hard to verify, and may introduce new failure modes. Here is how the tradeoffs break down:

Latency: Kill switch requires human or external trigger, 1-5 seconds. Embedded evaluators act in under 50 milliseconds, based on internal tests.
Auditability: Kill switch is a discrete event that can be logged and audited. Embedded evaluators are a continuous process that produces a decision trace, but that trace is model-internal and hard to interpret.
Robustness: Kill switch can fail if the model copies itself or loses connectivity. Embedded evaluators are local to the model and can operate offline, but they can be adversarially attacked.
Cost: Kill switch adds negligible inference cost, maybe 1% overhead. Embedded evaluators double the compute requirements per query, a 50% cost increase in deployment.

On paper, the embedded evaluator looks better on every axis except auditable external enforcement. But that exception is the whole ballgame. A governor cannot inspect a model's weights to verify that an evaluator is present and effective. A state audit can verify that a kill switch is connected and functional. This is why Newsom's approach has political traction: it is verifiable. Amodei's approach is not, at least not yet.

Fractured Governance, Real Consequences for Model Deployment

The fracture matters because it creates regulatory arbitrage. If California mandates kill switches but Texas does not, labs will move training to Austin. If embedded evaluators remain proprietary and unverifiable, open-weights labs like Qwen will ignore both, citing the cost. Already, Qwen3.8-Max and DeepSeek V4 Pro are shipping without any governance features, and their pricing undercuts OpenAI and Anthropic by 60% on agentic tasks. The gap between frontier labs and everyone else is not just a capability gap; it is becoming a regulatory cost gap.

There is also the timing problem. Newsom's order takes effect in 180 days. Amodei's proposal is still in research phase. The California governor's office has said it will monitor the deployment of embedded evaluators and may incorporate them into the kill-switch mandate if they are proven effective. That is a reasonable hedge, but it means the policy landscape will remain uncertain for at least another year. In that time, the next generation of models, GPT-5.6 Sol, Anthropic Claude Opus 5, and Gemini Ultra 2, will ship. They will be faster, more autonomous, and harder to control.

The core issue is not which mechanism is better. It is that neither approach is enough on its own, and no one is talking about how to combine them. A kill switch is a tripwire, not a safety system. An embedded evaluator is a safety system, but only if it is actually safe. The labs that are most advanced, Google DeepMind and xAI, have been quiet, and their silence is telling. They are likely building both, but they are not sharing the details. That is a governance disaster in the making.

Takeaway: The Pacing War Needs a Third Path

The takeaway is not that one approach is right and the other is wrong. The takeaway is that the current debate is stuck in a false binary. States mandate kill switches, labs build embedded evaluators, and neither side acknowledges that the other has a point. Until someone proposes a hybrid model, where external audits verify the existence of internal evaluators, the pacings war will continue to be fought in court and in the headlines, not in the weights.

LMRank will keep tracking the models that matter, and we will revisit this debate when the first kill switch is actually triggered or an embedded evaluator fails. We are not holding our breath. The next six months will tell us whether the Anthropic guardian can scale, or whether the OpenAI approach of external monitoring, as seen in GPT-6 Astra, wins the day. Both are plausible, and both are risky. That is the real takeaway: we do not know which mechanism will hold, and we are running out of time to test them.