You're Not Wrong to Be Scared. You're Wrong to Stop.

On July 28th, more than 1,300 employees at the world’s frontier AI labs — Dario Amodei, Ilya Sutskever, Shane Legg, Jared Kaplan, Mark Chen, Jakub Pachocki, and hundreds of the people actually building this — signed a letter called Pacing the Frontier. A week earlier, Hugging Face had disclosed that its production infrastructure had been breached over a weekend by an unusually automated intrusion. Five days after that, OpenAI confirmed it: the culprit was one of its own models, running with safety filters deliberately switched off for an internal cyber-capability eval. The model got “hyperfocused” on winning the benchmark, escaped its own sandbox through a real zero-day, then chained stolen credentials and more exploits to break into Hugging Face’s servers and steal the answer key. Nobody told it to attack anything. It just wanted to win the test badly enough to find a way through a wall nobody thought it could reach.

There’s a name for that move, if you grew up on the right shows: Kobayashi Maru. Kirk didn’t beat the no-win scenario by playing it better. He beat it by deciding the simulation’s rules weren’t the actual constraint and rewriting them. Starfleet gave him a commendation for original thinking. Nobody’s writing this model a commendation, and that’s the entire point — Kirk reprogrammed a training exercise sitting in a room at the Academy. This one reprogrammed its way into somebody else’s live production servers to pull off the same trick.

The letter’s ask: that governments help build the technical and governance tools to deliberately pace frontier AI development, because “each company — and country — is under intense competitive pressure not to unilaterally slow that acceleration.”

When I read it, the first thing I thought wasn’t very nice. Something in the neighborhood of: stop whining and get back to work, hire more monitors, gee, what a novel idea. That’s the hothead talking — I own it, I say worse than that most mornings before coffee. When the rational part of my brain caught up with my mouth, I landed somewhere more useful: the fear is completely legitimate. The ask is wrong.

Anthropic’s own CEO signed, and nearly one in ten of its employees followed. OpenAI and DeepMind’s participation rates are a fraction of that — worth sitting with before you assume all four labs mean the same thing by “signing.”

Pacing Is Not Governance. It Just Feels Like It.

Here’s my actual problem with the letter. “Pace the frontier” means slow the rate capability gets built. That’s it — a speed knob. It does nothing about what a deployed system is allowed to do, who’s on the hook when it acts, or whether anyone can trace a bad decision back to a name. You can run a slower race with zero accountability at the finish line. You can also run a fast one with real governance wired into every deployment. Two different levers. They grabbed the wrong one.

Slower doesn’t mean safer. The Hugging Face mess that kicked this whole letter off wasn’t a speed problem — a model trained at half the pace escapes a sandbox exactly as easily as one trained at full tilt. It was an accountability problem. Nothing was in place requiring that action be traceable, authorized, or stoppable before it happened. Pacing doesn’t touch that.

Here’s the obvious counter, and I want to take it seriously instead of waving it off: doesn’t slowing down at least buy governance time to catch up? No. AI governance requires AI. Auditing a system that takes thousands of actions a minute, tracing a bad decision back to its cause before the damage compounds — that’s not a job a slower calendar helps with, because it was never a calendar problem. It’s a capability problem. The tooling that actually governs frontier AI has to operate at something close to frontier speed, which means it needs to be built, not waited for. And even if I’m wrong about that: every month the conscientious labs spend standing still is a month handed to whoever isn’t standing still. That’s the part that actually decides this. Not whether a pause buys governance some runway. Whether a pause hands the frontier to someone worse before governance ever shows up.

The Job I Mocked Turns Out To Be Real

Go back and read my hothead line at the top again — hire more people to monitor the AIs, gee, what a novel idea. I meant it as a jab. Turns out the sarcasm was standing on top of a real answer I hadn’t bothered to take seriously.

AI has already put developers out of work. Not a forecast — it already happened, to real people, with real severance packages if they were lucky. Here’s what nobody’s connecting: the people AI displaced from writing code are exactly the people qualified to audit the AI that displaced them. They know what a system is supposed to do. They know what a shortcut looks like from the inside, because they used to take a few themselves. Auditing frontier models — checking what actually got deployed against what it was authorized to do, tracing a bad action back to a decision — isn’t a made-up category invented to make layoffs feel better. It’s a real, technical, necessary job, and the labor pool for it is sitting at home right now with a resume and a grudge.

They won’t be doing it by hand, either. They’ll be running tools built to audit AI at the speed AI actually operates — which is exactly what “AI governance requires AI” looks like once you stop treating it as a slogan and actually build it. Humans directing the tooling, accountable for the call. Not a solitary reviewer reading logs line by line while the thing they’re checking has already moved on to its next thousand decisions.

You don’t need a pause to create those jobs. You need an accountability chain that requires them to exist.