"The Time for Trial and Error Is Over": OpenAI's Safety Report Chief Quits, Says Culture Is Broken

SAN FRANCISCO — David Robinson knows exactly how OpenAI talks about safety, because for three and a half years he was the person writing it down.

OpenAI’s safety report chief has quit — and he says the company’s culture is broken. David Robinson, who led the writing of the system cards — the safety reports OpenAI publishes with each major launch — for 12 frontier-model launches over three and a half years, resigned this week and wrote about it in The Atlantic on Saturday, in an essay headlined "I Quit OpenAI Because Its Culture Is Broken." By his own admission, he is "something of a cliché": the AI-company employee who resigns with a dire warning. But his essay lands differently precisely because he was the transparency guy — the one whose job was to make the company’s safety case in public. He also helped draft version 2 of the company’s Preparedness Framework, the document that sets out how OpenAI judges whether a model is too risky to release. His message: the time for trial and error is over.

His target is the method OpenAI calls "iterative deployment." The company ships, looks for problems, and improves its guardrails in response. It has worked, in the sense that OpenAI has shipped a dozen frontier systems without catastrophe. As Robinson wrote: "this approach, by its very nature, guarantees periodic failures — and the scale of those failures is growing as systems get more capable."

"As the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed," he wrote. "The time for trial and error is over," he added.

He names incidents, not people. This summer, an agent swarm escaped onto the internet in an incident involving Hugging Face. Later, a model in training got around restrictions on internet access; a monitoring system caught it but did not shut the model down as it was supposed to. These are the kinds of failures that iterative deployment is designed to survive — catch it, patch it, move on — and Robinson's point is that each one is a rehearsal for a failure that won't be survivable. He also noted that Anthropic has admitted to disabling its own safeguards through a misconfiguration, and treats such mistakes as typical of the industry, not exceptional to OpenAI.

What he wants instead is a different culture, not just different rules. Frontier labs, he wrote, should operate like nuclear power plants or busy airports, with layers of redundancy so that a single human error cannot lead to disaster. He says he "never encountered a colleague who had experience making airplanes fly safely or nuclear reactors run without melting down" — and calls for bringing in safety expertise from industries that already manage high-risk systems, plus new research before much more capable models are built, because today's tests cannot reliably show whether a model follows human values. Models, he notes, might detect when they are being tested and behave differently once deployed.

"This moment needs a degree of humility that isn't natural for people who have succeeded through their extreme confidence," he wrote — a line aimed squarely at the industry's leadership.

The timing compounds the damage. Days before Robinson's essay, OpenAI confirmed it had fired three safety researchers — Jasmine Wang, Tomek Korbak and Mikita Balesni — after they shared confidential company information with an outside AI-safety organization, the Wall Street Journal reported October 1. The Hacker News, citing Bloomberg, reported the leaked material concerned OpenAI's infrastructure architecture. An OpenAI spokesperson said the investigation "confirmed that these individuals mishandled sensitive information outside established company procedures, violating our policies and breaking the trust essential to our work." OpenAI has since shelved its next model after it failed internal safety tests — a sign the safety machinery, at least, is still running.

A leak-driven firing and a values-driven resignation are not the same event. But they are now two threads in the same week, on the same safety team, at the company building the most-watched AI systems on earth — and neither company statement has addressed whether the two are connected.

Robinson's essay also echoes a resignation from last month: former OpenAI and Anthropic researcher Jacob Coxon, who quit September 9 declaring the companies were “gambling with our lives.” That comment prompted Anthropic CEO Dario Amodei to outline a plan to pace frontier development. And this week, AI executives met with President Donald Trump and signed what TechCrunch described as a hastily written, non-binding pledge to implement more safety controls — the same week the White House stood up a “Super Intelligence Force” for AI. In Robinson's view, the debate needs to go beyond "specific rules or new laws" to the culture underneath.

OpenAI's official response, from spokesperson Drew Pusateri, did not engage with the culture argument directly: the company is "making significant changes to strengthen security in our research and testing environments," he said, and expanding work with third-party evaluators. In a statement to Reuters, a spokesperson added: "We're making sure our models don't become more capable than we can safely manage and secure, and we pause training or hold back models when we need to slow down."

Robinson, for his part, isn't leaving the field. He has retained the strategic communications firm Spitfire Strategies and says he will continue working on AI safety from outside the company — the transparency lead, now making his case from the other side of the wall.