The Job Nobody Keeps: Why AI Safety Chiefs Keep Walking

Someone takes the top safety or ethics job at a frontier AI lab. Within a year, they’re gone. Once is a personality clash. Twice is bad luck. When it happens over and over across multiple companies for years, you stop blaming the people and start looking at the chair they sat in.

A note on sourcing before we get into it: I went looking for the last 30 days of community discussion on this and came up nearly empty. No substantial Hacker News threads, no Reddit debate worth citing. So rather than pinning this to one specific recent departure, I’m writing about the pattern that’s been building since 2023. For the details of who left when, official announcements are the reliable source.

The list is longer than you remember

November 2023: OpenAI’s board fired Sam Altman, then reinstated him five days later. The board members most associated with safety concerns were the ones who didn’t survive the reshuffle.

May 2024: co-founder Ilya Sutskever and Jan Leike left within days of each other. Leike didn’t go quietly. He posted publicly that “safety culture and processes have taken a backseat to shiny products.” The Superalignment team they’d built together was dissolved shortly after.

Then a steady drip. The head of policy research. An advisor working on AGI readiness. By 2025, another name on the list barely registered as news. That’s the actual story — the fact that these departures stopped being surprising.

Product wins are visible. Safety wins are not.

Engineering leads leave. Product chiefs leave. But not this fast, and not this consistently. The difference is how you prove you did your job.

A product lead ships something. Usage climbs. Revenue follows. You can point at a chart.

A safety lead’s wins are things that didn’t happen. The incident you prevented. The launch you delayed. The feature you killed in review. When nothing goes wrong, someone in a budget meeting asks why the team exists. When something does go wrong, the same person asks what the team was doing. There is no version of the job where you get credit.

The incentives run the other way too. When a competitor ships a new frontier model next quarter, “let’s spend six more months on evals” is a proposal that costs the company money and market position. Safety teams exist on the org chart. In practice they function less like a brake and more like a warning light — and warning lights can be ignored indefinitely.

Responsibility without authority

Read the exit statements side by side and the same complaints repeat. Promised compute never materialized. Not in the room for the decisions that mattered. Concerns raised, concerns noted, concerns filed away.

When Superalignment launched, OpenAI committed 20 percent of its compute to the effort. Leike’s claim on the way out was that the team never actually got it. What safety orgs reliably receive is a title and accountability. What they don’t receive is budget and veto power. Put a genuinely capable person in a role with responsibility and no authority, and their competence works against you — they figure out the situation quickly and leave quickly.

The 2024 fight over OpenAI’s exit paperwork fits the same shape. Departing employees faced non-disparagement terms tied to their vested equity, which meant leaving didn’t buy you the freedom to talk. The company later said it wouldn’t enforce those provisions. But the default had been to make silence the price of your shares.

Safety as a person, not a process

Most AI labs handle safety the same way: hire an exceptional individual, hand them the judgment calls, trust their instincts. When that person leaves, the judgment leaves with them. The team gets absorbed into other groups. A few months later there’s a new team with a new name and a new lead, starting over.

That’s treating safety as a person rather than an institution. No company runs its financial controls this way. Auditors change; the procedures don’t. Audit has reporting obligations that reach outside the company, and executives can’t unilaterally shut it down. Most AI safety orgs have neither. They report internally, to the same leadership whose launch timelines they’re supposed to be checking.

So each departure resets the whole apparatus. Nothing compounds.

The three questions worth asking

Next time one of these announcements lands, “who left?” is the less useful question. Try these instead. How much budget and headcount does the successor actually control? Can they stop a launch, or only comment on one? Does their assessment reach the board directly, or does it route through the executives it might criticize?

If the answers are unchanged, only the nameplate changed.

And this isn’t an OpenAI problem. Google, Meta, and Amazon have all dissolved or shrunk their AI ethics groups over the past few years. Google’s 2020 ouster of Timnit Gebru is still the reference case, and the industry hasn’t produced a counterexample since. Commercial pressure and internal brakes move in opposite directions — that’s structural, not cultural.

From the outside, we get very few real signals about how these decisions get made. Here’s one we do get. A company where the head of safety changes every year is a company where safety decisions aren’t being made in that office. Worth asking, about whichever AI product you have open right now: who is making those calls, and what power do they actually hold?