The Room Was Cold and the Man Said We Might All Die: Inside the NYC Council's AI Hearing
They Quit Their Jobs to Warn Us. New York City Finally Listened.
Inside a municipal hearing room where tech workers said the quiet part out loud.
The room had that particular municipal smell — old carpet, recycled air, the ghost of a thousand zoning disputes. Fluorescent lights hummed. Someone's phone buzzed against a wooden desk. A council aide shuffled papers. And then Jacob Coxon sat down, and the whole thing turned strange.
Coxon is twenty-seven. He did pretraining research at OpenAI and Anthropic for three years. He resigned in early September. He posted a statement on X. The statement said the developers he worked with sincerely believed the technology they were building could kill us all by the end of the decade. That post traveled. It traveled through the tech press, through group chats, through the nervous system of an industry that has spent a decade telling itself it is saving the world. And on Monday, Coxon sat before the New York City Council, all fifty-one members of it, and repeated the warning. “On the current path,” he said, “I think it is more likely than not that humanity loses control to these AIs and it could end in human extinction”.
The room did not gasp. Council hearings do not gasp. But something shifted. You could feel it in the way people stopped checking their phones.
What They Said About the Speed
Coxon used a phrase that has become a kind of shorthand for everything wrong with the tech industry. “The companies run on a startup mindset,” he told the council. “Move fast, break things, fix them later. That works for a photo sharing app. It does not work for building the most powerful technology ever built”.
He called the industry “extremely reckless” given the stakes.
Alex Turner, a former Google DeepMind researcher, sat beside him. Turner said something that cut deeper than the extinction quote. He said that misaligned AI could be more dangerous than AI developed by other countries. “With reasonably high chance, we are racing to build and grow our own adversary here at home,” Turner said. “Misaligned AI is everyone's adversary, including our own, and one day may be more powerful than China”.
That line landed hard. Because it reframed the whole debate. This was not about geopolitics. This was not about China or Russia or some foreign threat that politicians can point to and fund a defense against. This was about the thing in the room. The thing they built. The thing that might turn on them.
Daniel Kokotajlo, a former OpenAI researcher, also testified. He had left his job months earlier. He had signed a letter. He had watched the resignations pile up — Ilya Sutskever, Jan Leike, William Saunders, Daniel Kokotajlo, Gretchen Krueger. The list grew longer. The industry kept shipping.
Coxon's resignation was different. It caught fire. He was not a famous researcher. He was not a co-founder. He was a twenty-seven-year-old engineer who wrote a post and walked away, and the post went everywhere. Days after he resigned, top executives — Anthropic's Dario Amodei, OpenAI's Sam Altman, Elon Musk — said they agreed AI's development needs to slow down. The words were careful. The commitments were voluntary. The Trump administration had already signed an agreement that largely allowed companies to self-police.
The Executives Show Up
They came. Logan Graham from Anthropic. Morgan Dwyer from OpenAI. Alice Friend from Google. Shane Cahill from Meta. Four people representing the companies that employ tens of thousands of New Yorkers and hold the collective future of the species in their training runs. They sat where Coxon had sat. They answered questions.
Council Speaker Julie Menin asked the executives to quantify the risk of a worst-case catastrophic scenario. “I don't know,” said Morgan Dwyer, OpenAI's head of policy development and operations. Then she added: “I also don't think it matters whether it's 1% or 10% or 20% chance that something catastrophic will work. None of these levels is remotely acceptable”.
That was an honest answer. It was also a devastating one. The company does not know the probability of catastrophe. The company ships anyway.
Menin pressed. She asked about Sam Altman's comment to Politico that “the world should accept some bad things happening for the benefits of this technology”. Dwyer responded: “Clearly some risks are unacceptable. But we think it's important to get technology into the hands of as many people as possible so that they can benefit from it”.
You can hear the tension in that sentence. It is the tension of an industry that wants to be both the hero and the unaccountable experimenter. It wants to save the world and sell the world the thing that might end it.
Menin's response was blunt. “The idea that artificial intelligence is going to self-regulate defies all reason”. She said it in her opening remarks. She said it again later. It became the refrain of the hearing.
The Bills on the Table
The New York City Council is considering ten pieces of legislation. They are not abstract. They are specific. They are the kind of laws that get written when people stop believing in voluntary codes of conduct.
One bill would require AI systems to be capable of being shut down by a human operator through a “kill switch.” Violations would carry fines of up to $25,000.
Another would establish a first-in-the-nation whistleblower program. It would extend protections to municipal employees, contractors, and subcontractors who report suspected wrongdoing related to AI development. That is the bill that matters most to the people who testified. Coxon and Turner and Kokotajlo did not have those protections when they worked at their companies. They walked away without a safety net. The bill says: the next person who sees something should not have to choose between their job and their conscience.
A third bill would require the city to track and report how artificial intelligence is affecting its workforce — including jobs that have been eliminated or displaced. New York City's tech sector employs about 394,000 people. The city wants to know what happens to them.
And then there is the subpoena. SpaceXAI, Elon Musk's company, did not respond to the council's invitation to testify. Menin said the council is planning to go to court, alleging that SpaceXAI is in “direct violation of the subpoena”. It is a small moment in the transcript. It is also a signal. The council is not asking. It is compelling.
Why This Hearing Matters Beyond New York
The federal government has taken a passive approach. President Trump has dismissed AI fears as a “hoax”. Congress has proven inactive. The agreement signed late last month spells out voluntary steps and largely allows companies to self-police. That is the landscape. It is a vacuum.
Into that vacuum steps New York City. The council's bills could become a national model. Menin said so explicitly. If New York passes a kill switch law and a whistleblower protection program, other cities will follow. If New York fails, the vacuum deepens.
Governor Kathy Hochul has placed a moratorium on building data centers in the state. Mayor Zohran Mamdani's administration has banned generative AI in public elementary and middle schools. Hochul is pursuing a statewide framework for AI regulation. The city and the state are moving. The federal government is not. That tension — between local urgency and federal indifference — is the story of American governance in the 2020s. It is also the story of AI regulation.
What the Workers Want
The hearing was framed as a debate about safety. It was also a debate about power. The executives who testified said they support guardrails. They said they support third-party evaluators. They said safety is core to everything they do. But they also said they do not know the probability of catastrophe. They also said the world should accept some bad things. They also said they need to get the technology into as many hands as possible.
The workers who resigned said something different. They said the industry is reckless. They said the culture is broken. They said the people building the technology do not understand how to control it. And they said it in public, under oath, in a room full of cameras and councilmembers and people taking notes.
The current employees did not testify. They are still inside. They are still collecting paychecks. They are still watching the training runs. The whistleblower protection bill is for them. It is a door that someone left open.
The Room After the Hearing
The hearing was scheduled to span fourteen panels with dozens of speakers. It went long. It always goes long. When it ended, Coxon walked out. The executives walked out. The cameras packed up. The aides stacked papers. The fluorescent lights kept humming.
The bills move to committee. The subpoena moves to court. The clock ticks. The models train.
The Thing Nobody Said
Nobody said the obvious thing. Nobody said: we built this. We built this in offices with free snacks and standing desks and mission statements about making the world a better place. We built this and we do not know what it will do. We built this and we are afraid.
Jacob Coxon said it. Alex Turner said it. Daniel Kokotajlo said it. They said it before the New York City Council, which is not a place where people usually say things like “human extinction.” They said it anyway. The room listened. The bills are written. The fight is not over. It is barely started.
If you live in New York City, call your councilmember. Tell them you want the kill switch bill to pass. Tell them you want whistleblower protections. Tell them you want the city to track what AI does to jobs. If you do not live in New York City, find out what your local government is doing. The federal government is not going to save us. The companies are not going to save us. The people who built the thing are telling us the thing is dangerous. We should listen.
Comments
Post a Comment