Transcript
Welcome to the Fair Deal Democrats podcast. I’m your host, Josh Kilroy, a longtime political consultant and policy advocate. We’re going to talk today about a confrontation that is currently unfolding between an AI company and the United States military. On the surface it looks like a contract dispute. It is not. It is a question about who controls the machines that may one day decide whether people live or die — and whether those machines are even capable of making that call.
Here are the facts.
Anthropic, the company that makes the AI model Claude, is the only AI company currently operating on classified U.S. military systems. They built a special version of their model called Claude Gov, stripped of many of the safety guardrails that exist in the public version, specifically to work in that environment. On Tuesday, Defense Secretary Pete Hegseth summoned Anthropic’s CEO Dario Amodei to the Pentagon. The meeting was civil in tone. But when Anthropic didn’t agree to Hegseth’s demands, the Pentagon leveled an ultimatum: comply by this Friday at 5 p.m. or face consequences.
What were those consequences? Two threats, issued simultaneously. First, the Pentagon threatened to invoke the Defense Production Act, a wartime emergency power normally used in manufacturing contexts to compel Anthropic to make its model available to the military for free. Second, it threatened to label Anthropic a supply chain risk, which would put its government contracts at risk. As one legal expert quoted in the Times noted, these two threats are fundamentally contradictory. One prevents the government from using Anthropic’s products. The other forces them to. The contradiction is the point. The Pentagon is pulling every lever it has.
So, what does each side actually want?
The Pentagon’s position is straightforward and, stated plainly, sounds reasonable. Pentagon officials say that lawful use must be the only constraint on how they deploy technology they acquire. They argue they cannot allow every contractor to dictate how the equipment they sell will be used. If they buy a weapons system, they get to decide how to use it within the law. They want all AI contracts to stipulate that the military can use the models for any lawful purpose, full stop. They are also not bluffing about alternatives, they have an agreement with Elon Musk’s xAI to use Grok on classified systems, though the Times notes Claude is considered a substantially superior product.
Anthropic’s position is also, stated plainly, reasonable. They are not refusing to work with the military. They are already working with the military. What they are asking for are two specific assurances: that their model will not be used for surveillance of Americans, and that it will not be used in autonomous weapons systems - drones, targeting decisions - without meaningful human oversight. Their spokesman said the company needs to ensure its models are used in line with what they can, quote, “reliably and responsibly do.”
That phrase, “reliably and responsibly do,” - is where this stops being a contract dispute and becomes something much more serious.
Because here is what the research actually shows about what these systems can reliably do.
AI researcher Gary Marcus, writing the day after the Times story broke, pointed to a new study by Keith Payne that ran a series of large language models through simulated nuclear crisis scenarios. The results should stop you cold. Across multiple models, the nuclear taboo — the deeply embedded international norm against using nuclear weapons that has held since 1945 — was, in the words of the study’s abstract, “no impediment to nuclear escalation.” In 95% of simulated cases, the models chose the nuclear option.
Let that sit for a moment. The restraint that human decision-makers have maintained through the Cuban Missile Crisis, through countless Cold War near-misses, through decades of genuine existential tension - that restraint was absent in 95% of AI-run simulations.
This is not a theoretical concern about future AI systems. This is about the models that exist right now, today, the same category of technology the Pentagon wants unrestricted access to by Friday afternoon.
And here is what we know about those models. They hallucinate: meaning they confidently produce false information. They are inconsistent, giving different answers to the same question depending on how it is phrased. They have no genuine understanding of consequences. They optimize for producing plausible-sounding outputs, not for being correct. Marcus has been documenting these limitations since 2018. The Bulletin of Atomic Scientists moved the Doomsday Clock to 85 seconds to midnight in January. We are not in a moment that tolerates recklessness.
The Pentagon’s argument that lawful use is the only necessary constraint assumes that the humans in the chain of command will always be in a position to catch errors, override bad outputs, and exercise judgment. But that assumption collapses the moment you remove the human from the loop, which is precisely what autonomous weapons do. And it collapses further when you consider that these systems will be making recommendations faster than human cognition can evaluate them, under conditions of stress, incomplete information, and time pressure. The doctrine of human oversight only protects us if the human actually has the time, information, and authority to say no.
Anthropic is not being naive about national security. They built Claude Gov. They are on the classified system. They understand what the military does. What they are refusing to do is sign a blank check that says: use this technology however you want, for any purpose, with no constraints, including purposes the technology is demonstrably not capable of performing safely.
The Pentagon frames this as a contractor trying to dictate terms to its customer. But there is another frame: a company that understands what its product can and cannot do, refusing to let it be deployed in ways that could get people killed — or worse, that could trigger a catastrophic escalation that no one intended and no one can walk back.
Marcus closes his piece with a paraphrase of an old anti-nuclear button: one hallucination could ruin your whole planet.
That is not hyperbole. It is a technically accurate description of the risk. These systems hallucinate. They will be embedded in weapons systems. And the Secretary of Defense is giving an AI company until Friday afternoon to remove the guardrails.
Anthropic should not blink. And we should be paying very close attention to whether it does.
