In 1954, years after leading the project that created the atomic bomb, physicist J. Robert Oppenheimer was called to testify before the Atomic Energy Commission (AEC). The apparent topic of the hearings was Oppenheimer’s position on the hydrogen bomb, a much more destructive version of the atomic bomb that the United States had first developed and tested two years earlier.
Oppenheimer, who in the years after the war had become increasingly conflicted about atomic weapons, initially opposed work on the hydrogen or thermonuclear bomb, partly for moral reasons and partly because he was skeptical that it would work. But then he changed his mind and supported the work on it. The AEC lawyers wanted to know why.
It wasn’t because Oppenheimer had changed his mind about the morality of thermonuclear bombs vaporizing cities. Rather, it was because American physicists had come up with a new design for hydrogen bombs that was not only viable, but positively elegant, or “technically attractive,” as he called it. For Oppenheimer that was enough. As he said at the AEC hearing: “When you see something that is technically good, you go ahead and do it, and you argue about what to do about it only after you have had technical success.”
What Oppenheimer described was a kind of moral impotence disguised as resolution: the attraction of a scientifically beautiful answer to an unpleasant problem, and the accompanying habit of maintaining moral accounting until after technical success. It’s one of the most honest things ever said by anyone who built the bomb, or any other world-altering object. And it has never stopped being relevant, because the people who now build the world-altering technology of our time continue to say versions of it too.
Jack Clark, co-founder and chief policy officer of Anthropic, the company behind the Claude models, is one of those people. So it was worth paying attention last week when Clark sat down for a lengthy public dialogue with Samuel Kimbriel, founding director of Philosophy and Society at the Aspen Institute, just six days after the federal government abruptly cut off access to Anthropic’s two most powerful models, apparently out of fear of what they might do.
Much of the conversation revolved around a single idea that will be familiar to those who read Clark’s work: a powerful AI is coming and presenting us with a choice, a choice we actively refuse to make by not regulating AI. (Disclosure: Future Perfect is funded in part by the BEMC Foundation, whose primary funder was also an early investor in Anthropic; they have no editorial input on our content.)
We regulate toothbrushes, Clark noted, and cars and nuclear weapons. “But we seem to have this attitude toward technology that is impossible to regulate,” he said. “It’s not impossible to regulate…we act as if, well, the tech industry is inevitably going to do things, which I think is a choice.” Its clearest example was the online platform shift that completely reshaped the last two decades. “Social media conducted an uncontrolled experiment on the world,” he said. “Now we all think and talk a little differently thanks to social media. That was a choice. We can choose for things to be different.”
This is the kind of talk that has long set Anthropic apart from other major AI companies: Its directors are willing to insist on the serious risks of advanced AI, risks that demand clear and even strict regulation. (About a week before the Aspen dialogue, and just a day before the Trump administration came down heavily on Anthropic’s latest models, CEO Dario Amodei published a blog post calling on the government authority to legally block or even reverse the deployment of frontier AI models that fail security tests against threats like cyber-hacking and biological weapons.)
Anthropic recognizes that advanced AI is an existential gamble, but it is also a gamble we must take. At the Aspen Dialogue, Clark spoke of a coming century that will be marked by brutal challenges (aging populations, institutions under strain, a warming planet) that can seemingly only be addressed with AI. Not moving forward with artificial intelligence would be to deprive ourselves of medical miracles we can only imagine and implicitly condemn those who might otherwise be saved.
Clark is right that there is a hidden choice in all of this. But the question that eludes his framing is exactly whose choice it is.
Sure, as Clark said, we regulate cars, toothbrushes, and nuclear weapons, but in each case someone built the device first, and the rest of us had to decide what to do with a world that already contained it. No one voted on whether the atomic bomb should exist. They gave us the consequences and we had to write the rules later.
The same goes for AI. The choice Clark wants the public to make about their government only became necessary once his industry created what government needs. He offers us a vote on what to do with AI, not a vote on whether to do it, because that vote has already been cast, privately, by him, a few hundred colleagues, and billions of dollars. But why No Can we comment? Why are we stuck in a world where, as Oppenheimer says, “we discuss what to do about it only after we have had technical success”?
I wasn’t the only person in the audience wondering this. Near the end of the exchange, a young woman directly asked Clark a sharper version of this question. All frontier laboratories now admit that the technology carries enormous risk, even existential risk, he noted. “So my question is, what gives you, Anthropic, and the rest of the frontier labs the right to continue building something that could destroy everyone, when none of us can opt out?”
Clark, to his credit, did not dismiss the question. But he didn’t respond completely either. He reframed it: away from the choice to build, towards the need for someone to take responsibility once built.
That someone can’t be the companies themselves, he said, describing an ideal future in which “external compliance, regulatory, testing and verification systems” would decide when each lab would be allowed to go further. Governments were already moving faster than anyone expected: the United States and the United Kingdom, he said, had created testing agencies whose tools were sometimes better than the companies’ own.
It was a polite response, even though it didn’t sit well with the reality that President Trump now appears to be regulating AI on a whim, but let’s look at what he concedes. When asked what gives his company the right to build something that could destroy everyone, the head of policy at a major artificial intelligence lab did not say. we have that right. He said the decision shouldn’t fall on companies like his, just to describe a system to take it off their hands that doesn’t fully exist yet. He and his colleagues are still building, on the frontier, as fast as science and computing allow, while telling the room that someone else should be in charge. AI is already loose in the world. AI regulation is still mostly the stuff of blog posts.
So why are they really doing this? Going back to Oppenheimer: because AI is “technically good.” It’s not the race with China, or trillion-dollar valuations, or even the laudable desire to cure diseases, although all of that is real. Beneath them is something simpler and much more difficult to govern: we are committed to building the beautiful. Clark practically said it, marveling that AI is “easier and simpler to build than many other aspects of science,” and that his chief scientist jokes that they would already have AGI if they just fixed the bugs in their code.
Humans are a tool-using species, Clark argued, and AI is the ultimate tool. Not that AI is exactly inevitable, but that it is so strangely simple to build once the foundation is laid that “almost any path you follow, [AI] appears”.
What Clark described is the appeal that Oppenheimer named in 1954: the appeal of an elegant solution that makes the question of whether to build it or not seem irrelevant.
I can feel it myself and I’m just a user. Put a capable model at your fingertips, ask it to do something you couldn’t do alone (write the program, find the defect, untangle what you’re stuck on), and then just watch it. do what you asked for, and you’ll experience a little electric thrill that has nothing to do with an aging population or the future of democracy. That emotion runs in an unbroken line from the user at the keyboard, through the engineer who trained the model to the executive who shipped it.
That’s why I suspect Clark’s speech on regulation, however sincere, follows a decision that was never really in doubt. Like Oppenheimer with the hydrogen bomb, the people building this technology feel they have no choice but to move forward and then hope the rest of us make the right decisions to govern what they themselves couldn’t prevent.
So far we’ve lucked out with the latest technically cool device that could still end the world. The hydrogen bomb has been around for 70 years without being used in anger, not because we have resolved the politics Oppenheimer warned about, but because the wiser option won. And because we were lucky.
Clark may be right that the choice is still ours: the bomb did not decide the Cold War, the people did, and the people can decide this too. But it would be helpful if the people who give us that option would slow down long enough to allow us to do so, rather than building as fast as they can and hoping that our luck and theirs will hold.
A version of this story originally appeared in the future perfect information sheet. Register here!