How do we know when the world has changed?
On June 1, a team of scientists published a preprint scientific paper claiming that they had edited human embryonic DNA more precisely than any previous attempt. As a technical achievement, the work is certainly impressive, largely avoiding the errors that had accompanied previous efforts to edit genes in embryos. With further development, such embryonic editing could free future children from fatal or debilitating genetic diseases, but as veteran science writer Carl Zimmer reported in the New York Times that same week, the real headline news was that the work “could open the way to engineered babies with particular characteristics”; in other words, designer kids.
On the same day the Times article was published, artificial intelligence company Anthropic published a post claiming that AI was already accelerating AI development, which the authors say may represent a first step toward recursive self-improvement (RSI): AI systems that design and build their own successors, increasingly faster. Most of the code Anthropic’s Claude runs was already written by Claude himself, which has helped the company’s engineers ship eight times more code than they did two years ago. While more is not automatically better, and Claude is still far from being able to guide himself, the possibility of self-improving AI is on the horizon, and “could come sooner than most institutions are ready for,” as Anthropic co-founder Jack Clark and Anthropic Institute director Marina Favaro wrote.
These two writings were published by academic biologists and employees of an artificial intelligence company, in two wildly disparate disciplines, but they nevertheless point to a possible near future that is fundamentally different from the world we live in now.
Both events are possible key steps towards unprecedented powers, over which we would not have firm control over all: newly designed intelligences and humans. What they both share is not just consequence, but bivalence: the possibility of both the miraculous and the catastrophic. The biological precision that could eradicate an inherited disease like Huntington’s could also pave the way to a genetic caste system. The power of AI, which could accelerate decades of scientific progress, could also completely disempower its creators: us.
The world may have passed through a historic door with both developments last week. But we still can’t know what type.
First take the biology step. If we leave aside the headlines (which come from the media, not the scientists themselves), the experiment is quite limited.
Using so-called base editors, which make a small nick in a genetic strand rather than cutting out an entire segment, as CRISPR does, Columbia University geneticist Dieter Egli and his team edited two genes: PCSK9 and HBG. You may have heard of the first one; PCSK9 produces a protein that affects the body’s ability to remove cholesterol from the blood, and certain mutations in the gene can dangerously raise LDL cholesterol levels. HBG encodes a form of hemoglobin that the body depends on before birth and normally deactivates afterwards. Being able to control these genes could prevent mutations that increase the risk of heart disease (PCSK9) and reactivate that fetal hemoglobin in adulthood, alleviating, although not curing, sickle cell anemia and beta-thalassemia (HBG).
The researchers placed their base editors in fertilized eggs and two-cell human embryos and, in some cases, managed to make the edits without the chromosomal damage that had been associated with previous attempts at editing using CRISPR.
The paper, which has not yet been peer-reviewed, is an impressive step forward in the effort to use gene editing technology on human embryonic genes with greater precision. But the impressive thing is still far from perfect, or even safe: Some edits landed in the wrong place in the genome and relatively few embryos developed normally. (The embryos, which had been donated by IVF patients, did not develop beyond very early stages and none were implanted.) Egli and his colleagues made clear in the article that any notion of using the base editing technique as it is now used for treatment is “premature.” But the paper shows that such editing can now apparently be done without destroying chromosomes.
When Chinese scientist He Jiankui used conventional CRISPR to edit human embryos in 2018, producing three children, his work was widely rejected not only on moral grounds, but also on technical grounds, as his clumsy gene editing caused real genetic damage. If the new paper’s results are confirmed, the technical obstacles to embryo engineering will begin to disappear.
Nobody knows what comes next. Certain genetic disorders, such as sickle cell anemia, can be fixed with a single gene edit, but preventing more complex health problems (or engineering traits that some people might dream of, such as height or intelligence) would require editing hundreds or even thousands of genes in combinations we still don’t fully understand. But if the technical barriers continue to fall, only the moral ones will remain, and the moral ones have rarely held back a technology for long.
As revolutionary as the ability to truly engineer human beings is, biology is still advancing slowly. The same cannot be said for the subject of the other document published last week.
Anthropic’s post uses over 5,000 words and many (I assume) graphs produced by Claude to make a single point: the proportion of human labor that goes into building AI is shrinking at every stage. The engineers who once wrote the code now mainly review what Claude himself writes. Experiments that were previously designed manually are now increasingly proposed and executed using the model. While humans still make decisions about what is worth building, Anthropic has therefore begun to change even that, as employees increasingly differ on what the model proposes to do next.
A research circuit increasingly dominated by AI itself could advance ever faster. Technology has always changed at the pace of humans: how quickly they can think, plan and act. An AI capable of self-improvement removes that speed limit, allowing for the very real possibility that it moves faster than any human being or any human-run institution tasked with governing it can keep up. Intelligence itself becomes critical: each more intelligent model builds a more intelligent one, and the reaction sustains itself.
This might seem like a lot for a few months of internal coding data from an AI company that has a vested interest in making its models appear as strong and intelligent as possible. (Especially if that AI company has a potentially record-breaking IPO on the horizon.) In the post, Anthropic itself admits that simply counting lines of code only goes so far, and that speed is only a partial metric of success at best. But independent research has shown that AI models can spend more and more time on a single task, allowing them to work not only faster but also deeper. We can object to the speed, but not the idea that AI is advancing and fast.
Powerful, blindingly fast AI could lead to rapid economic, scientific and medical progress – all the dreams Anthropic CEO Dario Amodei has laid out in his own writings.
But it also threatens to be existentially dangerous and deeply disempowering for most of us, similar to what human genetic enhancement could be for those left out. And the potential speed of such change is so great that Anthropic is making the unusual proposal of asking AI companies to collectively consider slowing down or even temporarily pausing frontier AI development, to allow social structures and AI alignment research to keep up. The authors of the Anthropic article specifically cite international regimes created to control dangerous technologies of the past, such as nuclear weapons, which, for all their problems, have so far kept the world from annihilating itself. But those institutions, like the International Atomic Energy Agency, took decades to build, and as Anthropic leaders point out, when it comes to self-improving AI: “We don’t have that much time.”
How do we know when the world has changed?
Sometimes it’s immediate. When Otto Hahn and Fritz Strassmann achieved nuclear fission in December 1938, experts understood the implications almost immediately: a nuclear bomb would be possible. Sometimes scientists see it and the rest of the world doesn’t. When Jennifer Doudna and Emmanuelle Charpentier published the seminal paper detailing CRISPR in 2012, initial press attention was virtually non-existent, and the institutions that would eventually need to govern it had no idea what had just happened.
The most difficult cases of all are those in which even experts can only see half of it. Fission pointed in one direction, toward a weapon, and people who understood it could do little to stop it. Each of the two advances from last week points in two directions at the same time. The same editing technology that could save a child from a deadly disease is what could eventually sort children into genetic castes. The same intelligence that could give us “a country of geniuses in a data center,” as Amodei once said, could also leave us as little more than spectators in the world.
So we stay where we started, at a threshold that we can’t see beyond. The danger is not only that we have entered through the wrong door. It’s just that we passed by there without realizing there was one.
A version of this story originally appeared in the future perfect information sheet. Register here!