|
Where’s the “intelligence explosion”?
Ramez Naam gives a skeptic’s take on Recursive Self-Improvement.
|
| |||||||||||||||
 |
One of my fundamental beliefs about the world is that Ramez Naam ought to blog more. Ramez is one of the world’s greatest futurists — he predicted the solar and battery revolutions long before these were widely understood. If you were reading Ramez in 2011, you were able to understand the future of both energy technology and climate change, long before other people did. His earlier book More than Human is still a great guide to the kind of biological enhancements that AI might make possible. Ramez is also an excellent science fiction author, having written a trilogy of novels in which nanotechnological telepathy is distributed as a party drug (I’m not sure if he actually expects that to happen, but it’s a very cool idea).
Unfortunately, although he does have a Substack (which you should absolutely follow), Ramez does not blog regularly. However, after having a lengthy private debate with him about Recursive Self-Improvement, I was able to prevail upon him to write up his thoughts for my blog.
To say that RSI is a big deal in the AI world would be a colossal understatement. Among AI researchers, entrepreneurs, and AI safety people, there’s a widespread belief that as AI gets better at improving itself, there will be a “fast takeoff” or “FOOM”, in which AI’s capabilities “take off” and create a technological Singularity. This event is a staple of science fiction, including works by my favorite sci-fi author, Vernor Vinge.
A lot of people in the industry believe that this moment is now close at hand, and are racing toward that prize:
But Ramez — normally among the most wide-eyed of techno-optimists — is highly skeptical that we’ll see anything like the “FOOM” of Vernor Vinge novels. In this lengthy, well-researched post, he explains his skepticism.
Personally, I’m agnostic. Ramez’s case necessarily rests on a lot of assumptions; although it’s cogently laid out, I think the real answer is that we’ll just have to wait and see whether the Singularity arrives. But even more fundamentally, I don’t know how much this debate matters in the practical sense — even without the kind of Singularity depicted in sci-fi novels, AI capabilities are improving so rapidly that they’re already superhuman in many respects, and soon will probably be strongly superhuman in most or all dimensions. The AI of 2040 is going to look godlike, whether or not it explodes into an actual god in 2027.
Still, it’s a very interesting argument, and Ramez’s thoughts on the future of technology are always worth listening to.
1. AI is Helping Improve Itself
AI is already helping improve itself. The question is whether even fully autonomous recursive self-improvement (RSI) would cause a runaway intelligence explosion.
The theory is that each generation of AI could build a better successor, faster than the last generation did. That could lead to a “fast takeoff,” with capabilities surging to artificial superintelligence (ASI) in a year, months, or even days.
Here’s my take: Given our best current data, the AI self-improvement loop would need to be roughly 5–10× stronger to sustain itself, let alone run away. I’ll explain this math in section 8. I expect incredibly rapid AI progress by the standards of nearly any other technology. But the evidence we have doesn’t suggest a sudden explosion to incomprehensible superintelligence anytime soon.
I could be wrong. Forecasters have repeatedly underestimated AI progress! I could well be next. One thing that’s clear is that we need b