In my understanding, technological progress almost always proceeds relatively smoothly (see algorithmic progress, the performance curves database, and this brief investigation). Brain emulations seem to represent an unusual possibility for an abrupt jump in technological capability, because we would basically be ‘stealing’ the technology rather than designing it from scratch.
Similarly, if an advanced civilization kept their nanotechnology locked up nearby, then our incremental progress in lock-picking tools might suddenly give rise to a huge leap in nanotechnology from our perspective, whereas earlier lock picking progress wouldn’t have given us any noticeable nanotechnology progress.
If this is an unusual situation however, it seems strange that the other most salient route to superintelligence – artificial intelligence designed by humans – is also often expected to involve a discontinuous jump in capability, but for entirely different reasons. Is there some unifying reason to expect jumps in both routes to superintelligence, or is it just coincidence? Or do I overstate the ubiquity of incremental progress?
Crossposted from my own comment on the Superintelligence reading group. Commenters encouraged to do it over there.
The steam engine.
There was a relevant discussion on Slatestarcodex.
Astute observation. The short answer is that they both involve a religiously driven quest for a miracle.
Individual technologies evolve discontinuously, but overall technological progress is continuous. Since AI is the ultimate future, the distinction between the evolution of individual technologies and of total technology can seem to collapse: the discontinuous evolution of AI technology implies the discontinuous evolution of technology in total.
But that requires conceiving of AI as a single technology, which produces the greatest degree of discontinuity, submerging the general continuity of technology: it’s a low-probability event, given that there’s a possible routes to transhumanism that don’t involve a single jump (actually, two jumps in the case of emulations).
[Perhaps I’ve read too much cyberpunk, but it seems like brain augmentation is the obvious gradualist alternative.]