Article by Luke Muehlhauser and Nick Bostrom.
Published in Think.
Humans will not always be the most intelligent agents on Earth, the ones steering the future. What will happen to us when we no longer play that role, and how can we prepare for this transition?
The human level of intelligence is an evolutionary accident – a small basecamp on a vast mountain side, far below the highest ridges of intelligence allowed by physics. If we were visited by extraterrestrials, these beings would almost certainly be very much more intelligent and technologically advanced than we are, and thus our future would depend entirely on the content of their goals and desires. But aliens are unlikely to make contact anytime soon. In the near term, it seems more likely we will create our intellectual successors. Computers far outperform humans in many narrow niches (e.g. arithmetic and chess), and there is reason to believe that similar large improvements over human performance are possible for general reasoning and technological development. . . .