If there's something else already out there in the universe, it would almost certainly have puts limits on our growth of intelligence. And the reason it would have put limits on us is because it doesn't want us to grow so intelligent that we would one day maybe take away their superpowered intelligence. Whatever advanced intelligence evolves, it always puts a roadblock in the way of other intelligences evolving. And the reason this happens is so nobody can take away one's power, no matter how far up the ladder they've gone.
Zoltan IstvanIn the future it's very possible you could have an artificial intelligence system that can run the country better than a human being. Because human beings are naturally selfish. Human beings are naturally after their own interests. We are geared towards pursuing our own desires, but oftentimes, those desires have contrasts to the benefit of society, at large, or against the benefit of the greater good. Whereas, if you have a machine, you will be able to program that machine to, hopefully, benefit the greatest good, and really go after that.
Zoltan IstvanMaybe we want to keep A.I. to the level of a 16-year-old or a 17-year-old adolescent, rather than some fully maxed-out artificial intelligence that becomes 10,000 times smarter than us in just a matter of years. Who knows what could happen? It could be a very dangerous scenario.
Zoltan IstvanOnce you've created an intelligence so smart, the real job of that intelligence is to protect itself from other intelligences becoming more intelligent than it. It's just kind of like human beings. The way you look at money or the way you look at the success of your child, you always want to make sure that as far as it gets, it can protect itself and continue forward. So I think any type of intelligence, no matter what it is, is going to have this very basic principle to protect the power that it has gained.
Zoltan IstvanI think whatever nation or whoever develops one artificial intelligence will probably make it so that artificial intelligence always stays ahead of any other developing artificial intelligence at any other point in time. It might even do things like send viruses to a second artificial intelligence, just so it can wipe it out, to protect its grounds. It's gonna be very similar to national politics.
Zoltan Istvan