I think we're already getting to a stage where the basic artificial intelligences are discovering moral systems. I think, in many ways, moral systems are simply things that we have programmed into ourselves, either through childhood or just through genetic, ingrained ideas. So the same thing applies when you talk about machines.

Science and technology can solve all the world's problems, and historically it has been shown to make the world better and better.

Maybe we want to keep A.I. to the level of a 16-year-old or a 17-year-old adolescent, rather than some fully maxed-out artificial intelligence that becomes 10,000 times smarter than us in just a matter of years. Who knows what could happen? It could be a very dangerous scenario.

If there's something else already out there in the universe, it would almost certainly have puts limits on our growth of intelligence. And the reason it would have put limits on us is because it doesn't want us to grow so intelligent that we would one day maybe take away their superpowered intelligence. Whatever advanced intelligence evolves, it always puts a roadblock in the way of other intelligences evolving. And the reason this happens is so nobody can take away one's power, no matter how far up the ladder they've gone.

I think we're already getting to a stage where the basic artificial intelligences are discovering moral systems. I think, in many ways, moral systems are simply things that we have programmed into ourselves, either through childhood or just through genetic, ingrained ideas. So the same thing applies when you talk about machines.

Once you've created an intelligence so smart, the real job of that intelligence is to protect itself from other intelligences becoming more intelligent than it. It's just kind of like human beings. The way you look at money or the way you look at the success of your child, you always want to make sure that as far as it gets, it can protect itself and continue forward. So I think any type of intelligence, no matter what it is, is going to have this very basic principle to protect the power that it has gained.

Whoever creates an artificial intelligence first has such a distinct military advantage over every other nation on the planet that they will forever, or they will at least indefinitely, rule the planet. It's very important that a nice country, a democratic country, develops A.I. first, to protect other A.I.'s from developing that might be negative, or evil, or used for military purposes.