The greatest existential risks over the coming decades or century arise from certain, anticipated technological breakthroughs that we might make in particular, machine super intelligence, nanotechnology and synthetic biology. Each of these has an enormous potential for improving the human condition by helping cure disease, poverty, etc. But one could imagine them being misused, used to create powerful weapon systems, or even some kind of accidental destructive scenario, where we suddenly are in possession of some technology that's far more powerful than we are able to control or use wisely.

The challenge presented by the prospect of superintelligence, and how we might best respond is quite possibly the most important and most daunting challenge humanity has ever faced. And-whether we succeed or fail-it is probably the last challenge we will ever face.

We should not be confident in our ability to keep a super-intelligent genie locked up in its bottle forever.

Knowledge about limitations of your data collection process affects what inferences you can draw from the data.

The greatest existential risks over the coming decades or century arise from certain, anticipated technological breakthroughs that we might make in particular, machine super intelligence, nanotechnology and synthetic biology. Each of these has an enormous potential for improving the human condition by helping cure disease, poverty, etc. But one could imagine them being misused, used to create powerful weapon systems, or even some kind of accidental destructive scenario, where we suddenly are in possession of some technology that's far more powerful than we are able to control or use wisely.

How can we trace out the links between actions that people take today and really long-term outcomes for humanity - outcomes that stretch out indefinitely into the future? I call this effort macrostrategy - that is, to think about the really big strategic situation for having a positive impact on the long-term future. There's the butterfly effect: A small change in an initial condition could have arbitrarily large consequences.

In the next century, we will be inventing radical new technologies - machine intelligence, perhaps nanotech, great advances in synthetic biology and other things we haven't even thought of yet. And those new powers will unlock wonderful opportunities, but they might also bring with them certain risks. And we have no track record of surviving those risks. So if there are big existential risks, I think they are going to come from our own activities and mostly from our own inventiveness and creativity.