How can we trace out the links between actions that people take today and really long-term outcomes for humanity - outcomes that stretch out indefinitely into the future? I call this effort macrostrategy - that is, to think about the really big strategic situation for having a positive impact on the long-term future. There's the butterfly effect: A small change in an initial condition could have arbitrarily large consequences.
Nick BostromThe first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.
Nick BostromItโs unlikely that any of those natural hazards will do us in within the next 100 years if weโve already survived 100,000. By contrast, we are introducing, through human activity, entirely new types of dangers by developing powerful new technologies. We have no record of surviving those.
Nick Bostrom