I think that in the immediate aftermath of a superhuman machine intelligence revealing itself, most people would feel very threatened but take solace in the thought that we can always pull the plug.

In fact, the very nature of an X-event is that it is both rare and surprising. So I would not say that any specific X-event is likely. What I would say, though, is that some X-event is not only plausible, but very likely in a time scale of a few years.

What distinguishes a mathematical model from, say, a poem, a song, a portrait or any other kind of "model," is that the mathematical model is an image or picture of reality painted with logical symbols instead of with words, sounds or watercolors.

Once the reality sets in of a superhuman intelligence being in control of every aspect of the infrastructures we rely upon for everyday life, we will simply have to try to come to an accommodation with that entity.

From the 1990s onward, the financial sector created a vast array of instruments designed to separate investors from their money, financial derivatives of an ever-increasing level of complexity. At some point, this complexity reached a point where even the creators of the derivatives themselves didn't understand them.

I believe is that we can forecast the "changing landscape of context," and thus get insight into when we are entering the danger zone of an X-event.

I don't think the emergence of a superhuman intelligence would be at all catastrophic but much more likely to be beneficial - just as long as we don't start trying to interfere with it!