If you want to build a recursively self-improving AI, have it go through a billion sequential self-modifications, become vastly smarter than you, and not die, you've got to work to a pretty precise standard.

After all, if you had the complete decision process, you could run it as an AI, and I'd be coding it up right now.

Our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted.

Okay, so either (a) I just teleported somewhere else entirely (b) they can fold space like no one's business or (c) they are simply ignoring all the rules.

I am tempted to say that a doctorate in AI would be negatively useful, but I am not one to hold someone’s reckless youth against them – just because you acquired a doctorate in AI doesn’t mean you should be permanently disqualified.

- With respect, Professor McGonagall, I'm not quite sure you understand what I'm trying to do here. - With respect, Mr. Potter, I'm quite sure I don't. Unless - this is a guess, mind - you're trying to take over the world? - No! I mean yes - well, NO! - I think i should perhaps be alarmed that you have trouble answering the question.

Do not flinch from experiences that might destroy your beliefs. The thought you cannot think controls you more than thoughts you speak aloud. Submit yourself to ordeals and test yourself in fire. Relinquish the emotion which rests upon a mistaken belief, and seek to feel fully that emotion which fits the facts.