Our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted.
Eliezer YudkowskyIf you want to build a recursively self-improving AI, have it go through a billion sequential self-modifications, become vastly smarter than you, and not die, you've got to work to a pretty precise standard.
Eliezer YudkowskyIf the iron is hot, I desire to believe it is hot, and if it is cool, I desire to believe it is cool.
Eliezer YudkowskyNot every change is an improvement but every improvement is a change; you can't do anything BETTER unless you can manage to do it DIFFERENTLY, you've got to let yourself do better than other people!
Eliezer YudkowskyI'm wondering if there's a spell to make lightning flash in the background whenever I make an ominous resolution.
Eliezer YudkowskyLet the winds of evidence blow you about as though you are a leaf, with no direction of your own. Beware lest you fight a rearguard retreat against the evidence, grudgingly conceding each foot of ground only when forced, feeling cheated. Surrender to the truth as quickly as you can.
Eliezer Yudkowsky