Our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted.
Eliezer YudkowskyIf you want to build a recursively self-improving AI, have it go through a billion sequential self-modifications, become vastly smarter than you, and not die, you've got to work to a pretty precise standard.
Eliezer YudkowskyI am tempted to say that a doctorate in AI would be negatively useful, but I am not one to hold someoneโs reckless youth against them โ just because you acquired a doctorate in AI doesnโt mean you should be permanently disqualified.
Eliezer YudkowskyWe underestimate the distance between ourselves and others. Not just inferential distance, but distances of temperament and ability, distances of situation and resource, distances of unspoken knowledge and unnoticed skills and luck, distances of interior landscape.
Eliezer YudkowskyTo confess your fallibility and then do nothing about it is not humble; it is boasting of your modesty.
Eliezer YudkowskyDo not flinch from experiences that might destroy your beliefs. The thought you cannot think controls you more than thoughts you speak aloud. Submit yourself to ordeals and test yourself in fire. Relinquish the emotion which rests upon a mistaken belief, and seek to feel fully that emotion which fits the facts.
Eliezer Yudkowsky