The illusion of self-awareness. Happy automatons, running on trivial programs. I'll bet you never guess. From the inside, how can you?
Vernor VingeHere I had tried a straightforward extrapolation of technology, and found myself precipitated over an abyss. It's a problem we face every time we consider the creation of intelligences greater than our own. When this happens, human history will have reached a kind of singularity - a place where extrapolation breaks down and new models must be applied - and the world will pass beyond our understanding.
Vernor VingeIn fact, there was general agreement that minds can exist on nonbiological substrates and that algorithms are of central importance to the existence of minds.
Vernor Vinge