The more improbable the message, the less "compressible" it is, and the more bandwidth it requires. This is Shannon's point: the essence is its improbability.
William PoundstoneThere were many at Bell Labs and MIT who compared Shannon's insight to Einstein's. Others found that comparison unfair - unfair to Shannon.
William PoundstonePeople tend to be clueless about prices. Contrary to economic theory, we don't really decide between A and B by consulting our invisible price tags and purchasing the one that yields the higher utility, he says. We make do with guesstimates and a vague recollection of what things are “supposed to cost.”
William Poundstone