It may be appropriate to quote a statement of Poincare, who said (partly in jest no doubt) that there must be something mysterious about the normal law since mathematicians think it is a law of nature whereas physicists are convinced that it is a mathematical theorem.
Henri PoincareThe mathematical facts worthy of being studied are those which, by their analogy with other facts, are capable of leading us to the knowledge of a physical law. They reveal the kinship between other facts, long known, but wrongly believed to be strangers to one another.
Henri PoincareI then began to study arithmetical questions without any great apparent result, and without suspecting that they could have the least connexion with my previous researches. Disgusted at my want of success, I went away to spend a few days at the seaside, and thought of entirely different things. One day, as I was walking on the cliff, the idea came to me, again with the same characteristics of conciseness, suddenness, and immediate certainty, that arithmetical transformations of indefinite ternary quadratic forms are identical with those of non-Euclidian geometry.
Henri Poincare