Eugene Wigner's famous essay The Unreasonable Effectiveness of Mathematics in the Natural Sciences from 1960 (which has its own wikipedia page) begins with a story about the unexpected appearance of the number $\pi$ in statistics:
THERE IS A story about two friends, who were classmates in high school, talking about their jobs. One of them became a statistician and was working on population trends. He showed a reprint to his former classmate. The reprint started, as usual, with the Gaussian distribution and the statistician explained to his former classmate the meaning of the symbols for the actual population, for the average population, and so on. His classmate was a bit incredulous and was not quite sure whether the statistician was pulling his leg. "How can you know that?" was his query. "And what is this symbol here?" "Oh," said the statistician, "this is pi." "What is that?" "The ratio of the circumference of the circle to its diameter." "Well, now you are pushing your joke too far," said the classmate, "surely the population has nothing to do with the circumference of the circle."
All students of statistics encounter this appearance of $\pi$ in the density function \[\frac1{\sqrt{2\pi}}\cdot e^{-x^2/2}\] of the normal distribution. The number $\pi$ enters because the probability of getting exactly $n$ heads and $n$ tails when flipping a coin $2n$ times is \[\frac12\cdot \frac34 \cdot \frac56 \cdot \cdots \cdot \frac{2n-1}{2n},\] which (for large $n$) is approximately \[ \frac1{\sqrt{\pi n}}.\]
In his talk "Why Pi?" from December 2010, Donald Knuth said about this (10-11 minutes into the video) that
This has always been a mystery, I think to everybody, why pi should occur... . It seemed to be one of those strange coincidences or facts of nature that there's a constant, and lo and behold, it agrees with pi... . However, two years ago a guy in Sweden named Johan Wästlund figured out why.
Happy pi-day to everyone!