Theoretical and experimental probabilities are linked by the Law of Large Numbers. This law states that if an experiment is repeated numerous times, the relative frequency, or experimental probability, of an outcome will tend to be close to the theoretical probability of that outcome. Here the relative frequency is the quotient of the number of times an outcome occurs divided by the number of times the experiment was performed.

The Law of Large Numbers is more than just a general principle. The Swiss mathematician Jakob Bernoulli (1659–1705) was the first to recognize the connection between long-run proportion and probability. In 1705, the year of his death, he provided a mathematical proof of the Law of Large Numbers in his book Ars Conjectandi ("The Art of Conjecturing"). This principle also plays a key role in the understanding of sampling...