Central Limit Theorem | Research & Encyclopedia Articles

Central Limit Theorem

The central limit theorem is the name of a fundamental result in probability theory in which the so-called bell-shaped curve appears. Suppose, for example, that a fair coin is tossed N times. By a {\it fair} coin we understand an ideal coin in which the probability of getting heads is 1/2, and so the probability of getting tails is also 1/2. It is easy to compute the probability that after N tosses there will be exactly M heads, and so exactly N-M tails. When a fair coin is tossed N times we can record the outcomes as series of H's and T's. Thus after tossing a fair coin 10 times we might have an outcome such as T H H T T T H T H T. If we toss a coin N times the number of outcomes that contain exactly M heads is equal to the number of ways of selecting M positions among N possibilities in which the letter H will occur. This is given by the binomial coefficient

Next we must divide this number by the total number of possible outcomes of any kind that can occur when we toss a fair coin N times. The total number of possible outcomes is clearly 2^N. Therefore the probability that there are exactly M heads when a fair coin is tossed N times is given by

Before we go further, consider the case in which the fair coin is tossed N = 1000 times. Since we are using a fair coin we might expect that the most likely number of heads to occur is 500. This is in fact the most likely number of heads, but what is the probability that exactly 500 heads will occur? By the previous formula the probability is only

Now suppose that we wish to determine an interval of integers close to 500 for which we can be much more confident that after 1000 tosses the number of heads will be in our chosen interval. How is this interval to be selected and what is the probability that the number of heads will be in the interval? For example, suppose that we select the interval [490, 510]. The probability that after 1000 tosses of a fair coin the number of heads will be greater than or equal to 490 and less than or equal to 510 is given by

The central limit theorem allows us to make approximate calculations of this sort when the number N tends to .

Now let and be real numbers with < . Assume as before that we toss a fair coin N times. We wish to determine the probability that the number of heads is in the interval [½N + ½N, ½N + ½N]. By our previous remarks, the probability that the number of heads is in this interval is exactly

It turns out that as N this probability converges to a relatively simple expression that depends only on the numbers and . The precise limit is given by

This is the central limit theorem as it applies to tossing a fair coin. The graph of is the so-called bell shaped curve. In probability theory it is known as the normal or Gaussian density function. The area under this curve is exactly one unit. The function

is the normal or Gaussian distribution function.

We have illustrated the central limit theorem with a simple example in which we have counted the number of heads that occur when a fair coin is tossed. But the central limit theorem applies also to much more general and complicated situations involving sums of independent random variables, and the Gaussian distribution function continues to appear in the limit.