Central limit theorem

Let \(X_1,\dots,X_n\) be independent, with the same distribution, mean \(\mu\) and finite positive variance \(\sigma^2\). The sum \(S_n\) has mean \(n\mu\) and variance \(n\sigma^2\); the average \(\bar X_n\) has mean \(\mu\) and variance \(\sigma^2/n\). The theorem says the standardized versions, \((S_n-n\mu)/(\sigma\sqrt n)\) and \((\bar X_n-\mu)/(\sigma/\sqrt n)\), converge in distribution to an \(N(0,1)\).

In practice it is used as an approximation: for large \(n\), \(\bar X_n\) is almost \(N(\mu,\sigma^2/n)\), and probabilities are computed by standardizing and consulting the normal distribution function. A common rule asks for \(n\ge30\); if the population is very skewed more is needed, and if it is already normal the approximation is exact for any \(n\).

The theorem complements the law of large numbers: that law says \(\bar X_n\) approaches \(\mu\), and this one says the error \(\bar X_n-\mu\) is of order \(\sigma/\sqrt n\) and normally shaped. That is why quadrupling the sample only halves the error. It is the basis of confidence intervals and hypothesis tests.

Typical errors: dividing by \(n\) instead of \(\sqrt n\) when getting the standard error, forgetting independence, or believing individual data become normal; what approaches a normal is the distribution of the sum or the average, not that of each \(X_i\).