Sample mean

Once the observations \(x_1,\dots,x_n\) are collected, their average \(\bar x\) is a number. But before observing, \(\bar X\) depends on chance: another sample would give another average. That is why it has its own distribution, called the sampling distribution of the mean, and it is what tells us how far to trust a specific average.

By linearity of expectation, \(E[\bar X]=\mu\): on average, the sample mean hits the population mean. By independence, \(\operatorname{Var}(\bar X)=\sigma^2/n\), so the standard deviation \(\sigma/\sqrt n\), called the standard error, shrinks with the square root of the size: halving it takes four times as much data.

A useful property is that the deviations \(x_i-\bar x\) always add up to zero, so \(\bar x\) is the balance point of the data. It is also sensitive to extreme values: a single very large data point can pull it. With samples of different sizes, the combined mean is computed by weighting by the number of data points.

The typical mistake is confusing \(\bar X\) with \(\mu\): the first is computed from data, the second is an unknown parameter. The law of large numbers says \(\bar X\) approaches \(\mu\) as \(n\) grows, and the central limit theorem describes its shape. Every estimator is judged with these ideas.