Random sample

In practice we can rarely observe a whole population, so we measure a part of it. For that part to say something reliable about the whole, each observation must come from the same mechanism and must not depend on the others. This idea is formalized with random variables \(X_1,\dots,X_n\): each \(X_i\) is the value we will get in observation \(i\) before measuring it, which is why it is random.

Independence lets us multiply probabilities: \(P(X_1\le a_1,\dots,X_n\le a_n)=\prod_i P(X_i\le a_i)\). Equal distribution means they all share the same law, with the same mean and variance. It does not mean they take the same value: two tosses of a fair coin have the same distribution and can give different results.

A guiding example is \(n\) independent tosses of a coin with \(P(\text{heads})=p\). Each specific sequence with \(k\) heads has probability \(p^k(1-p)^{n-k}\). The values already measured, \(x_1,\dots,x_n\), are called the observed sample; the \(X_i\) are their version before observation.

The typical mistake is calling any data set a random sample. Choosing acquaintances, taking a group whose members resemble each other, or drawing without replacement from a small population breaks independence or equal distribution. The sample mean and estimators are built on this foundation.