Statistical inference · Definition

Hypothesis test

statisticsinference

Before you start you need to know

It turns a question about the world into a decision that can be made from data with a known risk of error.

Definition

A hypothesis test is a rule that, from sample data, decides whether to reject the null hypothesis \(H_0\), which fixes a value \(\theta_0\) for a parameter \(\theta\), in favor of the alternative \(H_1\).

\[H_0:\theta=\theta_0\quad\text{versus}\quad H_1:\theta\ne\theta_0\]

A test starts from two hypotheses: the null \(H_0\), which represents the status quo (for example \(\mu=\mu_0\)), and the alternative \(H_1\), which is what we want to support. The sample is summarized in a statistic; if \(\sigma\) is known and the sample mean is normal, the statistic is \(z=(\bar x-\mu_0)/(\sigma/\sqrt n)\). We reject \(H_0\) when \(z\) falls in the rejection region, which is fixed before seeing the data.

The form of \(H_1\) determines the region. If \(H_1:\mu\ne\mu_0\) the test is two-sided and we reject if \(|z|>z_{\alpha/2}\) (with \(\alpha=0.05\), \(1.96\)). If \(H_1:\mu>\mu_0\) we reject if \(z>z_\alpha\) (\(1.645\)), and if \(H_1:\mu<\mu_0\), if \(z<-z_\alpha\). The choice of \(H_1\) is made before looking at the data.

There are two ways to be wrong. A Type I error is rejecting a true \(H_0\), and its probability is the significance level \(\alpha\); a Type II error is failing to reject a false \(H_0\), with probability \(\beta\). With \(n\) fixed, lowering one raises the other; the only way to lower both is a larger sample. The quantity \(1-\beta\) is called the power.

Not rejecting \(H_0\) does not prove it: it only says the data are not enough to discard it. This connects with confidence intervals: a two-sided test of level \(\alpha\) rejects \(\mu_0\) exactly when it lies outside the interval of level \(1-\alpha\). The next concept, the p-value, summarizes how much evidence there is against \(H_0\) without fixing \(\alpha\) in advance.

Think about it

A test does not reject \(H_0\). Can we claim \(H_0\) is true?

No: only that the data give insufficient evidence against it; the test may have had low power.

How can \(\alpha\) and \(\beta\) both be reduced at once?

By increasing the sample size; with \(n\) fixed, lowering one raises the other.

Exercises 10

  1. We test \(H_0:\mu=50\) with \(n=36\), \(\bar x=53\) and known \(\sigma=12\). What is the statistic \(z\)?
  2. With \(z=1.5\), a two-sided test and \(\alpha=0.05\) (critical value \(1.96\)), what is the decision?
  3. What is a Type I error?
  4. What is a Type II error?
  5. We test \(H_0:\mu=50\) with \(n=25\), \(\bar x=46\) and \(\sigma=10\). What is \(z\)?
  6. For \(H_1:\mu>\mu_0\) with \(\alpha=0.05\) we reject if \(z>1.645\). If \(z=1.8\), what is concluded?
  7. For \(H_1:\mu<\mu_0\) with \(\alpha=0.01\) we reject if \(z<-2.326\). If \(z=-2.0\), what is concluded?
  8. We test \(H_0:\mu=50\) against \(H_1:\mu\ne50\) with \(n=64\), \(\bar x=47.6\), \(\sigma=8\) and \(\alpha=0.05\). What is concluded?
  9. With a fixed sample, what happens to \(\beta\) if \(\alpha\) rises from \(0.01\) to \(0.05\)?
  10. For \(\sigma=12\) and \(n=36\) the \(95\,\%\) interval for \(\mu\) is \((46.08,\ 53.92)\). Testing \(H_0:\mu=55\) two-sided with \(\alpha=0.05\), what is concluded?

What comes next

References

  • George Casella, Roger L. Berger, Statistical Inference, 2nd edition, Duxbury / Cengage, 2002. §8.1 y §8.3
  • Ronald E. Walpole, Raymond H. Myers, Sharon L. Myers, Keying Ye, Probabilidad y estadística para ingeniería y ciencias, 9.ª edición, Pearson, 2012. cap. 10
  • Larry Wasserman, All of Statistics, 1st edition, Springer, 2004. §10.1