Statistical inference · Definition

p-value

statisticsinference

Before you start you need to know

It replaces a yes-or-no verdict with a number that says how surprising the data would be if the null hypothesis were true.

Definition

The \(p\)-value is the probability \(P\), computed assuming the null hypothesis \(H_0\) is true, of obtaining a result at least as extreme as the one observed.

\[p=P\big(\text{result at least as extreme}\mid H_0\big)\]

To compute it, first obtain the observed statistic \(z_{obs}\) and measure the tail it leaves. In a right-tailed test, \(p=1-\Phi(z_{obs})\); in a left-tailed one, \(p=\Phi(z_{obs})\); in a two-sided one, \(p=2\,[1-\Phi(|z_{obs}|)]\). With \(z=1.5\) this gives \(0.0668\) with one tail and \(0.1336\) with two.

The decision rule is simple: reject \(H_0\) at level \(\alpha\) when \(p\le\alpha\). That is why the p-value is the smallest significance level at which \(H_0\) would be rejected with these data. A small p-value says that, if \(H_0\) were true, such a result would be rare; not that \(H_0\) is improbable.

The most common misreading is taking \(p=0.04\) as "there is a \(4\,\%\) chance that \(H_0\) is true". The p-value conditions on \(H_0\); it is not a probability of \(H_0\). Nor does it measure effect size: with huge samples, tiny differences produce tiny p-values.

If \(H_0\) is true and the statistic is continuous, \(p\) is uniform on \((0,1)\), so \(P(p\le\alpha)=\alpha\): that is why the rule controls the Type I error. It is better to report the exact p-value together with the confidence interval, not just "significant" or "not significant".

Think about it

Why is the two-sided p-value double the one-sided one for the same \(z\)?

Because the two-sided test counts equally extreme results in both tails, and by symmetry each tail contributes the same.

With \(p=0.20\), is there evidence in favor of \(H_0\)?

No: only that the data are not surprising under \(H_0\); it is not rejected, but that does not prove it.

Exercises 10

  1. Two-sided test with \(z=1.5\). What is the p-value?
  2. Right-tailed test with \(z=2.0\). What is the p-value?
  3. A p-value of \(0.03\) and \(\alpha=0.05\). What is decided?
  4. A p-value of \(0.03\) and \(\alpha=0.01\). What is decided?
  5. Two-sided test with \(z=-2.4\). What is the p-value?
  6. We test \(H_0:\mu=50\) against \(H_1:\mu>50\) with \(n=36\), \(\bar x=53\) and \(\sigma=12\). What is the p-value?
  7. We get \(p=0.04\). Which interpretation is correct?
  8. Two-sided test with \(z=1.96\). What is the approximate p-value?
  9. If \(H_0\) is true and the statistic is continuous, what is the probability that \(p\le0.05\)?
  10. A statistic equals \(z=1.7\) and is tested at \(\alpha=0.05\). Which statement is correct?

References

  • George Casella, Roger L. Berger, Statistical Inference, 2nd edition, Duxbury / Cengage, 2002. §8.3.4
  • Ronald E. Walpole, Raymond H. Myers, Sharon L. Myers, Keying Ye, Probabilidad y estadística para ingeniería y ciencias, 9.ª edición, Pearson, 2012. cap. 10
  • Larry Wasserman, All of Statistics, 1st edition, Springer, 2004. §10.2