Probability foundations · Definition

Probability measure

probabilityfoundations

Before you start you need to know

To go from “what can happen” to “how likely it is”, we need a rule that spreads one unit of certainty over the events.

Definition

A probability measure \(P\) is a function that assigns to each event \(A\) a number between 0 and 1, equals 1 on the sample space \(\Omega\) and adds over mutually exclusive events.

\[P(\Omega)=1,\qquad P\Big(\bigcup_i A_i\Big)=\sum_i P(A_i)\]

Kolmogorov's axioms ask for three things: \(P(A)\ge 0\) for every event \(A\), \(P(\Omega)=1\), and additivity: if \(A_1,A_2,\dots\) are mutually exclusive, the probability of their union is the sum of their probabilities. Everything else in the theory follows from them; on infinite spaces, \(P\) is also defined only on a family of admissible events, a detail we will not need in finite cases.

Several useful rules follow from the axioms. Since \(A\) and \(A^c\) are mutually exclusive and add up to \(\Omega\), \(P(A^c)=1-P(A)\); in particular \(P(\varnothing)=0\) and \(0\le P(A)\le 1\). If \(A\subseteq B\), then \(P(A)\le P(B)\). And for any events, \(P(A\cup B)=P(A)+P(B)-P(A\cap B)\), because the intersection is counted twice when adding.

On a finite space it is enough to assign a nonnegative number to each outcome so that they add up to 1: the probability of an event is the sum of those of its outcomes. For a fair die each face gets \(\frac16\) and “even” has probability \(\frac36=\frac12\). A loaded die would be a different measure on the same \(\Omega\).

A common mistake is to believe that \(P(A)=0\) means \(A\) is impossible, or that \(P(A)=1\) means it is certain; on continuous spaces an event of probability zero can occur. The axioms do not say how to choose the numbers: the model decides that. The next concept studies the simplest case, where all outcomes weigh the same.

Think about it

Can a probability measure satisfy \(P(A)=0.7\) and \(P(A^c)=0.4\)?

No. Since \(A\) and \(A^c\) are mutually exclusive and their union is \(\Omega\), their probabilities must add up to 1, and here they add up to \(1.1\).

If \(P(A)\le P(B)\), can we conclude that \(A\subseteq B\)?

No. Containment implies the inequality, but not the other way around: two disjoint events can have \(P(A)=0.2\) and \(P(B)=0.5\).

Exercises 10

  1. A fair die has \(P(\{k\})=\frac16\) for each face. What is \(P(\{2,4,6\})\)?
  2. If \(A\) and \(B\) are mutually exclusive with \(P(A)=0.3\) and \(P(B)=0.5\), what is \(P(A\cup B)\)?
  3. Let \(\Omega=\{a,b,c\}\). Which assignment for \((P(a),P(b),P(c))\) defines a probability measure?
  4. If \(P(A)=0.35\), what is \(P(A^c)\)?
  5. Which of these values cannot be the probability of any event?
  6. If \(P(A)=0.4\), \(P(B)=0.5\) and \(P(A\cap B)=0.2\), what is \(P(A\cup B)\)?
  7. If \(A\subseteq B\), \(P(A)=0.3\) and \(P(B)=0.5\), what is \(P(B\setminus A)\)?
  8. It is known that \(P(A)=0.5\), \(P(B)=0.6\) and \(P(A\cup B)=0.9\). What is \(P(A\cap B)\)?
  9. If \(P(A)=0.6\) and \(P(B)=0.7\), what is the smallest possible value of \(P(A\cap B)\)?
  10. If \(P(A)=0\), what can be said for certain?

References

  • Joseph K. Blitzstein, Jessica Hwang, Introduction to Probability, 2nd edition, CRC Press, 2019. §1.6
  • Sheldon Ross, A First Course in Probability, 10th edition, Pearson, 2019. §2.3
  • Larry Wasserman, All of Statistics, 1st edition, Springer, 2004. §1.3