Conditional probability · Definition
Law of total probability
Before you start you need to know
When an event can happen through several routes, this formula lets you compute it case by case and then combine the results.
Definition
The total probability of an event \(A\) is the weighted average of its conditional probabilities \(P(A\mid B_i)\) over a partition \(B_i\) of the sample space, with the probability of each case as its weight.
The idea is divide and conquer. If the events \(B_1,\dots,B_n\) are mutually exclusive and cover the whole sample space (a partition), every outcome of \(A\) falls in exactly one \(B_i\). So \(A\) splits into the disjoint pieces \(A\cap B_i\), and adding their probabilities with the product rule \(P(A\cap B_i)=P(A\mid B_i)P(B_i)\) gives the formula.
It is a weighted average: each \(P(A\mid B_i)\) is multiplied by the weight \(P(B_i)\), and the weights add up to 1. Hence the result always lies between the smallest and the largest conditional probability. With three machines making 50 %, 30 % and 20 % of the parts and producing 1 %, 2 % and 5 % defectives, the overall rate is \(0.005+0.006+0.010=0.021\), not the plain average of 1, 2 and 5.
The most common case is the smallest partition, \(\{B,B^c\}\): \(P(A)=P(A\mid B)P(B)+P(A\mid B^c)P(B^c)\). It also handles sequential problems: the probability that the second card of a deck is an ace, conditioning on whether the first one was, is \(\tfrac1{13}\), the same as for the first.
Typical mistakes: using cases that overlap or fail to cover the space, and forgetting the weights. This result is the denominator of Bayes' theorem, which uses it to invert conditional probabilities.
Think about it
If \(P(A\mid B_i)=0.3\) for every case of a partition, what is \(P(A)\)?
It is 0.3: a weighted average of equal values whose weights add up to 1.
Why is the probability of an ace on the second card the same as on the first?
Averaging over what the first card could have been, symmetry makes no position in the deck special: \(\tfrac4{52}\cdot\tfrac3{51}+\tfrac{48}{52}\cdot\tfrac4{51}=\tfrac1{13}\).
Exercises 10
- Events \(B_1,B_2\) form a partition with \(P(B_1)=0.4\), \(P(B_2)=0.6\), \(P(A\mid B_1)=0.5\) and \(P(A\mid B_2)=0.2\). What is \(P(A)\)?
- Urn A has 3 red and 2 blue balls; urn B has 1 red and 4 blue. A fair coin picks an urn and a ball is drawn. What is the probability that it is red?
- Three machines make 50 %, 30 % and 20 % of the parts, with 1 %, 2 % and 5 % defective respectively. What fraction of all parts is defective?
- What must a family \(B_1,\dots,B_n\) satisfy to be used in the law of total probability?
- It rains with probability 0.3. If it rains you are late with probability 0.5; otherwise 0.1. What is the probability that you are late?
- Two cards are drawn without replacement from a 52-card deck. What is the probability that the second is an ace?
- Given \(P(A)=0.5\), \(P(B)=0.5\) and \(P(A\mid B)=0.8\), what is \(P(A\mid B^c)\)?
- A die is rolled. On a 6 a ball is drawn from an urn where P(red) is 1/2; otherwise from another where it is 1/10. Probability of drawing red?
- If \(0<P(B)<1\) and \(P(A\mid B)=P(A\mid B^c)=p\), what is \(P(A)\)?
- A disease affects 2 % of a population. The test is positive for 90 % of the sick and 5 % of the healthy. What fraction of everyone tests positive?
What comes next
References
- Joseph K. Blitzstein, Jessica Hwang, Introduction to Probability, 2nd edition, CRC Press, 2019. §2.3
- Sheldon Ross, A First Course in Probability, 10th edition, Pearson, 2019. §3.3
- Larry Wasserman, All of Statistics, 1st edition, Springer, 2004. §1.4