Conditional probability · Definition

Conditional probability

probabilityconditional

Before you start you need to know

Almost every useful probability is computed with partial information; this concept says how to update it when new data arrives.

Definition

The conditional probability of \(A\) given \(B\), \(P(A\mid B)\), is the probability that \(A\) occurs when \(B\) is already known to have occurred; it is computed within the space reduced to \(B\).

\[P(A\mid B)=\frac{P(A\cap B)}{P(B)},\qquad P(B)>0\]

If we know that \(B\) occurred, the only possible outcomes are those of \(B\), and the probability of \(A\) must be measured only inside them. This explains the formula: out of the total \(P(B)\) we keep the part where \(A\) also occurs, namely \(P(A\cap B)\). That is why \(P(B)>0\) is required; conditioning on an impossible event makes no sense.

With a die, let \(B=\) «even» and \(A=\) «a 6». Here \(P(A\cap B)=1/6\) and \(P(B)=1/2\), so \(P(A\mid B)=1/3\): after discarding the odd faces, the 6 goes from one in six to one in three. Information may raise, lower or leave a probability unchanged.

Solving the definition gives the product rule \(P(A\cap B)=P(B)\,P(A\mid B)\), which computes probabilities of sequences: two kings in a row without replacement have probability \(\tfrac{4}{52}\cdot\tfrac{3}{51}\). Moreover, for fixed \(B\), \(P(\cdot\mid B)\) is a probability measure: it satisfies the same axioms, for example \(P(A\mid B)+P(A^c\mid B)=1\).

The most common mistake is to confuse \(P(A\mid B)\) with \(P(B\mid A)\); in general they differ, since their denominators differ. This concept prepares independence (when conditioning changes nothing) and total probability, which combines conditionals over distinct cases.

Think about it

If \(P(A\mid B)=P(B\mid A)\) and both are positive, how are \(P(A)\) and \(P(B)\) related?

Equating the formulas, \(P(A\cap B)/P(B)=P(A\cap B)/P(A)\), and since \(P(A\cap B)>0\), \(P(A)=P(B)\).

Why is the answer 1/3 and not 1/2 in the two-children example?

Because «at least one girl» leaves three equally likely cases (GG, GB, BG), not two; birth order distinguishes the two mixed cases.

Exercises 10

References

  • Joseph K. Blitzstein, Jessica Hwang, Introduction to Probability, 2nd edition, CRC Press, 2019. §2.2
  • Sheldon Ross, A First Course in Probability, 10th edition, Pearson, 2019. §3.2
  • Larry Wasserman, All of Statistics, 1st edition, Springer, 2004. §1.4