3.5 Conditional Probability and Independence
Suppose we toss a fair die, and consider events A = {observing an even number} and B = {outcome < 3}. If we know that event B occurs, will it affect the chance that event A occurs? If a mother has breast cancer, will it affect her daughter’s probability of having breast cancer? More generally, if we acquire knowledge that some event has occurred, does it affect the probability of the occurrence of some other event? Such questions can often be addressed with conditional probabilities.
The probability that event A happens given that event B has occurred is called a conditional probability, denoted as [latex]P(A|B)[/latex], read as the conditional probability of A given B.
Two events A and B are independent if [latex]P(A|B) = P(A)[/latex], which means whether event B occurs or not won’t affect the probability of A. Similarly, we have [latex]P(B|A) = P(B)[/latex], which means whether A occurs or not won’t affect the probability of B.
In general, if A and B are any two events with [latex]P(B) > 0[/latex], then the conditional probability of A given B can be calculated as
[latex]P(A|B) = \frac{P(A \: \& \: B)}{P(B)}.[/latex]
In the case of the breast cancer, the probability of a daughter having breast cancer given that her mother has breast cancer is the probability of both the daughter and mother having breast cancer divided by the probability that the mother has breast cancer.
Examples: Conditional Probability
Suppose we roll a fair die, so that the sample space is S = {1, 2, 3, 4, 5, 6}. Consider the following events:
- Observing an even number, A = {2, 4, 6}.
- Observing an outcome that is less than 3, B={1, 2}.
- Observing an odd number, C={1, 3, 5}.
- Find the conditional probability [latex]P(A|B)[/latex].
- Method 1: [latex]\frac{f}{N}[/latex] rule
Given that B has occurred, the sample space reduces from S = {1, 2, 3, 4, 5, 6} to B = {1, 2}. A & B = even numbers less than 3 = {2}. Therefore, [latex]P(A|B) = \frac{f}{N} = \frac{1}{2} = 0.5[/latex]. - Method 2: by the conditional probability rule
[latex]P(A|B) = \frac{P(A \: \& \: B)}{P(B)} = \frac{1/6}{2/6} = \frac{1}{2}= 0.5.[/latex]
- Are the events A and B independent?
By the f/N rule, [latex]P(A) = \frac{3}{6} = 0.5[/latex], since [latex]P(A|B) = P(A)[/latex], events A and B are independent.
3. Find the conditional probability [latex]P(A|C)[/latex].
Since events A (observing an even number) and C (observing an odd number) are mutually exclusive,
[latex]P(A\: \& \: C)=0 \Longrightarrow P(A|C) = \frac{P(A \: \& \: C)}{P(C)}=\frac{0}{P(C)}=0.[/latex]
4. Are the events A and C independent?
Since [latex]P(A|C)=0\ne 0.5=P(A)[/latex], i.e., [latex]P(A|C)\ne P(A)[/latex], events A and C are dependent. Note that events A and C are mutually exclusive and hence must be dependent, since the occurrence of one event eliminates the chance of the other.
One common difficulty that students have with conditional probability problems is translating the text of a “word problem” into the correct probability statements. The following example illustrates different ways of writing a conditional probability.
Example: Different Ways of Writing a Conditional Probability
Suppose we roll a fair die, so that the sample space is S = {1, 2, 3, 4, 5, 6}. Consider the following events:
- Observing an even number, A = {2, 4, 6}.
- Observing an outcome that is less than 3, B={1, 2}.
We calculated the conditional probability [latex]P(A|B)=0.5[/latex] in the previous example. There are all ways we can write the conditional probability [latex]P(A|B)[/latex]:
- Given that the outcome is less than 3, the probability of observing an even number is 0.5.
- If the outcome is less than 3, then we have a probability of 0.5 of observing an even number.
- 50% of outcomes that are less than 3 are even numbers.
- If the outcome is less than 3, there is a 0.5 probability of observing an even number.
Multiplying P(B) on both sides of the conditional probability equation, we obtain the general multiplication rule of probability: [latex]P(A \mbox{ & }B)=P(A)\times P(B|A) \mbox{ or }P(A \mbox{ & }B)=P(B)\times P(A|B).[/latex]
The proof is as follows:
[latex]\begin{align*}P(A|B)= \frac{P(A \: \& \: B)}{P(B)}&\Longrightarrow P(B)\times P(A|B)=P(B)\times \frac{P(A \: \& \: B)}{P(B)}\\&\Longrightarrow P(A \: \& \: B)=P(B)\times P(A|B).\end{align*}[/latex]
If two events A and B are independent, i.e., [latex]P(A|B) = P(A)[/latex], the general multiplication rule becomes the special multiplication rule:
[latex]P(A \: \& \: B) = P(B) \times P(A|B) = P(B) \times P(A).[/latex]
More generally, if events A, B, C are independent, then
[latex]P(A \: \& \: B \: \& \: C \: \& \cdots) = P(A) \times P(B) \times P(C) \times \cdots.[/latex]
Key Facts: Check Whether Two Events are Independent
Two events A and B are independent, if and only if ANY of the following holds:
- [latex]P(A|B) = P(A)[/latex] or
- [latex]P(B|A) = P(B)[/latex] or
- [latex]P(A \: \& \: B) = P(A) \times P(B)[/latex]
The first two equations are due to the definition of independence, and the last one is due to the special multiplication rule.
- Any of the above equations can be used to check whether two events A and B are independent. DO NOT use all three, pick the one and the only one that is the easiest to calculate and check.
- Mutually exclusive and independent are two different concepts. Mutually exclusive events are those that cannot occur simultaneously; independent events are those for which the occurrence of one does not affect the probabilities of the others. Actually, mutually exclusive events must be dependent, since the occurrence of one event eliminates the chance of all the others.
