Introduction to Statistics · Chapter 3 · Probability Topics

Two Basic Rules of Probability

Two rules turn a compound probability statement — both happen, either happens — into arithmetic on the pieces you already know, and each collapses to a shortcut exactly when the events are independent or mutually exclusive.

bookSHelf  ·  Introduction to Statistics  ·  §3.3  ·  a self-paced section

§3.3.1

The Multiplication Rule

What is the chance that two events both happen?

§3.3.1 — a two-step recipe, either order

Read it left to right

To have both $A$ and $B$ happen, first $B$ has to happen — that costs you $P(B)$. Then, in the shrunken world where $B$ already happened, $A$ has to happen too — that costs you $P(A \mid B)$. Multiply the two costs together.

The rule works just as well with the letters swapped, because “both” does not care which event you name first: $P(A \text{ AND } B) = P(A) \, P(B \mid A)$. Use whichever version matches the conditional probability the problem actually handed you.

§3.3.1 — solving for the conditional

The same rule, rearranged

Divide both sides by $P(B)$ and the multiplication rule rearranges into a formula for a conditional probability:

$$P(A \mid B) = \frac{P(A \text{ AND } B)}{P(B)}$$

In words: the probability of $A$ given $B$ equals the probability of $A$ and $B$ together, divided by the probability of $B$ — the same conditional probability from §3.1, now derived as a consequence of the multiplication rule.

§3.3.1 — the only situation the shortcut is allowed

Independence licenses the shortcut — and only independence

Independence is exactly the statement that knowing $B$ happened tells you nothing new about $A$ — the filter changes nothing, so $P(A \mid B)$ is just $P(A)$. Multiplying two plain probabilities when the events are dependent is the single most common error in this chapter.

This also runs in reverse, and that is how you will usually use it: if you can compute $P(A \text{ AND } B)$, $P(A)$, and $P(B)$ separately and the product does not match, the events are not independent. You will do exactly that test three times in this section.

Insight Note — the vertical bar is a filter, not a comma

$P(A \text{ AND } B)$ asks about the whole crowd.

$P(A \mid B)$ throws out everyone who isn't in $B$ first, then asks about the crowd that's left. Same numerator, smaller denominator.

§3.3.2

The Addition Rule

What is the chance that at least one of two events happens?

§3.3.2 — the subtraction is the whole point

Every outcome counted exactly once

If you just add $P(A)$ and $P(B)$, every outcome that belongs to both events gets counted twice — once when you counted $A$ and again when you counted $B$. Subtracting $P(A \text{ AND } B)$ removes the second copy, leaving each outcome counted exactly once.

Context Pause — why the overlap is subtracted once, not twice

Count the students who play soccer or run track.

Add the two rosters and anyone on both teams is counted twice, so the total comes out too high. Subtract the double-counted group once and you are back to a headcount of real people.

§3.3.2 — a different question than independence

Mutually exclusive is not independent

Mutually exclusive means the two events cannot both happen on the same trial, so there is no overlap to double-count. Do not confuse this with independence: mutually exclusive asks whether the events can co-occur; independent asks whether one changes the odds of the other. Two events can be neither, either, or — in the degenerate case where one is impossible — both.

§3.3.3

Using Both Rules on the Same Problem

Most real problems hand you a few probabilities and then ask four or five questions about them at once — an AND, an OR, an independence check, and a mutually-exclusive check. The work is always the same: write down what you were given in symbols, pick the rule that matches the question, and finish with arithmetic.

§3.3.3 — the habit that prevents errors

Label every event before you compute

Before you compute anything, label every event with a letter and write each given number next to the symbol it belongs to.

Half the errors in this section are not arithmetic — they are computing $P(A \mid B)$ when the problem gave you $P(B \mid A)$.

Try it in rāSHio

Both rules on one set of numbers

Open rāSHio, choose Calculators → Probability, switch the mode to Conditional, and enter $P(A) = 0.65$ and $P(B) = 0.65$ with Assume independence ticked. The panel returns $P(A \text{ AND } B) = 0.4225$ — because assuming independence is exactly what makes the AND equal the product.

That is the number the independence test compares against: the real AND for Nora is 0.585, so the panel's assumption is false and the two shots are not independent. Run it as the counterfactual, not as the answer.

§3.3.4

Probability and Medical Testing

A medical screening test is one of the most useful places to practice these two rules, because the question a patient actually cares about is a conditional probability that is easy to confuse with a different one. “How often does the test miss the disease?” and “how often does a negative result mean the patient is healthy?” are different questions with very different answers, and the difference is entirely a matter of which event you condition on.

§3.3.4 — two examples, one setup

Same numbers, flipped to their complements

The next two examples use the same setup and are meant to be read together. The first works from the disease to the test result; the second flips several of the same quantities to their complements. Keep your list of symbols in front of you.

Try it in rāSHio

Ask the tool what a useless test would look like

Open rāSHio, choose Calculators → Probability in Conditional mode, and enter $P(B) = 0.143$ and $P(N) = 0.85$ with Assume independence ticked. The panel hands back $P(N \mid B) = 0.85$ — the same as $P(N)$, because that is what independence means.

Now compare it to the 0.02 this example actually gives you. The gap between 0.85 and 0.02 is the test doing its job; a screening test whose result really was independent of the disease would return the panel's number, and would be worth nothing.

§3.3 — the headline result

The same two rules that compute $P(A \text{ AND } B)$ and $P(A \text{ OR } B)$ are also the tests that tell you whether the shortcut was ever allowed in the first place.

A useful screening test is deliberately not independent of the disease it tests for — Example 3.3.5's $P(N \mid B) = 0.02$ versus $P(N) = 0.85$ is the gap that makes the test worth running.

§3.3 — conclusions

What §3.3 leaves you with

Next: §3.4 — Contingency Tables, where these same AND/OR/conditional questions get read straight off a table of counts.