Probability

A random experiment is one whose outcome cannot be predicted with certainty in advance, even though every possible result is known beforehand. Tossing a coin, rolling a die, or drawing a card from a well-shuffled pack are all random experiments: you know the menu of possibilities, but not which one will turn up on a given trial.

Each individual result of the experiment is an outcome. The set of all possible outcomes is the sample space, written $S$. For a single die, $S = \{1, 2, 3, 4, 5, 6\}$; for one toss of a coin, $S = \{H, T\}$. Every element of $S$ is called a sample point, and $n(S)$ is the total number of sample points.

An event is any subset of the sample space — a collection of outcomes we choose to single out. "Getting an even number on a die" is the event $E = \{2, 4, 6\}$, a subset of $S$. This is the bridge to the previous chapter: events are sets, and the sample space is the universal set. Everything we do with probability is set theory wearing a different hat.

Because events are sets, the set operations carry direct meanings. The complement $\overline{A}$ (also written $A'$) is the event "$A$ does not happen". The union $A \cup B$ is "$A$ or $B$ (or both)", and the intersection $A \cap B$ is "$A$ and $B$ together".

Two events $A$ and $B$ are mutually exclusive when they share no outcome, so they can never happen on the same trial. A collection of events is exhaustive when their union is the entire sample space, so at least one of them must occur. When events are both mutually exclusive and exhaustive, they partition $S$ cleanly into non-overlapping pieces that together account for everything.

Deeper Insight — probability is set theory with a measuring tape: The reason Chapter 1 on Sets sits in the same syllabus as this chapter is no accident. A sample space is just a universal set, an event is just a subset, and "or", "and", "not" translate exactly into union, intersection, and complement. This is why the vocabulary feels familiar: "mutually exclusive" is the everyday name for disjoint sets, and "exhaustive" simply restates that a family of subsets covers the universal set. Once you accept that an event is a set of outcomes, the laws you already know — De Morgan's laws, the distributive laws — transfer wholesale. Probability then adds one new ingredient on top of this set machinery: a way to measure how large an event is, relative to the whole sample space. Get the set picture firm first, and the numerical rules in the next two topics will feel like bookkeeping rather than new ideas.

Once outcomes and events are in place, we attach a number — the probability — to each event, measuring how likely it is. The axiomatic approach, due to Kolmogorov, fixes three simple rules that any sensible probability must obey, and everything else follows from them.

For a sample space $S$ and any event $E$, the axioms of probability are:

In words: (1) every probability lies between $0$ and $1$, (2) the sure event has probability $1$, and (3) for mutually exclusive events the probabilities simply add. The impossible event follows immediately: $P(\varnothing) = 0$.

When all outcomes are equally likely — a fair coin, a fair die, a well-shuffled pack — the probability of an event reduces to a ratio of counts. This is the classical (Laplace) definition you will use most:

Notice that this formula only applies when the outcomes are equally likely. A loaded die or a bent coin breaks the assumption, and then you must use observed (empirical) frequencies instead.

A constant companion of any event is its complement $\overline{A}$, the event "$A$ does not occur". Since $A$ and $\overline{A}$ are mutually exclusive and together exhaust $S$, their probabilities add to $1$:

This complement rule is often the fastest route to an answer: when "at least one" appears in a question, computing $1 - P(\text{none})$ is usually far shorter than counting the favourable cases directly.

Deeper Insight — why axioms beat a single "definition": Earlier you may have met probability as nothing more than favourable over total. That classical formula is genuinely useful, but it quietly assumes the outcomes are equally likely — an assumption that fails for a weighted coin, an unfair die, or a continuous quantity like a person's exact height. The axiomatic framework sidesteps this by not committing to how the numbers arise; it only demands that whatever numbers we assign are non-negative, sum to $1$ over the whole sample space, and add up over disjoint events. The familiar $\tfrac{n(A)}{n(S)}$ is then just one way to satisfy the axioms, the way that happens to work when symmetry makes every outcome equally likely. Because the complement rule, the addition theorem, and every later result are all derived from these three axioms, learning to reason from them — rather than memorising isolated formulas — is what makes harder problems tractable.

The complement rule handles a single event and its negation. The addition theorem handles the probability that at least one of two events occurs — the event $A \cup B$, read "$A$ or $B$".

For any two events $A$ and $B$ of the same sample space:

The reason for the subtraction is geometric. If you simply add $P(A)$ and $P(B)$, the outcomes lying in both events — the overlap $A \cap B$ — get counted twice, once inside each event. Subtracting $P(A \cap B)$ removes the double count and restores the correct total. On a Venn diagram, you are adding two overlapping discs and then peeling off the lens-shaped middle that you covered twice.

When the two events are mutually exclusive, they have no common outcome, so $A \cap B = \varnothing$ and $P(A \cap B) = 0$. The theorem then collapses to the simple additive form, which is exactly the third axiom:

The result extends to three events, again alternately adding and subtracting overlaps:

A pairing with the complement rule is worth memorising: "neither $A$ nor $B$" is the complement of "$A$ or $B$", which by De Morgan's law gives a handy shortcut.

Deeper Insight — the addition theorem is inclusion–exclusion in probability dress: If the subtraction in $P(A \cup B) = P(A) + P(B) - P(A \cap B)$ feels familiar, that is because you have already met its set-counting twin: $n(A \cup B) = n(A) + n(B) - n(A \cap B)$ from the chapter on Sets. The two are the same identity. Divide every term of the counting version by $n(S)$ and each count becomes a probability, turning the inclusion–exclusion principle for sizes into the addition theorem for likelihoods. This is the clearest illustration of the chapter's central message: probability is built directly on set theory, and the overlap must be subtracted for exactly the same reason a Venn diagram cannot count its shared region twice. Recognising this lets you reuse one mental tool in two settings, and it explains why the three-event formula alternates signs in the inclusion–exclusion pattern.