The law of large numbers: why the frequency settles at p

Law of large numbers

A coin has no memory, so why does the share of heads close in on one half as the tosses pile up?

Three recorded runs of 1000 coin tosses (or die rolls) wander at first and then squeeze towards p. Out of 1000 runs, the board counts how many are still more than 0.05 away at each n, and compares that with the bound p(1 − p)/(nε²) derived from the variance.

High school
Frequency and probability
Variance of the mean p(1 − p)/n
Chebyshev's inequality
Gambler's fallacy

\text{trial}

coin, p = 1/2

six, p = 1/6

\text{step}

P(|\bar X_n - p| \ge \varepsilon) \to 0

$n$: 100
$\bar X_n\ \text{(blue run)}$: 0.5500
$\sqrt{p(1-p)/n}$: 0.0500
$\text{share of runs off by} \ge 0.05$: 0.382
$\dfrac{p(1-p)}{n\varepsilon^2}$: 1.000

measured share ≤ boundyes

Guided tour

1Ten tosses tell you little
After $10$ tosses the three runs read $0.7$ , $0.3$ and $0.4$ . Out of $1000$ runs, $73.1\%$ are still $0.05$ or more away from $\tfrac12$ . The bound $\frac{p(1-p)}{n\varepsilon^2}=10$ is true but says nothing yet.
2A hundred tosses
At $n=100$ the typical distance $\sqrt{p(1-p)/n}$ is $0.05$ , and $38.2\%$ of runs are still off by that much. The bound has come down to $1$ .
3A thousand tosses
At $n=1000$ the runs read $0.526$ , $0.514$ and $0.49$ , and none of the $1000$ runs is $0.05$ away. The bound guarantees at most $10\%$ ; reality is far better. The spread shrinks like $\frac1{\sqrt n}$ : ten times the tosses, about a third of the wobble.
4Same law for a die
Count sixes instead, $p=\tfrac16$ . After $1000$ rolls the runs read $0.161$ , $0.173$ and $0.158$ , all within $0.05$ of $0.1667$ , and the bound is $0.056$ . A smaller $p(1-p)$ means a narrower funnel.

Explanation

1
What the law says
Repeat a trial with success chance $p$ independently, and let $\bar X_n$ be the share of successes in the first $n$ . For any margin $\varepsilon>0$ ,
$P\big(|\bar X_n-p|\ge\varepsilon\big)\to0\quad(n\to\infty)$
It does not say that any particular run reaches $p$ , only that the chance of being far from it can be made as small as you like.
2
Step 1: the mean wobbles less
From the binomial chapter, the number of successes has variance $np(1-p)$ . Dividing by $n$ divides the variance by $n^2$ :
$D(\bar X_n)=\frac{np(1-p)}{n^2}=\frac{p(1-p)}{n}$
So the typical distance from $p$ is $\sqrt{p(1-p)/n}$ , the dashed funnel on the board ( $\pm2$ of it).
3
Step 2: Chebyshev's inequality
The variance is the average of $(\bar X_n-p)^2$ over all outcomes. Throw away every outcome within $\varepsilon$ of $p$ ; each remaining one contributes at least $\varepsilon^2$ . So
$\frac{p(1-p)}{n}=D(\bar X_n)\ge\varepsilon^2\cdot P\big(|\bar X_n-p|\ge\varepsilon\big)$
which rearranges to
$P\big(|\bar X_n-p|\ge\varepsilon\big)\le\frac{p(1-p)}{n\varepsilon^2}\to0$
That is the whole proof. The board compares the bound with what $1000$ simulated runs actually do: the bound always holds and is usually generous.
4
No memory needed: the gambler's fallacy
After five tails, heads is no more likely; the coin does not compensate. The share still settles because old surprises are diluted, not corrected: $5$ extra tails are a large share of $10$ tosses and a negligible share of $10\,000$ . The difference between heads and tails typically grows like $\sqrt n$ , while the share converges.
The runs on the board are a recorded pseudo-random simulation with a fixed seed, so every visitor sees the same three runs and the tests can re-run them.

Check yourself

Q01

A fair coin has landed tails $5$ times in a row. What is the chance the next toss is heads?

Q02

To halve the typical error of a sample proportion, the sample size must be:

Q03

For a fair coin, $n=500$ and $\varepsilon=0.05$ , what does Chebyshev's inequality guarantee?

Keep exploring

Turn this lab into a videoUse this lab as the script for a narrated explainer you can share. The prompt below goes to the video creator, where you can edit it first.

Make the video

View · display