Margin of error: the plus-or-minus
What '52%, plus or minus 3' really means, why a thousand people is usually enough, and when a lead in a poll is no lead at all.
Margin of error (MoE)
How far a poll's result could be from the true value just because of which people happened to be picked. The result plus and minus the margin gives a range called a confidence interval: the values the truth probably lies between.
"52% ± 3" means the true figure is probably somewhere between 49% and 55%.
Check yourself
A poll finds 40% support for a new park, with a margin of error of ±4 points. Where does the true level of support probably lie?
- Exactly 40%
- Between 36% and 44%
- Between 38% and 42%
- Anywhere from 0% to 100%
Show the answer
Between 36% and 44%
Right. 40 minus 4 is 36, and 40 plus 4 is 44. That range is the confidence interval.
Step through it
A poll of 1,000: 52% to 48% Two bars on a scale from 40% to 60%: candidate A at 52% and candidate B at 48%, from a poll of 1,000 people (n = 1000). On bars alone, it looks like A is ahead.
Add ±3: the ranges overlap Lilac whiskers show the margin of error, ±3 points: A could really be anywhere from 49% to 55%, and B from 45% to 51%. The orange band from 49% to 51% is where the two ranges overlap, so this poll can't tell who is ahead.
Ask 4,000: the margin shrinks to ±1.6 The label changes to n = 4000 and the whiskers shrink to about ±1.6 points: A from 50.4% to 53.6%, B from 46.4% to 49.6%. The orange overlap band is gone. Now the gap between them is bigger than the noise.
Four times the people, half the margin A note on the right sums it up: n × 4, an arrow, then MoE ÷ 2. Four times the people buys only about half the margin, which is why most national polls stop at around a thousand or two.
Check yourself
A poll of 1,000 people: A 52%, B 48%, margin of error ±3. Can the newspaper say A is ahead?
- Yes, 52 is bigger than 48
- No, the ranges overlap, so it's too close to call
- Yes, but only by exactly 4 points
- No, because the poll should have asked everyone
Show the answer
No, the ranges overlap, so it's too close to call
Right. A's range (49 to 55) and B's range (45 to 51) overlap, so the true order could be either way. Reporters call this a statistical tie.
WHAT "PROBABLY" MEANS
95% confidence
Margins of error are usually worked out at 95% confidence. Picture running the same poll, the same way, a hundred times, each with a fresh random sample. Each run gives its own result and its own range. About 95 of those 100 ranges would contain the true value; about 5 would miss it. So a single poll's range is very likely, not certain, to hold the truth.
If 20 honest polls are published in an election season, it's normal for about one of them to be off by more than its stated margin, just by chance.
Rule of thumb for a simple random sample
(95% confidence, results not far from 50%):
margin of error ≈ 1 / √n
n = 100 → 1 / 10 = 0.10
n = 400 → 1 / 20 = 0.05
n = 1000 → 1 / 31.6 ≈ 0.032
n = 4000 → 1 / 63.2 ≈ 0.016n = 100 → about ±10 points
n = 400 → about ±5 points
n = 1000 → about ±3 points
n = 4000 → about ±1.6 pointsGoing from 1,000 to 4,000 people costs four times as much and only halves the margin. That trade-off is why most national polls stop at around 1,000 to 2,000.
Check yourself
A pollster's usual sample of 1,000 gives about ±3 points. She wants about ±1.5. Roughly how many people does she need?
- About 1,500
- About 2,000
- About 4,000
- About 10,000
Show the answer
About 4,000
Right. Halving the margin takes four times the people: 1,000 becomes about 4,000.
Check yourself
A poll of 1,000 people works for a small country, but a big country needs something like 100,000.
Show the answer
False
False. Once the population is much bigger than the sample, its size barely matters. A random 1,000 gives about ±3 points in a country of 5 million or 300 million alike.
Reading the small print
- The margin on the gap between two candidates is roughly twice the stated margin. With ±3, a 4-point lead is inside the noise.
- Subgroups have bigger margins. If only 100 of the 1,000 were aged 18 to 24, their figure is about ±10.
- A 1 or 2 point change between two polls is usually not real movement.
- A bigger country does not need a much bigger sample.
Check yourself
Which of these does the margin of error NOT protect against?
- Random chance in which people happened to be picked
- A sample drawn only from people who own landline phones
Show the answer
A sample drawn only from people who own landline phones
Right. Random chance is exactly what the margin measures. A landline-only sample is biased: it leaves out whole groups, and no plus-or-minus accounts for that.
Check yourself
- Week 1: a random poll of 1,000 people puts Party X at 44%, margin of error ±3.
- Week 2: a new random poll of 1,000 people, same method, puts Party X at 46%. A headline reads "Party X surges".
What is the fairest reading of these two polls?
- Party X has clearly gained 2 points of support
- The 2-point change is well inside the noise; support may not have moved at all
- The second poll must be biased
- Party X has lost support
Show the answer
The 2-point change is well inside the noise; support may not have moved at all
Exactly. Each poll could be about 3 points off by chance, so a 2-point change between two of them is what you'd expect even if nothing changed. "Surges" is reading noise as news.
Lesson recap
- The margin of error says how far a poll could be off just from who happened to be picked: 52% ± 3 means probably 49% to 55%.
- At 95% confidence, about 95 of 100 such ranges would contain the true value.
- With A at 52% and B at 48%, ±3, the ranges overlap: too close to call. At n = 4000 the margin is about ±1.6 and the overlap disappears.
- Four times the people buys half the margin, and the population's size barely matters.
- Subgroups have bigger margins, and the margin never covers bias or bad questions.