On the sample size and margin of error…

It’s finally out! One of the very first polls of likely Republican Iowa caucus-goers about their 2012 presidential preferences. Romney’s in the lead! Bachmann is surprisingly close behind! What happened to Pawlenty!? We’re about to be inundated with polls like this, so here is one important thing to keep in mind:

First, check out the sample size and margin of error. Larger sample size = smaller margin of error = more accuracy. A good poll will have at least 1,000 respondents in the sample which is equivalent to about a 3% margin of error.

When a poll says that there’s a “3% margin of error”, it means that there’s a 95% chance that the actual value in the wider population for whatever statistic they’re giving you is within 3% above or below that figure. For example, something like “Obama’s approval rating is 48%, margin of error 3%” means that there’s a 95% chance that his real approval rating among the entire population of interest (usually the entire United States unless they specify otherwise) is somewhere between 45%-51%.

That’s why I chuckle a little when headlines that say something like “Romney leads in new Iowa poll”. According to the Des Moines Register article, Romney had support of 23% of likely GOP caucus-goers, followed by Bachmann at 22% and Cain at 10%, with a 4.9% margin of error. This means that Romney’s actual support could be anywhere from 18.1% to 27.9%, Bachmann anywhere from 17.1% to 26.9%, and Cain anywhere from 5.1% to 14.9%. In other words, it’s possible (although unlikely), that Romney and Cain are only about 3% apart!

Bottom line: at this stage, polls are helpful for spotting trends and patterns, but little more can be made of them with any degree of confidence until the caucus time draws nearer and larger poll samples are taken. Most news reporters are pretty good at explaining this, but many politicians and pundits are not.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s