The polls are all over the place...

Or are they? This is the first of a series of monthly Vox Populi blogs which my colleagues at Ipsos have asked me to write reviewing the opinion polls between now and the next election from my experience of nearly 40 years conducting and observing polls and how they are reported. Let's look at the record, as the politicians say. Just this past fortnight we've seen two polls early in August, ICM in the Sunday Express and BPIX in the Sunday Mail, both being conducted over the same few days, yet one had a 15 point lead for the Conservatives (ICM) and the other had a 24 point lead (BPIX). They can't both be right I hear the poll pickers say. Or can they? Then over the same identical days in mid-month, 15th-17th, ICM for the Guardian had 15 point Tory lead again, while Ipsos had 24, published by the Press Association. Well that proves it, doesn't it? Or does it? No, it doesn't, if you look at the share, not the lead. For polls are subject to the laws of statistics (as any statistician will tell you, if you ask them), and polls using properly designed samples, and we all do, operate within a so-called `confidence interval' of plus or minus three percent for a sample of c. 1,000 people. And the four polls above, plus the two done by YouGov in between them, are all, six out of six, within a three point `margin of error' of the average share for the Conservatives, Labour, Liberal Democrats, and Others. That's as good as it gets folks, and if you double the sample size, you don't improve the accuracy much. And that holds whether you're talking about a poll in America's 240 million voters, Britain's 45 million, or Trinidad's 1.3 million or even the Maldives' 200,000 electorate, holding their first ever contested presidential election this coming month. Hard to believe. Yeah, I know, but that's the way it works. Still don't believe me? Take all the polls in the first quarter of 2008. There were 23 polls published by one or another of the seven pollsters making the running. The Tories? 23 out of 23 were within the three percent margin as were the Liberal Democrats. Labour? 20 out of 23, as were the collective others. So in the first quarter of this year, 93% of the 92 party shares were within the plus or minus three percent margin. And what does statistics tell us? That the +/-3% rule you can expect to work 95% of the time if done at the same time, and these 23 polls covered three months! Ok, a fluke? What about April? 92%. May 84% (not too good, shape up pollsters). June? 94% (that's better). July? 97% (wow). August so far? 100%! Six polls out of six hit the average for the month (to date), the Tories at 46%, Labour 27%, Liberal Democrats 17% and Others 11% within plus or minus three percent margins. So why oh why do most folks believe the polls are all over the place. Some folks don't believe in them at all, any more than Quiji boards or reading tea leaves. Another reason is that most folks aren't statisticians. Fair enough, but most of those commenting on the polls in the newspapers, on radio and television, and certainly many who comment on them in the blogs and their responders, don't really understand how they work, why those of us in the trade believe in them (otherwise we wouldn't bother doing them, as nobody makes much money on them and they divert us from doing more profitable business). And if you're wondering, of the 64 polls published since the beginning of the year, the accuracy score is a credible 93.5%. Do the weather people do so well? Punters? Do you? If you want the numbers, email me, [email protected]. I'd be happy to send the evidence winging back. The full table is on our web site, and we'll be keeping a running total from here on into the next election.

Keep taking the polls...

Bob

Ipsos interviewed 1,005 British adults 18+ by telephone 15-17 August 2008. Data were weighted to reflect the adult British population.

Please click here to view Ipsos August 2008 Political Monitor