Reading The Polls

The British pollsters for some years have followed the Gallup method of determining voting intention, asking a two part question, "How do you intend to vote in the General Election on 7th June?", and leaving aside those who say at that point they would not vote, asking the remainder, usually about 90%, "Which party are you most inclined to support?" (asked of those who say they are undecided or refuse to say for which party they would vote for). As a frame of reference, figures of 30% undecided, 10% would not vote and 3% refused would not be unusual at the first question, and when asked the second, the undecideds drop to 10% and the refusals to 2%.

These "don't knows" are then dropped out of the final sample for this question, and the figures are repercentaged to give the final result. We are often asked, even accused, of ignoring the "don't knows", but every article in the monthly series of MORI polls for the Times since its inception over 20 years ago has reported these figures. The theory in dropping the don't knows is that for the most part don't knows don't vote, and that it is as good an approximation as any other in estimating what public opinion is at the time the interviews were conducted.

Following the Market Research Society's Review following the 1992 British General Election when the pollsters were castigated for their error in "forecasting" (sic) the result, some polling organisations have attempted not to measure public opinion at the time the interviews were conducted, but to forecast how people will vote in a hypothetical election some weeks or months or even years ahead before the general election is called, or once the Prime Minister does set the date, to forecast weeks and then days ahead what the outcome is likely to be.

I have often said that polling is a simple business, and that all you have to do is ask the right sample the right questions and add up the figures correctly. For fifty years and more, the questions asked were more or less the same by all polling organisations, the sampling was more of less the same type of quota samples or taking names at random off the electoral role, and the figures were added up in the way outlined above.

It isn't like that any more.

Before the invention of the Social Democratic Party, the voting intentions questions were asked as the fourth or fifth question(s) in the questionnaire by Gallup, Marplan (the predecessor to ICM, although Marplan is still in existence as a market research company, but does not do political polling), NOP, Harris (originally ORC) and MORI following after leader satisfaction and issues questions. When the SDP was established, empirical tests showed that asking about satisfaction with the party leaders, including David Steel for the Liberals and David Owen for the SDP, boosted the SDP's subsequently asked share of vote by c. 3%.

Marplan, polling on the telephone in the 1983 general election, continued to ask the voting intention question(s) in the traditional fifth place, but the others polling in that (1983) election started asking voting up front (somewhat to the consternation of interviewers) in both face-to-face interviews and on the telephone. In the election that year, the final polls by Gallup, Harris, MORI and NOP said they expected the SDP to get 26% of the vote; Marplan 29%. On the day, the SDP got 26%.

In the MRS Review a great deal of methodological testing was undertaken, including an examination of telephone v. face-to-face, one-day polls v. multiple day polls, all weekend interviewing v. weekday interviewing, etc., and there was no statistical difference between any of these methodologies.

Since then all polling organisations other than MORI have moved to telephone polling. Only MORI continues to interview in home, face-to-face, now using CAPI (Computer Aided Personal Interviewing), in its polls for the Times, and undertakes telephone polls for its other media clients.

In the polls since 1 November, during the 'flat-lining' period, 36 polls were conducted up to the calling of the election. The average voting figures from all these polls stands at 32% for the Conservatives, 48% for Labour, and 14% for the Liberal Democrats, a Labour lead of sixteen points. Over these six months, only one poll (Gallup) reported a figure for the Conservatives outside the +3% range, at 27%, six polls put Labour's lead below 45% (three points below the average; ICM, the Tories' private pollsters, the lowest) and four polls put Labour's lead above 51% (three points above the average; MORI and Gallup, at 53%, the highest and one the same week). The Liberal Democrat share was put outside +3 percentage points of their average in one poll (ICM).

During this election however, ICM's methodology and reporting practice has become clear, involving asking the voting intention question(s) differently than Gallup, MORI and NOP, and it now appears that their reported figures are weighted differently that the others have been, and they are reporting not from the total sample expressing a voting intention, which the rest have up until the start of this election period, but also have been reporting on those they think most likely to vote.

Gallup has now also changed the way it asks its voting intention question(s), away from the traditional open-ended way, so that it now lists the names of all the parties, and also asks respondents how they intend to vote in their own parliamentary constituency. They have also joined ICM in reporting on those they think most likely to vote, which they derive from positive answers to a number of questions.

NOP continues to ask voting intention in the traditional open-ended way, without reading out the names of the parties, as does MORI, and neither make any reference to voting 'in your constituency'.

MORI meanwhile is clearly out of line with some others' findings in this election, with Labour some four points higher, the Conservatives 3-4 points lower, and the Liberal Democrats 2-3 lower than ICM, but in line with both Gallup and NOP.

The new entrant, Rasmussen Research, polling by computer speaking clock for the Indy, gave figures of 32% Conservative, 46% Labour, and 13% Liberal Democrat, on their poll of 12/13 May. Their automated system conducts an interview with whoever answers the phone, so long as they are aged 18+. They also attempt to filter out likely voters, using screening questions on voting history, intention to vote, and 'other matters'. After weighting (typically, they interview too many women and older people), they conduct a 'sanity check' to make sure their data is comparable with other results. As I mentioned last week, Rasmussen Research is an American outfit, and in the 2000 American Presidential election was 15th out of the 16 polls reporting in the final week of the campaign.

There is also the British Elections Survey, so far unpublished, which can be found on the www.essex.ac.uk site The BES asks voting intention in questions 6 and 7, after asking first interest in the election (currently a quarter "very" interested, 40% "somewhat" interested, a third "not very/at all" interested), the single most important issue in this election, which party they think best able to handle this issue, likelihood of voting (using an 11 point scale from 0 to 10), if the respondent has decided (38% say they are undecided and 2% say they won't vote) and then they ask: "Which party will you vote for?" of those who say they are decided, and then "Which party do you think you are most likely to vote for?"

The BES is also using a controversial sampling technique, the "rolling poll", having Gallup ask approximately 150 interviews daily and aggregating them into seven/eight day samples of c. 1,100 people reporting daily, dropping out the earliest results as they add in the latest. Thus their reporting is on average four days earlier than a 'quickie' poll such as the MORI survey for the Times, done on the Tuesday and reported on the Thursday (Wednesday night on the 7:00 Channel 4 News and on the 10:00 BBC and ITN news broadcasts).

The BES latest results have the Labour Party on 54%, the Conservatives on 29% and the Liberal Democrats on 12%.