Headline News?

The latest Gallup poll has Labour up two points since last month, the Conservatives down one, yet to judge from the Daily Telegraph's front page headline Friday (9.6.00) morning, its poll carries awful news for the government: "LABOUR'S LEAD OVER TORIES IS HALVED". What does that convey, knowing that Gallup polls for the Telegraph monthly and is published within a couple of days of the end of fieldwork? Surely that the government has suffered a catastrophic loss of support in the last month, and that this was the position as measured a couple of days ago. In fact, such an impression would be entirely untrue.

The first false impression created is over the timescale. One tends to assume, and anyone seeing the headline without reading the poll probably would do so, that an unqualified statement about the change revealed by a poll indicates the change since the last poll in the series; to headline the poll in this way when the change referred to is the change since this time last year is misleading, to say the least.

Furthermore, even accepting this, the change is exaggerated. The lead in May 1999 was 27 percent, so for it to have been halved it should have fallen below 13.5 percent; in fact the figures given for the lead in May 2000 were 14.8 percent. Not a huge difference, it is true, but poll reports should not exaggerate.

Worst of all, the poll figures cited were not only old news and not the most up-to-date figures available, but the more up-to-date figures directly contradicted the thrust of the story. Every month, Gallup produces two sets of figures: its "Political Index", data aggregated over the previous calendar month, and a "snapshot poll", over a single week. The snapshot poll is the newest data, and this is what is normally considered to be the new Gallup poll. In this instance, the Gallup Index consisted of just over 4,000 interviews on 3-31 May, the snapshot of just over 1,000 interviews on 31 May- 6 June. The Gallup Index figures are therefore for a period which has already been extensively polled for publication - twice by MORI, twice by ICM and once by Gallup. These polls all, like the Gallup Index, found the government's position deteriorating, though the last, MORI's for The Times [18-23 May 2000], found signs of an upturn after the birth of Mr Blair's baby. But the new Gallup snapshot, the first published poll in June, found the opposite: Labour up two points since last month, the Conservatives down one. The Telegraph headlined the old poll, not the new one, with the first mention of the snapshot buried eight paragraph's into Political Editor George Jones's front page article, and not covered at all in Professor Anthony King's commentary (though that was not as misleading it stood beside a table clearly laying out all the detailed figures).

Headlines can be one of the most irritating factors for pollsters when we work for the Press. We work closely with our clients to ensure that the initial reports are as accurate as possible; indeed, we make it one of our contractual conditions that we see and approve all copy to ensure that it faithfully reports the poll and does not mislead about the findings, even inadvertently or by implication. For example, Peter Riddell of The Times floats over to our offices faithfully for each and every poll, and joins a group of MORI analysts who gather together to go over the findings; following the conference, Peter's copy is faxed over to us and is gone through in word-by-word detail, and any qualifications or afterthoughts then dealt with by telephone. We work directly (and very happily) with the Times' graphics department, and check their artistic interpretations of our crude graphic suggestions to ensure the integrity of the data. This happy and symbiotic relationship has now existed for over 20 years, without acrimony, posturing or even serious disagreement over all that time (perhaps with the occasional twinge shared with Peter when a careless headline writer oversteps the mark, with an arresting 'GAP SLASHED' splash added to Peter's otherwise fully justified story.

There tend to be worse problems with the secondary reporting of polls. Here the story will usually be written up by news staff, rather than by a poll specialist. It will almost certainly be selective, simply for lack of space. Today's (London) Evening Standard picked up the Telegraph's story, mentioning the poll figures in the context of a forthcoming speech by the Conservative leader, William Hague, which will attack Mr Blair as arrogant and out of touch. What the Standard said was entirely accurate as far as it went - indeed, in some ways it was better than can often be expected, for it not only gave the "Political Index" figures but explained that they cover the entire month of May; failure to report fieldwork dates is generally one of the besetting sins of the secondary reporters. However, while the Standard reported the "Political Index" findings, it made no mention at all of the more up-to-date snapshot poll; anybody reading only the Standard's account would have no way of knowing that, far from a continuing decline in Labour's rating the sole polling indicator during the last ten days shows a slight upturn. Furthermore, although it did report in passing that the survey monitors opinion over a month and that the figures cited were for May, it also commented that "The survey was taken before Wednesday's Wembley fiasco [Mr Blair's speech to the WI]"; true, it finished over a week before that speech. But the impression on the casual reader must be that the poll measured the situation immediately beforehand.

But it is hard to blame the Standard for following the Telegraph's lead on the Telegraph's own poll, and there was nothing wrong with the Standard's headline (which wasn't about the poll anyway). The Telegraph has fewer excuses. It is rightly quick to attack the government when it announces the same spending increases several times to disguise the lack of new initiatives; yet this coverage was doing exactly the same with the poll figures.

And the moral is - the Press hypes its stories, it is in the nature of the beast. (Or, indeed, of The Beast).

• Rely on the text in preference to the headline, and in the tables with the actual numbers therein.
• Rely on the original report rather than the secondary reporting if possible. Secondary reporting (of which the MORI Research Review is an example, albeit we hope a good one) can only at best be a paraphrase, and accidents will happen. Use the secondary report to alert you to the poll's existence and, if you can, go back to the original.
• If there is table or panel giving fuller details, consult it - you are not forced to accept the journalists' interpretation of the figures or judgement of what is most important. If full details are not available in the newspaper, you should be able to obtain them from the pollster. (For all MORI's published polls, you can normally find details on www.ipsos-mori.com).
• Watch the share, not the gap. It is of much greater moment to see that the Conservative share is at 30%, or 35%, when they need to get to 40% to be in the frame, than to know that Labour has a 12% or a 7% lead, which more likely than not is accounted for by a change in the share for the Liberal Democrats.

Polling is, as far as we can achieve it, a scientific and objective means of measurement of people's views. Interpretation of the data, with the best will in the world, is another thing entirely. Trust the data.