Lies, Damned Lies And Opinion Polls

What's one to do? Polls are being dumbed down daily, and no matter how hard I try, the polls' equivalent of Gresham's Law seems destined to drive out quality, to the detriment of the proper use of polls to support advocacy, illuminate debate, control demagoguery, and inform people what others are thinking.

The internet has become everyman's library, available virtually to anyone, anywhere, at a very small, if any, cost to the enquirer. Recently an old friend posted a critique of a poll published in Newsweek, which although carried out to a high standard as one would expect from the Princeton Research people, was reported in a shockingly sloppy way by a journalist who clearly didn't know the first thing about the understanding and reporting of public opinion (the name of the course I taught at City University's Graduate Centre of Journalism for ten or so years).

The friend who posted the critique, Marty Plissner, formerly the executive political director of CBS-TV News, one cracking expert on American politics and public opinion, wrote in the chat room of the American Association of Public Opinion Research that the September 29 issue of Newsweek carried a report of a poll by-lined by Laura Fording that (former General, now candidate for the Democratic nomination for President Wesley) "Clark may have only entered the race on Thursday, but he is already the Democratic front-runner. In a survey of 377 (sic) Democrats and Democratic leaners, he is said to be supported by 14% --'outpacing' Howard Dean and Joe Lieberman (12% each), John Kerry (10%) and Dick Gebhardt (8%)."

Anyone with a modicum of understanding of the limitations of opinion poll sampling knows at least that a sample of (say) 1,000 has a statistical reliability of plus or minus three percent, so that at its most basic level, all five candidates are within plus or minus three percent of the midpoint of the group, 11%. Clark, the 'frontrunner' with 14%, three points over, and Gebhardt, lagging, with 8%, three points under. And that would be with 1,000 interviewed, without any consideration of non-sampling errors.

In her article, she informed readers that in fact the error margin is plus or minus six percent. In other words, anyone falling between 17% and 5% could be said to be within the margin of error. In fact, even that 'fact' is wrong, and the statistical reliability is narrower than plus or minus six, but I needn't go into that unless there's an anorak out there who wants more detail, in which case, [email protected] will continue the arcanery.

The message to you in the PR business, and to journalists happening to read this for that matter, is that even a well conducted poll can be misreported.

But this was a well conducted poll, even if the sample size is much more thin than we would condone in this country. The Guardian's Duncan Campbell, reporting from Los Angeles (October 25) reported American's view of 'afterlife' reporting answers to such questions as found that 76% believed in heaven, and 30% seeing it as 'an actual place of rest and reward', based on a 'survey' of 'around 100 people' carried out by the Barna Research Group in Oxnard, southern California (their web site indicates that they provide 'marketing research services relating to American cultural trends and the Christian church'). Campbell, you've been had. Even worse, why did any editor approve running such nonsense?

I recall in the general election of 1979, my client then, Derek Jameson, published a 'poll' of 500 blacks conducted by Sultan Mamood and his team of immigrant researchers over half of whom were under 35, asking such questions as 'Do you intend to live in Britain forever?'. When I tackled Derek about this nonsense, he response was 'Makes a good story, don't it? Besides, it only cost me 500 quid.' With such editorial irresponsibility, what hope is there for any quality standards in polling?

And I used to fight with the Manager of LBC who ran phone polls asking various questions; one participant is one of the LBC phoney polls wrote to the London Evening Standard saying how glad his side 'won', as he'd voted 157 times himself.

Now everybody is doing it, the New Statesman, never reporting how few people participate, the BBC, whose own guidelines warn against it, most of the tabloids and even some of the broadsheets, not to mention Sky's daily feeble effort. Recently the Kent Messenger, Britain's leading weekly newspaper, reported that 'of the 317 people who have voted so far, 256 (80.8%) opposed... Not 80%, 80.8%! Mathematically correct, statistical garbage. Any poll carried out to a decimal point suggest spurious accuracy.

What's this got to do with the PR business?

Recently too many PR agencies are on the game, trying to peddle cheap shots of spurious polls based not on proper sampling and proper question construction, but on leading questions of internet or other self-selecting 'samples', which my friends on the newsdesks say get binned, pronto. And the flow now on the nationals is more that a dozen of these a day. My suggestion, self seeking though it may be for my own firm, is either do it properly or don't do it at all, otherwise, you'll be considered by the better and more discerning media as the PR equivalent of spammers.

Remember, to be credible, a poll has to ask the right sample, the right questions, and add up the figures correctly.