Online is the now, not the future, so why bother with face-to-face surveys?

Why don’t we just use online interviewing for all our surveys now? It was a question raised last week at Mediatel’s Future of Media Research seminar when Jens Torpe, from City A.M, called for the National Readership Survey (NRS) to abandon traditional face-to-face interview research in favour of web-based panels. It’s a good question, so did he have a point?

We all know that it is now the most popular method for collecting research data from people (according to ESOMAR). It’s cheap, easy to administer and, for many, a much more convenient way of participating in the research process in their own time, at their own pace.

It is certainly true that particular research methods work well or less well for particular tasks. If you want to study a well-defined group of people or those in a specific geography, for example, it may be possible to get to them via email ‘lists’ or by attempting to recruit them through websites they are known to frequent – in which case online may be a perfectly valid way to recruit them. Especially if, as Mr Torpe points out of his own title’s readers, they are less likely than the average person to be at home in the evenings and reside in a particular area of the country. This is true for any niche or specialist title.

But what happens if, in fact, you do want to speak to the ‘average’ person – or as good a cross-section of the population as a whole as you possibly can? Let’s differentiate between recruiting these people and collecting data from them – which can quite easily use different methods.

If you want to recruit a good cross-section of the whole population, there is no list of email addresses you can draw from that represents them all. There is no online panel with everybody in the country on it.

Sure, there are lots of people who have chosen to be members of online panels – they join them for a variety of reasons, including the desire to earn rewards of one sort or another, to express their opinions and so on. But are they representative of the whole population, a key factor when estimating audience figures?

In fact they tend to be heavier internet users than the average (after all, many are members of several such panels and spend a lot of time filling out surveys). And as such, they are typically heavier consumers of (particularly online) media.

That said, for many products and services, online panels can provide useful insight for relatively low cost. But when it comes to measuring readership of newspapers and magazines in all the forms in which they appear, will a heavy online user with time on their hands be representative of the whole population? Will titles as diverse as the Daily Telegraph, the Daily Mirror, Vogue and the Radio Times be read as much by these people as by those who have never joined an internet panel?

Because if not, then exclusively recruiting people online to take part in general readership surveys would surely be wrong. What about other methods? Could we recruit them by phone and ask them to fill out a questionnaire online? We could – and we do in certain countries - but again there are challenges. The rapid rise of mobile phone only households and in the use of answer machines for call screening has made telephone sampling perhaps less good than it was.

On the other hand, people's houses have stayed pretty much where they always were. Almost everybody lives in a private household and can, in theory, be located and recruited for research. The questions then become: will they be willing to take part in the research and is this affordable?

The NRS samples the population in a strictly random way beginning with household addresses – a far more accurate starting point than any list of telephone numbers or email addresses could ever be. We then find that half of a randomly selected section of the known total population – and half is pretty good - are quite willing to take part in research. The question of whether this is affordable is a separate one.

But from the point of view of ‘accuracy’ there is really no competition.

John Carroll is a Senior Director in Ipsos MediaCT. Follow John on Twitter @MediaCarroll.