Introduction
By Jean-Marc Lech
Il n’y a pas d’experts muets, c’était l’antienne d’Ipsos il y a peu. Et aujourd’hui, il y a urgence : les clients veulent de l’expertise, de la réflexion, des initiatives, de l’engagement, des idées.
Alors oui, les experts doivent s’engager, et mieux encore : il est urgent de se faire écouter.
Dans un orchestre de jazz, ce sont les cuivres que l’on veut entendre, et pas seulement la plainte de la contrebasse.
« There are no silent experts » was the refrain of Ipsos until recently. But today, it is even more urgent: clients want expertise, insights, initiatives, commitment and ideas.
Experts have to commit themselves and more than that, it is urgent for them to be heard. In a jazz orchestra, we want to hear the brass and not only the protest of the bass.
Top 5 articles
E-mail alert
REGISTER NOW
Access to archives
Bobby Duffy
Bobby runs the Ipsos Public Affairs specialism in the UK, which is made up of the Social Research Institute and the Reputation Centre. He joined MORI in 1994 from an economic policy think tank and has worked in social research ever since. He has particularly focused on major evaluations of government programmes, as well as policy development in public service reform. He also set up and ran a number of key initiatives, including the Research Methods Unit and Participation Unit, and produces many thought leadership reports. He writes and presents widely on public policy and political issues, and has been a User Fellow at the Centre for Analysis of Social Exclusion at LSE and spent time working in the Prime Minister's Strategy Unit.
Predicting elections using 140 characters or less...
The recent general election in the UK was the most polled in our country’s history – the public were bombarded with over 90 surveys during the four week campaign. And there have been nearly as many post mortem articles, working out who won and lost among the pollsters and methods (purely as an aside, and not at all to crow, the exit poll we helped run was called “the crowning triumph of the opinion research business”).
One particularly interesting debate is about the growing role of social media data as a serious alternative to predictive polls. Is there really a “wisdom of clouds” that can be tapped into, if we know how?
In the UK, an experiment by a political website using Twitter came up with a result remarkably similar to the final polls. It’s true that the experiment wasn’t a complete success – their most spectacular failure was predicting victory for a television personality standing for office when she was in fact beaten into fourth, with only 4% of the vote. Pre-existing fame (or notoriety) needs to be better controlled for.
The US, Germany and Japan were all ahead of the UK in trying similar experiments – and each came up with similar, encouraging results. The scale of data available makes the point: one US study analysed one billion tweets across the course of a year to look at presidential popularity and consumer confidence – and again this tracked the polls pretty closely. One billion of just about anything is almost bound to tell us something useful.
Of course there is much more to web listening than this type of surrogate polling – it would be missing the point to judge its usefulness on these tests alone.
Equally there are lots of research applications that web listening will never be appropriate for – and there are a number of issues to overcome for all uses.
Firstly, accurately measuring sentiment (whether what people are saying about organisations or issues is positive, neutral or negative) through automatic programmes is one of the main challenges. Recent reviews suggest that the accuracy claimed for the many programmes available may be spurious: the vast majority of online comments measured are neutral and relatively easy to categorise as such, but it is the positive and negative ones that really count – and here all tools tested are nowhere near the 70-80% accuracy claimed. But some do better than others – which suggests this is bound to improve as companies compete and natural language processing becomes more sophisticated.
The other major objection is clearly the representativeness of what we’re measuring. A tiny fraction of the population regularly tweet, and as a German experiment points out, 40% of all tweets they looked at were from 4% of twitterers – like any crowd, some shout louder and say a lot more than others.
But whatever the arguments, the proof of any research is whether it works, and this is why elections are such a crucible for the reputation of new methods – they provide high profile, verifiable results to compare our predictions against. The accuracy of online panel surveys in past elections helped give them a legitimacy that could otherwise have taken years to build up – it seems that social media analysis could benefit in the same way.
June 7, 2010