Yes, that's a fine result but what does it really mean?

The face of public engagement is changing with more open consultation and fewer representative surveys being done. Both approaches are valid, but which one is the best approach for gathering the information that you need? Rose Neville discusses the benefits of both in her blog.

Yes, that’s a fine result but what does it really mean?

How public services engage with the public and find out what the public thinks has changed in recent years. Now everything quite rightly is being consulted on: from what to do with the BBC3 channel to whether we want to build the proposed old people’s home on my street. Sometimes it appears that almost regardless of how many responses are received to an open consultation (and the number can vary enormously) a lot of store is put by the results. This begs the question: is an open consultation always the best way to engage with residents?

Don’t get me wrong, engaging with the public is important and very worthwhile. Research into how residents want to interact on issues consistently shows a significant proportion who want to get involved and will respond to an open consultation. However, it also shows a large proportion who are less engaged but will give their views if asked. How do public services make sure the views of this silent majority are also heard?

Certainly the responses to a consultation can tell you the strength of feeling among those who respond. They can also suggest alternatives and new approaches which might not have been considered and provide valuable insight into how those most engaged feel about the issues being consulted on. What the responses can’t tell you is how representative the results are; for that you need a more structured approach to engagement.

  As a researcher, my training says that you need to have a representative sample of your universe (however, you define it) in order for the results to be representative and be robustly extrapolated to your universe. In research we spend a lot of time ensuring that the profile of who we talk to is balanced to reflect the profile of our universe. However, in a consultation – which by definition is an open invitation to anybody to respond – this is not the case. This can lead to very different results, a couple of examples are shown to the left.

However you choose to interpret these results, and both are valuable sources of information, it is impossible to say that the results and views they show are the same. The views expressed by those who respond to a consultation are much more polarised, and generally more negative, than those expressed by a representative survey. I’m not saying that one engagement approach is better than the other simply that public services need to be aware of what the results to their engagement are really representing and to decide whether they want to gather the views of those most affected (in which case an open consultation) or to understand how the overall population feel (which would need a representative survey).

Rose Neville is an Associate Director with Ipsos Social Research Institute in Manchester.

More insights about Public Sector