Introducing our new voting intention methodology

Since 2008, our approach to political polling for voting intention had been based on a telephone quota sample using a mix of landline random digit dialling and mobile numbers. This approach worked well for many years, allowing us to accurately measure and understand public views of the political parties in Britain including being the most accurate prediction poll in the 2019 European and general elections. In 2024, we were pleased that our polling throughout the campaign helped tell the main story of the election, but the final prediction was not as accurate as it could have been. While our estimates of most parties were within +- 2 percentage points of the election result, we were disappointed that we underestimated the Conservative vote share by 5 points. Our analysis suggests that in large parts this was likely due to an unusually high level of refusals, especially among voters aged 65+. 

We believe this is the right moment to transition to a new methodology and have carried out our latest voting intention polling on our online random probability KnowledgePanel©. The KnowledgePanel© was set up in the US in 1999 and in the UK in 2020, and is now present in another 7 European countries. It is based on probability principles, with addresses randomly selected from the Postcode Address File and invited to join the panel. Those who are digitally excluded are offered a tablet and restricted internet access free of charge to enable them to complete surveys online.

What are the benefits of a KnowledgePanel© approach?

A probability panel approach brings several benefits:

• Gold standard probability sampling approach minimises self-selection bias, only randomly selected households can join, and randomly selected individuals can complete a survey;
• Postal recruitment, alongside supplying tablets to those who are digitally excluded, ensures all households have a known and non-zero chance of being included, with design weights accounting for unequal probabilities of being selected (e.g. individuals in larger households have less of a chance of being invited to a survey);
• Rich panellist profiling;
• Collecting immediate post-election data on voting behaviour, mitigating against false recall;
• Collecting and processing complex data such as social grade (coded by professional coders), or voting eligibility information.

Voting intention on the KnowledgePanel

Our pre-election test of our KnowledgePanel, replicating in large parts our previous telephone methodology, successfully estimated most party vote shares within the margin of error, but overestimated Labour and underestimated Reform UK.  Other than the obvious change of mode, since then we have made further changes to our method to improve our estimates:

1. Stratifying by past vote. Probability samples do not use quotas in the same way that opt-in surveys do. Instead, invited samples are usually ‘stratified’, meaning the sample is randomly selected to take part, but controls are imposed to ensure it is representative on key characteristics. Being able to collect voting information the day after the general election means we are able to stratify our sample on past vote collected very near the time, ensuring it is balanced in terms of 2024 vote, but mitigating against issues related to false recall.
2. Prompting for Reform UK. Up to and including the 2024 election, our voting intention question only prompted for the Conservatives, Labour, Liberal Democrats and SNP and Plaid Cymru in Scotland and Wales respectively, other smaller parties not prompted. Indeed, in 2024 our CATI methodology got the Reform UK share exactly right without prompting.  Since then, though, given the wide appeal of Reform UK, and reflecting that our KnowledgePanel slightly underestimated them, we are now prompting Reform UK alongside other major parties.
3. Updating our weighting scheme. Alongside our longstanding weighting scheme which we have developed over years of conducting political research (age and gender, region, ethnicity, qualifications, work status and work sector (public sector/other), social grade (office coded), number of cars in household), we are now also weighting on print and digital newspaper readership (our previous approach only included print papers) and constituency type (marginal/non-marginal), and have also updated sources used for some of the other targets (such as public sector workers).  The results of our 2024 KP prediction test at the time of the election, and the impact of the new weighting scheme specifically on that, is shown below.  This suggests the new weights improve the accuracy of the KP prediction to within the margin of error for all parties, though still showed a slight overestimate of Labour and underestimate of Reform UK, which our other changes aim to address.

Other questions

Finally, as we always say it is a mistake to focus too much on voting intention figures and ignore the many other important findings from a poll.  Along with voting intention, we have also asked our longstanding satisfaction rating questions on our KnowledgePanel survey, continuing trends going back to the late 1970s.  This also means we have to be aware of any mode effects moving from an interviewer administered survey to one done by participants online.  Fortunately our satisfaction questions are very straightforward, which helps to minimise mode effects, and indeed our testing suggests there is very little difference in the ratings of the government and Prime Minister (an average of about 1 percentage point). There is slightly more difference in the ratings of leaders of other parties (who tend to have higher don’t knows), with KP being on average 4ppts more negative to leaders of opposition parties from across the political spectrum. We will continue to monitor this, but in the meantime will make trend comparisons on ratings of government and Prime Minister, but not for leaders of other parties yet. 

Overall, we believe our KnowledgePanel is based on robust methodological principles, and the  improvements made since the 2024 election help reduce biases we saw at the time. It is still the case that all surveys are subject to sources of error, and rather than focusing too much on a single poll results should be interpreted in the round along with other sources of data. As always, we'll continue to review our new approach, and make more adjustments if necessary as we conduct more analysis.