Predicting 2015 - "just a bit of fun"

This is strictly what Peter Snow used to call "just a bit of fun". As we say over and over again, opinion polls do not predict the future, they are only a measure of the mood at the moment. (Until the election campaign has started, we don't ask people how they think they will vote when the election comes, but how they would vote tomorrow.)

But, of course, you wouldn't expect the opinion poll findings, even early in a parliament, to be completely unrelated to what will eventually happen at the ballot box. The mood of the nation now is probably one of the factors that will contribute to the result of the election, even if there are plenty of "events, dear boy, events" to come that will make it less predictable. If you had to make a prediction of the next election result now, you would certainly want to take into account what you know about public opinion. Judging from the past, are the poll figures at the moment good or bad for the government?

Which poll figures, though? Is what people think of the parties a better guide to how they will vote in the end or, in this age of personality politics, is it what they think about the leaders that matters most? For many years we have been tracking voting intentions regularly, and also whether the public are satisfied with the way the party leaders are doing their jobs. We got to wondering which of these two series of figures is more useful for predicting what will happen at the next election. And so we went off to find out.

Using all Ipsos's and MORI's monthly polls dating back to the start of Mrs Thatcher's first term in 1979, we tried predicting by how much the Tories would win or lose the next election by using the leader satisfaction scores and also by using the voting intention polls, and we compared the two to see which was best. We split the whole exercise into three: we looked separately at polls taken at least 3 years and 5 months before an election (i.e. up to the same point in previous parliaments as we have reached in this one, assuming a May 2015 election), at those in the two years following that, and those in the last 16 months before an election.

However, we didn't simply assume that a poll lead of x percent now should be taken as a prediction of an x percent lead come election day. We got the computer to look for patterns in the previous data and produce the equation which, by plugging in all of the poll findings we had, would have given us the best prediction of the election result; and, since it might be pretty silly to assume that the meaning of a poll five years before an election was the same as one five weeks before, we also fed the passage of time into our calculations. This is what statisticians call "building a model". (For the technically minded, what we used was simply an ordinary least squares linear regression, one of the most basic statistical techniques for estimating the relationship between two or more series of figures.)

And what we found was this: in the early days of a parliament (up to the stage we have now reached), the satisfaction scores produce a much more effective prediction than do the voting intentions. However, in the middle years of the parliament (between 1 year 5 months and 3 years 4 months before the election), there is not much to choose between the two sets of figures as predictors, while in the last 16 months the voting intention figures are significantly better. In fact, voting intentions get steadily better as a predictor as time goes on, while the satisfaction scores are barely a better guide in the last few months than they are in the first few. This probably shows, perhaps as you would expect, that people form their views of the leaders early and these are important, but other factors are also important and they are prepared to change their minds about which party they support at any time if the circumstances warrant it. It is also worth noting that while satisfaction scores and voting intentions are both reasonable predictors on their own, the prediction is better still when you use them together.

Some important cautions, though. This really is just a bit of fun. It is based on far too little data to take it very seriously. With only 7 general elections in the mix, anything unusual about one or two of them might be given far too much weight and come out looking like a hard and fast rule when it's nothing of the kind. To take just one example, the three elections over this period where the parliament ran the full five years all featured a government in trouble. That's why there was a fifth year – John Major in 1992 and 1997 and Gordon Brown in 2010 didn't fancy an early election. But this time round the rules have changed, and – in theory, at least – we'll have to wait the full five years for the election even if David Cameron would rather go to the country after four.

There's another important point. The figures show that in the past we'd have been able to make even better predictions if we took into account support for the Liberal Democrats or their predecessors and the ratings of their leader. But of course in all the periods we have been able to look at, they have been a second opposition party; in this parliament, they are a second government party. That potentially changes the meaning of all the figures, and might easily mean that the signs which were a good precursor of the election outcome in the past won't be this time. In fact, that's essentially the drawback of most methods of prediction, whether of election outcomes or anything else – they depend on the future resembling the past, and it has a nasty habit of unexpectedly ceasing to do at the most inconvenient moments!

But, naturally, what you really want to know is this: what does our data actually predict now? Well, using the most powerful model we found (combining the Cameron and Miliband satisfaction scores with their parties' voting intention ratings, averaged over the last 3 months), and assuming that nobody finds a way round the new Fixed Parliament Act and the election really comes in May 2015, the Conservatives will win by 3.5 percent of the vote. If that's the way it comes out, you heard it here first. And if it isn't, we'll deny everything…