Winston Churchill told the king he expected to win. It was July 1945 and the prime minister — two months after declaring victory over the Nazis — believed a Tory victory at the polls, perhaps with an 80-seat majority, was within reach. His Labour opposite, Clement Attlee, agreed, as did the public, who expected a Conservative win by 54 per cent to 38.
The result, of course, was a shock 146-seat landslide win for Labour and the humiliation of Britain’s wartime leader. Yet it was no shock to the readers of the News Chronicle: a day before voting the newspaper had published an “interim forecast” — its polite word for a poll — suggesting a six-point Labour lead.
The newspaper was so apprehensive about the results that they were heavily caveated. “It is impossible to base upon these results any forecast as to the probable distribution of seats in the new House of Commons,” it said. It needn’t have worried: this was British polling’s breakthrough moment.
Gallup, the American firm that carried it out, had already outfoxed The Literary Digest magazine’s poll of 2.3 million readers in the 1936 US election. Using a sample of just 50,000, Gallup had correctly predicted Roosevelt’s landslide victory and had proved that, when it comes to polling, size doesn’t matter — it’s how you use it.
We learn about this golden rule and many more in Polling UnPacked, a comprehensive yet surprisingly fun overview of modern opinion polling. Its author, Mark Pack, has served as president of the Liberal Democrats since 2020, but has been involved with the party long enough to recall polling Cleggmania.
Advertisement
He is also a self-confessed geek, knowingly responsible for 28 per cent of all the reports of graffiti on bus stops to Transport for London in a single year. His book is essential reading for anyone seeking to understand modern politics and the basic question it sets out to ask is: can we trust the numbers that gauge the public mood, launch politicians to stardom and consign careers to failure?
Polling has had a mixed decade, and Pack does not shy away from a post mortem of some of its greatest failings: Harold Wilson’s surprise defeat in 1970, John Major’s victory in 1992, and the annus horribilis of 2016 in which pollsters missed Brexit and Trump. Many of these problems, we learn, are down to pollsters not reaching enough of the right people, whether they be “shy Tories” in 1992 or the white working class in Trump’s America. These fine margins make big differences. Pollsters did not see Cameron’s 2015 majority coming, partly because they asked too many people in their late sixties and not enough of those in their seventies and eighties.
Yet polls overall are not as inaccurate as we think. Between 1942 and 2017 the average polling error — the gap between figures in the final pre-election polls and what the parties or candidates actually secured — was just 2.1 percentage points.
Some of Pack’s excuses for pollsters’ failings will not satisfy everyone. The final polls from eight different companies gave Remain an average of 52 per cent — four points above the actual return. Based on those figures, Pack points out, we should have expected Remain to lose about one time in six. Although “one time in six certainly feels high if you’re playing Russian roulette,” he concedes.
Yet there is a greater problem that pollsters have to contend with: people don’t know what they think. In 2018 Michael Ashcroft, a former deputy chairman of the Tory party, showed how changing the wording of a poll can produce strikingly different results. When asked “Do you want a second referendum on Europe?”, just 38 per cent were in favour, and 51 per cent against. Yet phrasing the alternative as “no deal” flipped the result: when asked “Once Brexit negotiations are complete would you support or oppose holding a referendum on whether to accept the terms, or leave without a deal?”, more people were in favour of the referendum (39 per cent) than against it (31 per cent).
Advertisement
Context matters. Leading questions can manipulate answers and even get people expressing opinions on things that aren’t real. In 2013 YouGov found that 9 per cent of people were in favour of repealing the fictitious 1975 Public Affairs Act. When Labour voters were told that “Labour politicians” had called for its repeal, they were even more likely to support it.
This isn’t really a problem with polling — rather, it shows the “fragility of public opinion in the first place”. The ballot paper ordering phenomenon, where the first name on a two-person ballot is 0.6 per cent more likely to win, has even led some countries to explore circular ballot papers.
Polls, though, are getting more sophisticated. Take the multilevel regression and poststratification (MRP) polls, which rose to prominence in 2017; YouGov’s one correctly predicted that Theresa May’s Tories would lose their majority.
Instead of directly polling every single constituency — which would require a mammoth sample of 650,000 voters to do it properly — MRP splits us up into types of people. How likely are fiftysomething men who work in former industrial towns in the Midlands to vote Tory? The method then works out how many of each category live in each constituency.
It is complicated, and not always better. Ashcroft’s 2017 MRP poll wrongly predicted a huge Tory majority of between 162 and 180 seats. Yet generally they get it right: US pollsters predict the winner 79 per cent of the time. In the information age, amid a deluge of survey results, data points and propaganda, Pack’s book is a timely manual for those wanting to spot good polls from bad ones.
Advertisement
Polling UnPacked: The History, Uses and Abuses of Political Opinion Polls by Mark Pack
Reaktion £15.99 pp288
