The Wisdom of Crowds

June 19, 2019 6:45 AM
  • Andrew Tottenham — Managing Director, Tottenham & Co
June 19, 2019 6:45 AM
  • Andrew Tottenham — Managing Director, Tottenham & Co

The UK does not have a Presidential system – we do not vote for a Prime Minister but for a party. The winning party forms the Government, and the leader of that party becomes the Prime Minister. And when a governing party chooses a new leader, as the Conservative Party in the UK is in the process of doing, the new leader automatically becomes the next Prime Minister.

Story continues below

Sir Francis Galton, a noted Victorian mathematician, developed a number of statistical concepts, including “correlation” and “regression to the mean”. Galton used then newly-developed statistical methods to study intelligence and heredity and to do research on what he named eugenics. Galton, with his fascination for statistics, was supposed to have stumbled upon what James Suroweicki called, in his book of the same name, “The Wisdom of Crowds”.

In the early 1900’s Galton attended a County Fair where people were asked to guess the weight of a full grown ox. To his surprise, the average of all of the answers came pretty close to the actual weight of the creature, although no one person came closer.

Galton hypothesised that if the guesses were from non-experts, people who genuinely had no idea about the weight of oxen, then there would be a standard distribution of guesses, with as many people overestimating the ox’s weight as underestimating it, and by similar amounts. Hence, the median of all the guesses would be close to the animal’s actual weight.

There is some debate as to whether to use the mean or the median of all of the guesses. Galton believed the median would give the best answer, although in this instance the mean was closer – a one pound difference from the actual weight as opposed to nine pounds.

What has Galton and the next Prime Minister got to do with gambling? The answer has to do with prediction markets. Prediction markets are nothing but a glorified name for betting. People place bets on the likelihood of a specific outcome of an event. In the UK, bookmakers can accept bets on more or less anything – the name of the next royal baby, whether it will snow on Christmas Day, or who will be the next Leader of the Conservative Party, for example.

For an election, people place their bets on who they think will win, which is equivalent to their guesses of what they expect the outcome to be. The prices (odds) change as people bet, reflecting the amount of money bet on each candidate. The more bet on one candidate, the shorter the odds offered for that candidate winning. At the time of writing, Boris Johnson is the clear favourite with odds of 1/6; the next candidate in line is Rory Stewart, with odds of 10/1, recently shortened from 20/1 due to a good performance in a TV debate. (Who is Rory Stewart? You may well ask.) If this is a reflection of what will be the actual outcome, then Boris Johnson will romp home and be the next Prime Minister.

Do prediction markets ever get it wrong? Of course they do, sometimes spectacularly. Favourites do not always win. When it comes to betting on political events, bettors often rely heavily on polls; if all the polls show one outcome as a clear favourite, more money will be bet on that outcome. Polling companies adjust their results to reflect what they think will be the make up of voters on voting day, more of an art than a science. By contrast, voters tend not to change their vote based on polls; in the privacy of the polling booth they vote the way they want. Nor do people always provide their actual voting intentions to pollsters.

The outcome of the Brexit referendum and Trump’s dominance of the Republican primaries are two cases where prediction markets failed miserably. On the eve of the Brexit referendum the polls had the Remain campaign winning, albeit by a small margin, and on the betting markets it was possible to get odds of about 10/1 on Brexit happening. Yet the Leave campaign won. Why didn’t the Wisdom of the Crowds work?

From a pollster’s perspective the actual Brexit vote was within most polls’ margin for error, but the betting markets were completely off. Experts have put this down to a number of reasons, mainly that the bets did not come from a random sample but from a smaller, fairly homogenous cross section of society with a Remain bias. Betting markets take account of the weight of money and move the odds accordingly, rather than using the number of bets each way. In other words, if one person bets £100 on Remain to win and another bets £1 on Leave to win, the opinions are evenly balanced but the money and therefore the odds offered are not.

What does this tell you? The Wisdom of Crowds can be a useful tool when you want to weigh an ox but not necessarily when you want to know the outcome of the next election.