The other victors

How number crunchers won the US election

The long months of campaigning in the US saw a clash between two schools of political forecasting. Like the elections themselves, it ended with a decisive victory. The winners were several groups of apolitical number crunchers who predicted results with astonishing accuracy and in detail, relying on similar models using public data from opinion polls. All the number crunchers consistently predicted an Obama victory, with daily updates on margins and details. The most prominent was former baseball statistician Nate Silver. Mr Silver called the exact Electoral College (EC) split of Mr Obama’s victory with correct results in all the states. This improved on his 2008 predictions, when he got 50/51 right. He was also correct in 36 out of 38 Senate 2012 predictions. Mr Silver’s blog – FiveThirtyEight (the number of votes in the EC) – is hosted by The New York Times, which is the sixth-most popular site on the internet. On election eve, it drew over 20 per cent of all traffic to the Times website. Political science professor Drew Linzer of Emory University and neuroscientist Sam Wang of Princeton were as accurate. Professor Linzer even got popular vote margins correct. Also, Professors DeSart and Holbrook at Utah Valley University as well as the betting market Intrade came quite close.

Their methods could be replicated by anyone with an internet-connected computer, given the rich data from American opinion polls. They weighted averages of state-wide opinion polls (in Mr Silver’s case, adding data like employment numbers). Their models were constructed using standard tools of probability theory to run multiple simulated elections. The simulations gave them probabilities of success, which they published and updated as new opinion polls were incorporated into the models. Their success starkly contrasted with the failure of traditional political experts, who relied on gut feel and intimate knowledge of the personalities and arcana of American politics. Some talking heads said the elections were too close to call. Others, mostly on the right, were openly partisan, cherry-picking numbers to suit narratives. Some even derided the methods employed by the number crunchers and indulged in distasteful personal attacks as well.

The number crunchers drew huge eyeballs. Given their accuracy in the face of floundering by the innumerate, they will draw even more in future. Apart from the accuracy, it was the sheer detail with vote shares being predicted state by state – along with confidence levels – that was impressive. Both Republicans and Democrats are likely to pay more heed to these methods now, and it is likely that 2012 will be a watershed in US election coverage. The New York Times’ collaboration with Mr Silver has been successful for both. Other media outfits will look to tie up with similar competencies. The high penetration of sophisticated US opinion polling, along with the dominant two-party system, makes it relatively easy to do this. Applying similar methods to India is inherently more difficult, but people are bound to try. The Indian political map is more heterogeneous, apart from differences in scale; multi-party contests routinely lead to wafer-thin margins and are harder to model than two-horse races. The opinion polling mechanism is less developed and more likely to be biased in India. But the rewards for getting it right could be so large that it may well trigger investments into this particular nexus of applied social science.