Devangshu Datta: The Stupidity of Groups

There are many aphorisms about the "Wisdom of Crowds". There are also (pseudo) zen koans about "Never underestimating the power of stupid people in large groups". There is a certain amount of truth to both these contradictory aphorisms.

Crowds can be wise when they discover the "right" price for an asset. They can pick the right answers in a multiple choice quiz show, or vote for the best moves in a chess game. No single member of the crowd may know the right answer, or the best move, and yet, the crowd in aggregate, is liable to find it.

One of the earliest examples of such collective wisdom was cited by the statistician, Francis Galton, who jotted down each guess in a "Guess the ox's weight" contest at a fair circa 1905. Nobody guessed the weight correctly. But the average of all guesses came very close to the real weight. (They slaughtered the ox, weighed it and ate it.)

But crowds can also be very stupid and give exceedingly foolish answers, or believe very absurd things. For example, back in the 1950s, the psychologist, Read Tuddenham, manipulated a bunch of American college students (in their early 20s, and presumably, the children of considerably older parents) to collectively assert that the life expectancy of male Americans was 25 years! Every student in that group must have known the answer was wrong.

This sort of collective error can arise out of a desire on the part of the individual to conform to what it thinks the group knows. We are, after all, herd animals. Many, if not most individual human beings, are hard-wired to trust collective opinion more than their own judgement.

It is also apparently easier to trust the judgement of somebody who is similar to you. This desire for conformity may be an useful evolutionary bias. Back in the day when the nights were dark and full of terror, an individual was more likely to survive if he panicked when the herd panicked, even if the individual had no personal reason to panic.

So the "conforming to stupidity" phenomenon seems to work most strikingly in groups with clear collective identities. When the crowd has a shared demographic identity, and everyone "knows" everyone's opinions, the "Stupidity of Groups" comes into play. Conversely, the "Wisdom of Crowds" works best when the crowd is demographically diverse and importantly, when individuals are unaware of the opinions of other crowd members.

This desire to conform can be exploited by cleverly seeding opinion biases across information sources. The internet and the power of social media makes it much easier to quickly create a highly-coloured bubble of false opinions around most issues. At its least consequential, the stupidity of groups leads to people listening to terrible music, or watching bad movies. The stupidity has more consequences when investors buy over-valued stocks because "everyone" is buying those self-same stocks. The desire to conform may be even more consequential, if everyone with the same sense of shared identity votes for the wrong person.

Sometimes, the herd votes for the wrong decision in a referendum. This is what must have happened in the Brexit. It was blindingly obvious from day one that the UK would markedly damage its own economic prospects if it opted out of the European Union. Almost everybody in the UK will end up worse off in material terms because of the Brexit. Younger people by and large, figured that economic conundrum out and voted for "remain".

But there are far too many pensioners and people over 50 in the UK. Older Britons remember World War II. The Brexit campaigners cheerfully played up their fears by pointing out that Germany is the biggest player in the EU. Again, one understands. By analogy, it would be very hard to convince the Partition generation to contemplate a common market in the Subcontinent.

The shared identity of older British citizens also appears to be one of insularity. Most seem to be terrified by the prospect of unfettered immigration. (Their children who would have actually compete with immigrants in the workplace were okay with the prospect).

Identity, tribalism. Prime reasons why the Stupidity of Groups outpointed the Wisdom of Crowds.
Twitter: @devangshudatta