Madhukar Sabnavis: Going beyond the numbers

The author explains what the debate over political opinion polls teaches us about marketing

Opinion polls have been the flavour of the season for the last few months, especially during the run-up to the state elections in December 2013. There is already much debate about how accurate they are and how they could mislead voters. In fact, whether the results do influence voters is an open question. There is one school that believes voters have independent minds and that such polls just provide content for the media to grab eyeballs and readership. However, others postulate that many voters are often fence sitters and that opinion poll results could swing them to a supposed winning side - either by assuring voters that their vote is not wasted or by reassuring them that they are voting for the right party. It is true that in India popularity is a big driver for the final decision. Notice, for instance, how people flock to a crowded store rather than to an empty one, even if the products seem similar. This could be attributed to the psychological belief that if crowds are there, the product must be better. This piece, however, is not about the impact of research results. It's about diving deeper into understanding the accuracy of predictive research, using opinion and exit polls as a database.

Opinion polls conducted months before the actual polling have obvious limitations. In a market where local candidates play a role in the final decision, views in pre-poll research are based on broad strokes that are nowhere near reality. A statement to support one party could be altered by a better candidate fielded by the opponent locally - on the basis of either performance credentials or caste - or by the preferred party not fielding a candidate, and giving it to a non-acceptable coalition partner. One knows brand equity doesn't get transferred from one brand to another just because a brand says so. Layer it with sampling limitations (more on that later) and the numbers can, at best, be indicative and directional rather than definitive. Not surprisingly, most opinion polls were way off the mark in the final tally.

Interestingly, even the exit polls did not get much right. Exit polls are closer to actual behaviour since they are conducted after the polling. If one compared the final numbers in the three Assemblies - Madhya Pradesh, Rajasthan and Delhi - with the predictions of four polls - C fore, C-Voter, Chanakya and CNN-IBN/CSDS - the actual variation would be quite significant. For Madhya Pradesh and Rajasthan, the predictions were directionally right. However, none of the four were anywhere near the final figure of 165 and 162, respectively, that the Bharatiya Janata Party (BJP) finally got, or nowhere as low as what the Congress achieved, at 58 and 21, respectively. In Delhi, all four polls clearly underestimated the effect of the Aam Aadmi Party (AAP). The best was a prediction of 13-21, still distant from the 28 the party finally got.

Why so way off the mark? However scientifically done, errors often creep into the conversion of vote share into seats. But the larger question is: how representative is the sample of a vast and diverse country like India? In a country of 1.2 billion people with varied languages, religions, caste and socio-economic backgrounds, factoring in all these characteristics and more while choosing the right respondents - who then must answer accurately - is a Herculean task. This makes predictions based on seemingly large samples still a challenge. Where there is a definitive swing, the results are strongly indicative; the challenge arises when the market scenario - in this case, Delhi - is mixed. With no definite trends, the predictions could be highly elastic.

Marketing and business have to live on research, which is often quantitative. Given the millions of rupees invested in products and advertising, it's not surprising - or even wrong - for companies to depend on research results to take business decisions. However, it needs to be approached with caution. Samples are never as large as the ones done for such opinion polls - brands are not as passionate subjects as politics. So the interpretation of data is as important as its collection and the selection of respondents.

Whatever the sample size, there can always be questions on how representative it is of the larger Indian market. Even if caste and local issues play no role in brand decisions, factors like the consumer's mindset about experimentation within a particular category or her receptiveness to brand communication could play a role in a respondent's response, skewing the numerical findings. To assume that this is evened out across studies while benchmarking against the norm is definitely dangerous, as errors tend to have a multiplier effect when they are carried forward. "Liking", for advertising stimulus, and "intention to purchase", for a brand post-product concept or advertising, could get over-favourable responses - since the downside of giving a positive response is low. After all, just as in opinion polls, saying "I like" or "will buy" doesn't mean one needs to do it when it comes to actual behaviour. Hence, data thus generated need to be viewed more carefully and interpreted right.

There are umpteen examples of product and advertising concepts that have failed quantitative windows and yet have done well in the actual marketplace - just as there are enough examples of concepts that have worked through such tests and also flown in the marketplace. This is not a case against quantitative research, because it is a rigorous way of working out how consumers feel. However, it is about looking at research as stimulus rather than solutions, and about judiciously interpreting them to take a more considered decision. New methodologies are being adopted constantly to both recruit the right samples and get closer to real responses - including the introduction of neuroscience to understand consumer responses to advertising stimuli. All of it is welcome because there is a need to get some reassurance that one is doing the right thing, given the millions of rupees put into new campaigns and products. Depending only on individual judgement has its risks, but taking the human element out and depending only on data to make decisions are equally dangerous.

Expecting opinion polls to give exact results would be overestimating their power. But looking at them to give us trends and directional views, after examining the robustness of their samples, is worthwhile. Any research honestly done has its value. Let's neither debunk it nor think of it as God's gospel. The truth is somewhere in between. Something worth thinking about.

The writer is vice chairman, Ogilvy & Mather, India.
Views expressed are personal. madhukar.sabnavis@ogilvy.com