)
As a clutch of pollsters predict a National Democratic Alliance (NDA) win in the coming elections, a sting operation by a television news channel last month showed opinion polls can be rigged, forcing the Election Commission to take notice.
Opinion polls typically depend on timing, sample size and sampling method, questions and their sequence, and analysis. Each of India's top polling agencies -C-Voter, Today's Chanakya, the Center for Study of Developing Societies (CSDS), IMRB international and Nielsen AC - differs in its approach, and their predictions, naturally, differ. Business Standard looks at what makes polling in India a particularly perilous exercise.
Sample size and errors
The Congress is opposed to opinion polls and has demanded their withdrawal, with various studies suggesting the party could lose over 100 of its existing seats.
"The sting operation proves opinion polls are manipulated with black money. A new form of election rigging and we support the Election Commission's views to ban opinion polls till expiry of polling, in the wake of recent expose of manipulated opinion poll," Congress spokesperson Ajay Maken tweeted last month.
The spectre of manipulation raises questions about the margins of error that pollsters tolerate when predicting outcomes. "Ideally, the margin of error should be in the range of two to three per cent and if companies predict their outcomes at the 99 per cent confidence level, results will be similar across platforms if the study is undertaken at the same time," says Rahul Verma of Lokniti-CSDS. His firm has been in the business since the 1970s, and Verma says it only predicts outcomes when it is sure about its study.
Experts say sample size does not really affect the outcome, but budget constraints do affect business. CSDS will have to spend close to Rs 1 crore on studying voting patterns because the company is looking at a sample size of 25,000 and the cost per person polled is in the range of Rs 400. "Globally, the sample size has always been small. In the US, the sample size is in the range of 4,000-5,000," an industry expert points out.
"During 2009, the University Grants Commission had paid a part of the expenses and the rest of the tab was picked up by media houses that wanted to use the data. Funding always remains a constraint," says Verma of CSDS.
The impression across polling companies is sample size is not an area of concern, though it only selects one in 50,000. "If the objective is to predict seats, this sample size is ideal to to represent the electorate in each constituency. However, smaller samples might just work if the idea is to present opinions of the electorate and vote share at a very broad level. Given the complexity involved in a diverse country like India, large samples need to be appropriately supported by strong methods of sampling. This needs a good understanding of the political landscape and factors that determine electorate behaviour at the local level," says A V Surya, vice-president, IMRB International.
Pollsters use neither a uniform sample size nor a sampling method. While some get away with a five per cent margin of error, researcher judgment and resource constraints also affect predictions. "The basic problem with polling in India is that people never truly reveal who they will vote for. So, findings could be faulty from the start. Statistically, research agencies do keep a margin of error of about three per cent - the global standard. Quite often, agencies play around with this window," says Alok Verma, chief executive officer (CEO) and editor-in-chief at Newzstreet Broadcast Services.
Does it really work?
Today's Chanakya, owned by RNB Research, recently predicted the Aam Aadmi Party was likely to win elections in Delhi. "We try to use equal quality in the sample as against the quantity. And the representation used is one of the best," V K Bajaj, CEO of Today's Chanakya says.
"The strength of research comes from asking the right questions that respondents can correctly answer at the time of the survey. If conducted early, a survey can limit itself to a few critical questions that respondents relate to. These can be 'how satisfied citizens are with the performance of the government', 'is the electorate seeking a change in government', etc. Robust sample sizes and statistical methods might not yield accurate predictions if the political scenario is dynamic and the electorate is yet to make up its mind," Surya of IMRB says.
"Political marketing targets the entire country's adult population and successfully engages many of them as elections approach. Individual voters are at different points on the decision-making trajectory. Only a few segments would have actually made up their mind early about which party to vote for," he adds.
How challenging is the process?
Money apart, there may not be enough student field investigators during examination time.Training of researchers varies as they are widely spread out. Sometimes, local dialects come in the way.
Polling agencies also need to understand the local context that influences the decisions of voters, the political landscape, history and voting patterns in order to ask the right questions and interpret the data.
"The business of poll predictions is worth about Rs 100 crore. Typically, a research agency uses a sample of 10,000 people per Lok Sabha constituency. It takes about 30-45 days to complete an opinion poll, from fieldwork to tabulation. Research agencies charge Rs 300-400 for an individual surveyed - the fieldwork alone costs Rs 30 lakh. Over and above this are the statistician's charges," says Alok Verma, who previously also worked as executive producer at Aaj Tak and editor at Zee News.
"The big concern is the sample size and how the questions are framed. Is there too much intervention happening from stakeholders? As long as solutions to these problems are not found, the business of poll predictions will remain hazy," he adds.
