India's fact checking on toilets rests on asking the right questions

Accurate data on sanitation behaviour and toilet use can be captured if surveys ask questions on sanitation practices of all individuals in a household instead of one member, say experts

More than a year after the government declared that all rural households have access to toilets and are open defecation-free under the Swachh Bharat Mission (SBM), the National Family Health Survey 2019 (NFHS-5) shows differences between SBM portal data and the NFHS data.

On October 2, 2019, Prime Minister Narendra Modi announced that 699 districts and 599,963 villages had "declared themselves as free from open defecation" under the SBM. The flagship rural sanitation programme launched in October 2014 has increased sanitation coverage from 39% to 100% in five years, the Centre claimed.

But the National Family Health Survey-5 (NFHS-5, 2019-20), a household-level demographic and health survey, revealed that of the 18 states/Union territories for which NFHS-5 state reports are available, rural households in just five states--Kerala, Manipur, Mizoram, Nagaland, Sikkim--had 100% toilet access. Bihar, Gujarat, Karnataka and Maharashtra had less than 80% toilet access in rural areas.

Researchers IndiaSpend spoke to said the SBM portal's toilet coverage data and NFHS data vary because the former are based on what the respondents tell the surveyors, which is then physically verified by government officials. The NFHS survey has a different approach where sanitation questions are asked of a single eligible household member. This may lead to incorrect figures because each household member is not separately questioned on their sanitation practices.

Among the eight socio-economically backward states--Bihar, Chhattisgarh, Jharkhand, Madhya Pradesh, Odisha, Rajasthan, Uttarakhand and Uttar Pradesh--NFHS-5 is only available for Bihar. The largest discrepancy between NFHS-5 and SBM portal data is for this state--NFHS-5 reports that 57% of rural households have toilet access. SBM portal data report 100% access. Nearly half the global population lacked safe sanitation and 494 million people practised open defecation, found a July 2021 World Health Organization/Unicef Joint Monitoring Programme for Water Supply, Sanitation and Hygiene (JMP) report. But India reported the "largest drop" in terms of absolute numbers for open defecation since 2015.

Eliminating open defecation through toilet use is important because open defecation has been shown to increase child stunting and risks of non-partner sexual violence against women who do not have access to private toilets, and cause the weakening of nutritional interventions to improve child health. "Women who are more exposed to open defecation are thinner, on average, than women who live in villages and neighborhoods where more people use latrines," said Diane Coffey and Dean Spears, researchers and co-founders of Research Institute for Compassionate Economics (r.i.c.e.), in a 2017 interview with IndiaSpend.

Survey differences

Until 2019, states had spent 91% of the Rs 62,146 crore released for SBM by the Centre and the largest share of the expenditure, over the years, has been for individual household latrines, according to an analysis by Accountability Initiative, a transparency and accountability research group.

The revised 2017 SBM guidelines note that sanitation is a state subject, so states must verify their own open defecation status by developing mechanisms for such verification. It recommends that the gram panchayat [elected local government] should pass the resolution for ODF, followed by rounds of verification at district and state levels for cross-verification.

But ODF declaration at the village level does not mean that open defecation has been eliminated. More than one in three toilet owners defecated in the open, IndiaSpend reported in August 2018.

The SBM is supposed to physically verify toilet access and whether open defecation has stopped at the gram panchayat level, with checks at the district and state levels.

The NFHS process is different. There are three questionnaires--for men, for women, and for households. The sanitation questions (from 40 to 46) appear on the household questionnaire and are asked to an eligible member of the household. They can be women aged 15-49, men aged 15 to 54 years as well as overnight visitors, according to the NFHS-4 (2015-16) interviewer manual.

While state and district-level government functionaries fill in administrative data [where every toilet demanded and constructed has to be verified] on the SBM portal, NFHS-5 is a household-level sample survey, said Avani Kapur, Fellow at Centre for Policy Research who leads the Accountability Initiative. "One always needs both sources--while administrative data records the status, independent surveys are critical for accountability and verification of recorded information," she said.

State variations

More rural Indians in 2018 owned a latrine than in 2014; yet 44% of them defecated in the open in Bihar, Rajasthan, Uttar Pradesh and Madhya Pradesh. Bihar has seen a drop in the percentage of households that do not use any sanitation facility for defecation, from 67% in NFHS-4 to 39%, in NFHS-5 This is still the highest among states for which NFHS-5 data are available. Kerala, Manipur, Mizoram, Nagaland and Sikkim are the only states where toilet access is close to 100% for urban and rural populations, per NFHS-5. In four states--Bihar, Gujarat, Karnataka and Maharashtra--at least 20% of households practice open defecation, according to NFHS-5.

Reasons why surveys and SBM portal data may differ

The purpose of surveys like the NFHS is to capture details about the health of the population, not sanitation and toilet construction, a senior official in the Department Of Drinking Water And Sanitation (DDWS) who did not want to be identified told IndiaSpend. They said that the government's objective is to ensure access to toilets through own, shared, or community toilets, and noted that Bihar has constructed "around 11,000 community toilets" under the SBM. The exact reasons for differences between SBM and NFHS are unclear, but the official believed that it could be because respondents underreport toilet availability in surveys for financial benefits. The government allocates Rs 12,000 for the construction of each toilet in SBM. Also, the time period of the NFHS is longer and different and the NFHS does not separately count access to community toilets [No 45 in the questionnaire], they said.

The government's target for toilet construction was based on the baseline survey for SBM conducted in 2013, said V.R. Raman, head of policy, WaterAid India. The NFHS-5 was conducted in 2019 when toilet demand may have increased, thus including households in the survey that were left out from the 2013 baseline. However, "the guidelines and plans for SBM 2.0 continue to reflect the need to ensure that no one is left behind, which is a very positive sign", Raman added.

SBM can collect more detailed information on sanitation, whereas this may be only one of many questions that a survey like the NFHS collects, he added. In order to get disaggregated usage data, SBM could choose to collect information on all the individuals in a household, by gender and age, and also ask questions on the frequency of toilet access.

Open defecation is underestimated

The scale of open defecation in the country may be underestimated depending on questions asked on sanitation behaviour, according to a September 2019 study that looked at different types of survey questions. When each individual in a household is questioned, surveys find more instances of open defecation in contrast to asking one member to report on the toilet behaviours of all household members. Surveys must also collect information on whether latrines are individually owned or community-owned to assess latrine infrastructure for each family, and measure open defecation more accurately, the study noted.

An individual level question led to a 20-percentage-point increase in open defecation figures compared to asking a household level question that assumed that everyone in the household has the same sanitation behavior, Nikhil Srivastav, a researcher at r.i.c.e and one of the authors of the study told IndiaSpend.

The National Annual Rural Sanitation Survey (NARSS), Round-3, 2019-20, that surveys 92,010 households and occurs more often than the NFHS, indicates that 94% of the households have toilet access and 95% of individuals use it, said an official at DDWS requesting anonymity.

Both the SBM and the NFHS measure open defecation at the household level, Aashish Gupta, a demographer at the University of Pennsylvania, told IndiaSpend. "The SBM assumes that owning a toilet implies using a toilet, for all household members. We now know that the SBM claims of household access are vastly overblown." Regardless of whether a survey asks questions to every household member or a single member representing the household, there is still work to be done on reducing open defecation in rural India, emphasised Srivastav.

Challenges remain

It is hard to sustain momentum once a programme has been declared as having achieved its goal. SBM has been very ambitious in its scale and speed. Human resources were diverted from existing work making it unsustainable in the long run to continue with the same momentum, said Kapur.

The pandemic has caused delays to the NFHS-5 and also led to challenges in maintaining the momentum of SBM's next phase. "We have to ensure that there are no slipages," said the DDWS official. Sanitation is a continuous process and ODF can be maintained only through sustained communication, nudging people to change their behaviour, the official added.