Sliding statistics standards

Govt must make data more timely, comparable and accessible

The recent Economic Survey made a case for what it called “agile” policy that responded swiftly to changes in the overall economic landscape, using newly available high-frequency data. Much of the data quoted in the Survey, however, was from non-government sources such as mobility indicators collected by Google. Those that were from government sources were often new-age tech-based indicators such as goods and services tax collection and highway tolls. The fact is the data indicators that are collected by the government itself are neither collected often enough, nor collated in a timely fashion. Worst of all, they are not released in an easily readable or analysable format. It is 10 years since the government set up the data.gov.in web site, and in that period it has barely been updated either in content or format. It is hard to search, it does not present data in a logical way, and its Application Programming Interfaces are hard for third-party software to use. An analysis by this newspaper demonstrated that 85 per cent of the APIs on the web site were for just three of its 35 sectors.
 

The problems with the accessibility of government data run deep. State budgets are in general almost impossible to analyse because of incomparable formats, so understanding changes in state-level outlays and receipts usually has to depend on the Reserve Bank of India’s work, which arrives fairly late. While the Union Budget itself has a well-understood format, it is unnecessarily hard to access time series longer than a few years for many crucial variables without collating individual numbers from various Budgets across time. Ministry web sites typically have data only in text or PDF format, rather than machine-readable worksheet formats. Even those documents have inexplicable gaps; for example, monthly accounts on the Union power ministry’s website are complete for 2019, with all 12 months, but for the year 2020 only exist for the month of January — and for only five months of 2021. They also only seem to show cumulative numbers. Fortunately, many macro-economic figures are required by India’s participation in the International Monetary Fund’s data dissemination protocol to be updated regularly on a national summary page maintained by the Union finance ministry — but even there the ministry has to warn that state government data after 2019-20 is “provisional” and incomplete.

The government is clearly failing in its duty to provide clear data to its citizenry. This is not only important from the point of view of informing citizens’ judgments about its performance, but also it disadvantages the Indian private sector, which has to invest in its own indicators. The Survey sang praises of data-informed policy. Surely it is obvious that the private sector and regular Indians also want to make choices informed by data? The government must work harder to collect data, to make it available in a timely and machine-readable manner, and to ensure that databases are compatible with each other and over time. This is a basic requirement for a well-functioning and mature economy, and India has allowed statistical standards to slide for too long.