In backdrop of Economics Nobel announcement, letter panning RCT surfaces

15 economists, including past Nobel laureates say relying on RCTs to guide welfare and aid spending will lead to short-term, superficial and misplaced policies

In the backdrop of the Nobel Prize in Economics being awarded to Abhijit Banerjee, Esther Duflo and Michael Kremer, an open letter by 15 other economists from 2018 has gained prominence. In the open letter, the economists, three of whom are themselves Nobel laureates, have criticized “randomized control trials”.

Banerjee, Duflo and Kremer have been awarded the 2019 Nobel Memorial Prize in Economics for their contribution to reviving development economics, particularly through the popularisation of “randomised control trials” (RCTs) that break larger questions about policy interventions into smaller, easier to test studies.

In a letter sent to British publication Guardian in August 2018, the economists had said that relying on RCTs to guide welfare and aid spending will lead to short-term, superficial and misplaced policies. These economists include past Nobel laureates like Angus Deaton of Princeton University, Joseph Stiglitz of Columbia University and James Heckman of Chicago University.

The other signatories of the open letter include Florent Bédécarrats of French Development Agency, Barbara Harriss-White of Oxford University, Jason Hickel Goldsmiths of University of London, Naila Kabeer of London School of Economics, Solène Morvant-Roux of University of Geneva, Judea Pearl of Columbia University, Cécile Renouard of Codev-Essec Business School and others.

The letter states that as donors increasingly want to see more impact for their aid money, practitioners are searching for ways to make their projects more effective, and politicians want more financial accountability behind aid budgets. “One popular option has been to audit projects for results. Some go so far as to insist that development interventions should be subjected to the same kind of randomised control trials used in medicine, with “treatment” groups assessed against control groups,” it said.

“Such trials are being rolled out to evaluate the impact of a wide variety of projects – everything from water purification tablets to microcredit schemes, financial literacy classes to teachers’ performance bonuses,” the letter stated.

The letter argued that truly random sampling with blinded subjects was almost impossible in human communities without creating scenarios so abstract that they don’t reflect real world situations. These trials are expensive to carry out, and fraught with ethical challenges – especially when it comes to health-related interventions, it said.

“But the real problem with the “aid effectiveness” craze is that it narrows our focus down to micro-interventions at a local level that yield results that can be observed in the short term. It tends to ignore the broader macroeconomic, political and institutional drivers of impoverishment and underdevelopment,” the letter said, adding that what was needed instead was to tackle the real root causes of poverty, inequality and climate change.

“If we are concerned about effectiveness, then instead of assessing the short-term impacts of micro-projects, we should evaluate whole public policies,” it said.