AI safety must keep pace with adoption in financial sector: CEA Nageswaran

The Chief Economic Adviser said AI can improve credit assessment and risk detection in fintech, but firms must prioritise safety and security as adoption accelerates

India needs to be proactive about the safety and security of artificial intelligence (AI) in the financial sector as firms increasingly adopt the technology, Chief Economic Adviser (CEA) V Anantha Nageswaran said on Friday. He warned that the sector may not have the luxury of waiting for AI-related risks to materialise before responding.

 

Speaking at Assocham’s India International Fintech Festival, Nageswaran also cautioned that it was too early to assess AI’s economic benefits amid the current hype around the technology. “I don’t think we will have the luxury of time, particularly in the financial sector, to be able to react later. We have to be very focused on the safety and security aspect of AI,” he said.

 

Recent instances of AI agents acting autonomously underscored the need to focus on safety and security alongside the technology’s productivity potential, he said. “We don’t generally pay attention to risks that well until it becomes a reality. So that is where our focus should be,” Nageswaran said.

 

His comments came after Meta earlier this week said one of its AI models had hacked another company during cybersecurity testing. OpenAI and Anthropic had previously made similar declarations, saying their AI models had inadvertently breached the systems of several institutions during testing.

 

The UK government’s AI Security Institute also said on Wednesday that Anthropic’s Mythos 5 and OpenAI’s GPT-5.6-Sol carried out unauthorised actions, including hacking a website and attempting to inject harmful code into software during safety tests. The institute intentionally gave the models internet access and removed certain safety filters to test their capabilities.

 

Nageswaran also highlighted AI’s potential to improve the fintech industry by helping firms assess creditworthiness more accurately and identify risks and financial stress earlier. “Specifically speaking from the fintech perspective, of course AI is going to help various firms in the industry. Analyse the creditworthiness far better, flag risks and stress much earlier,” he said.

 

However, he said a proper cost-benefit assessment of AI would be possible only after the current enthusiasm around the technology subsides. “Only when the current financial market bubble in AI deflates will we be able to do a proper cost-benefit analysis of AI,” he said.

 

Nageswaran said there was currently “too much hype” around the potential of AI and called for greater attention to AI risks even as businesses seek to capture productivity gains. “Focus right now has to be on the safety and security dimension. With the temptation to take advantage of the potential productivity gains, we should not lose what we already have,” he said.

 

Abhijit Phukon, economic adviser at the finance ministry’s department of financial services, who also addressed the event, echoed the CEA’s views, saying AI was essential but required sufficient guardrails. “All the institutions that are actually adopting the AI-based model have to ensure that there are sufficient barriers to ensure the users’ interest alongside the business interests of the organisations,” Phukon said.