Police said, the three allegedly planned to attack 22 top leaders of the country including Prime Minister Narendra Modi.
They were allegedly involved in threatening embassies of various countries in India, they said.
Those arrested were identified as M Khareem, Asif Sultan Mohammed and Abbas Ali. While Khareem was arrested from Usmannagar, Asif Sultan Mohammed was picked up from G R Nagar and Abbas Ali from Ismailpuram, police said.
Explosive materials were also seized from them.
"Explosive materials and arms have been seized from them," police said, declining to elaborate.
The trio was allegedly running the Al-Qaeda unit in South Tamil Nadu. They were also allegedly involved in the explosions that took place in various courts in the country over a period of time, police added.
NIA were also searching for two other suspected Al-Qaeda operatives, Hakeem and Dawood Sulaiman, they said.