Fisheries department Assistant Director K D Gopinath said the naval personnel of Sri Lanka arrested the four, all hailing from Pamban, while they were fishing off Katchatheevu last night. Their boats were also seized.
The fishermen and their boats were later taken to Thalaimannar in Sri Lanka for further proceedings, he added.
With this, the number of fishermen arrested for allegedly fishing in Sri Lankan waters has gone up to 48 since July 3, Gopinath said.
The incident comes a day after more than 2,000 fishermen from Rameswaram were chased away by the island nation's authorities for allegedly fishing in their territorial waters.
Chief Minister Jayalalithaa had written to Prime Minister Narendra Modi on July 6, requesting him to take steps to secure the release of Indian fishermen and their boats seized by the Sri Lankan Navy.