The kidnapping took place on Wednesday off the northeast of Sabah state on the Malaysian side of Borneo island, Indonesia's ambassador to Malaysia said.
"(The vessel) was intercepted by a boat carrying four armed men," ambassador Herman Prayitno told AFP.
The kidnappers took the Indonesian captain after failing to get the 10,000 ringgit (USD 2,500) they demanded, Prayitno added. They released the two other crew members, an Indonesian and a Malaysian.
Indonesia's foreign ministry said it was still trying to find out which group was responsible.
The kidnapping is the latest in a string of incidents in the Sulu Sea, where groups of armed men have ambushed fishing vessels and seized Malaysian and Indonesian citizens for ransom.
Ten other Indonesians kidnapped in recent months by the Philippine Abu Sayyaf extremist group are still being held.
Jakarta has banned Indonesian-flagged vessels from sailing to the Philippines and pushed for joint maritime patrols in the waterway.
A handful of Malaysian sailors have also been kidnapped this year.
The group is a loose network of a few hundred Islamist militants, formed in the 1990s with seed money from Osama bin Laden's Al-Qaeda network.
