The boat capsized and sank in waters off West Java's Sukabumi district after being hit by high waves yesterday.
Survivors said about 100 people were aboard the vessel.
Twenty-eight people were rescued and taken to the Sukabumi immigration office for identification, Brig. Gen. Tatang Zainudin, the National Search and Rescue Agency's operation chief, said by phone from the scene.
He said 21 bodies were pulled from the water last afternoon, including seven children.
A helicopter and more than a dozen boats were being used today to search for about 35 people believed to be missing, but strong currents and high waves were hampering the operation, Zainudin said.
"We fear those who are still missing were unable to survive," he said.
There were conflicting reports about the exact number of people on the boat due to the lack of a manifest, but some survivors told officials that about 100 asylum seekers from Lebanon, Pakistan and Iraq were believed to be aboard, said a local police chief, Lt. Col. Deddy Kusuma Bakti.
Lebanon's official National News Agency said 17 Lebanese drowned in the incident. Nine members of a single family were among the Lebanese victims, with a woman and her eight children dying and her husband surviving, the agency reported.
The boat capsized and sank after being hit by up to 6-meter waves hours after leaving Sukabumi last morning, Zainudin said.
The incident came ahead of new Australian Prime Minister Tony Abbott's first visit to Indonesia next week. He is expected to meet with Indonesian President Susilo Bambang Yudhoyono.
Indonesia has said that the Australian navy's plan to intercept and force back Indonesian fishing boats crowded with asylum seekers could breach Indonesian sovereignty.