The violence erupted overnight in the El Abra prison yesterday, outside the city of Cochabamba and police lost control of the jail for several hours as inmates attacked each other after an alcohol-fueled party.
"So far we have counted four dead and 11 wounded. The dead have now been removed and the wounded taken to hospital," said Andres Cuevas, a regional government official.
Rivalry between Bolivian and foreign prison gangs was to blame, according to the regional police commander, Alberto Suarez.
He did not name the nationalities of the foreign inmates. Bolivian media reports said they were Colombians jailed for drug trafficking.
The four dead were all Bolivian and included a prison gang boss named Ariel Tancara, whose throat was reportedly slit.
Bolivian TV channels showed images of his sister outside the prison screaming: "I want my brother's body! Give me my brother!"
Witnesses said the inmates had organised a party Sunday night that included alcohol and live music.
It is the worst prison violence in Bolivia since August last year, when 35 inmates were killed in riots at Palmasola prison in the eastern city of Santa Cruz.
Bolivia's prisons are the most crowded in Latin America after El Salvador's, according to the Organisation of American States.
Its jails have the capacity to hold 3,738 prisoners, but currently house 13,840, OAS reports.
