The deceased were identified as Anju BK, Gauri Chiral, Milan Damai and Sujata Nepali of a village in the district.
They died today, a day after a local health assistant administered anti-measles drugs to them yesterday.
The exact cause of the death could not be established, according to officials.
Twelve other children who received the drug have also fallen ill, according to the district health office.
A medical team has been dispatched from the capital to the affected area for carrying out investigation.