Prime Minister Nuri al-Maliki responded by asking parliament to declare a state of emergency and announcing the government would arm citizens to fight the militants.
"All of Nineveh province fell into the hands of militants," parliament speaker Osama al-Nujaifi told journalists in Baghdad, adding the gunmen were heading south towards neighbouring Salaheddin province.
An army brigadier general told AFP hundreds of militants from the Islamic State of Iraq and the Levant (ISIL) launched a major assault on the security forces late yesterday.
Soldiers and police had stripped off their uniforms and fled, and the militants used loudspeakers to declare they had "come to liberate" the city of some two million people.
An AFP journalist, himself fleeing with his family, said shops were closed, a police station had been set ablaze and security forces vehicles had been burned or abandoned.
Hundreds of families were seen fleeing. Some were on foot, carrying what they could, others in vehicles with their belongings piled on the roofs.
Dozens of cars and trucks stretched out from one checkpoint on the boundary of the region, as people with plastic bags, suitcases and a pram waited to enter, some with young children in tow.
"The army forces threw away their weapons and changed their clothes and left their vehicles and left the city," said Mahmud Nuri, a displaced Mosul resident.
"We didn't see anyone fire a shot".
The assailants seized the provincial government headquarters and the Nineveh Operations Command as well as the airport, the army general said.
The Turkish consulate in Mosul said ISIL fighters had captured 28 Turkish truck drivers, while a foreign ministry official said Ankara hoped they would be released once they finished unloading fuel oil at a power station.
Maliki said the cabinet had decided to arm citizens to fight the militants.