Abdiweli Sheikh Ahmed received support from only 80 lawmakers, while 153 backed a motion to oust him. Ahmed has for months been at loggerheads with President Hassan Sheikh Mohamud, prompting international backers to warn that the infighting was putting at risk the fragile gains made in the war-scarred nation and with the Al-Qaeda-affiliated Shebab rebels still carrying out regular attacks.
Ahmed took the post in December 2013 after his predecessor Abdi Farah Shirdon was ousted by parliament after just over a year in office.
Political wrangling and reports of corruption have raised concern the government, like the last administration, is blighted by infighting and failing to unite in the face of the threat by the Shebab.
Somalia this week was again ranked worst in the world for graft, according to the Transparency International Corruption Perceptions Index.
The UN Office for the Coordination of Humanitarian Affairs (OCHA) warns people face a "deepening humanitarian crisis" with over three million needing aid. The number of people in crisis is growing for the first time since the end of a devastating famine three years ago.