The four were killed when the militiamen, also known as Huthis, blew up the house of Sheikh Yahia Taqi of the Islamist Al-Islah party at al-Makarib in the Arhab area, the source said.
Local tribes then ambushed an Ansarullah convoy of four vehicles carrying reinforcements, "killing or wounding around 30", a local official said.
That toll could not be independently confirmed.
Arhab, some 30 kilometres (20 miles) north of Sanaa, fell to the Shiite militiamen in mid-December as they sought to extend their influence after taking the capital in September.
