Ansari, the senior most foreign dignitary to attend the funeral, described the former senior commander of the Northern Alliance that fought Taliban as an "illustrious son" of Afghanistan and a "towering leader" of the Afghan people.
Fahim, 57, died of natural causes in Kabul on Sunday. Fahim, a close lieutenant of charismatic Northern Alliance commander Ahmad Shah Masood, visited India in 2011.
India has lost a "true partner", Ansari wrote in the condolence book.
Ansari too was India's envoy to Afghanistan in 1989-90.
Prime Minister Manmohan Singh's special envoy and point person for Afghanistan Satinder Lambah was also part of the delegation.
The pointed inclusion of former ambassadors to accompany an Indian dignitary at a funeral, is the first of its kind in the realm of India's foreign engagement and reflects the importance that India places on the strategic partnership with Afghanistan, sources said.
After the funeral prayers, Ansari had detailed talks with Afghan President Hamid Karzai at the Presidential Palace here.
Thanking the Vice President for braving snow and bad weather to attend the funeral, Karzai said this reflected the "all weather" nature of India-Afghanistan relations.
Karzai said whatever the outcome of the forthcoming elections and the orientation of the new leadership, friendship with India would remain a "top priority" for any new government in Afghanistan.
Ansari mentioned that friendship between the people of India and Afghanistan existed much before friendship between the two governments and there was unanimity in India about the continuance of good relations with Afghanistan.