75-year-old Ansari was administered the oath by President Pranab Mukherjee in the ornate Ashoka Hall of the presidential palace. He took the oath of office in English and in the name of God.
Ansari is India's 12th Vice President and only the second person after Sarvapalli Radhakrishnan to get a second term.
He will once again be the Chairman of Rajya Sabha by virtue of his being the Vice President.
Prime Minister Manmohan Singh, former President Pratibha Patil, UPA chairperson Sonia Gandhi, BJP leaders L K Advani, Sushma Swaraj and Arun Jaitley, and several dignitaries, including Union Ministers and Governors attended the ceremony.
A former diplomat, Ansari defeated NDA candidate Jaswant Singh with a comfortable margin in the August 7 election for the post. He had polled 490 votes against 238 cast in favour of Jaswant Singh.
A former ambassador to the United Arab Emirates, Iran Afghanistan and Saudi Arabia, Ansari was also the Vice Chancellor of Aligarh Muslim University and Chairman of the National Minorities Commission.
He is the grand-nephew of former Congress President Mukhtar Ahmad Ansari, a prominent leader of the freedom struggle.
Born in Calcutta on April 1, 1937, Ansari completed his schooling from St Edwards High School in Shimla, attended the St Xavier's College, University of Calcutta, and pursued MA in Political Science at the Aligarh Muslim University, from where he also got his doctorate and later worked as a lecturer.