Sharif, who is accompanied among others by his wife Kulsoom Nawaz and his son Hussain Nawaz, arrived at the technical Air Force base in Palam.
Before his departure, the Pakistan Prime Minister told reporters in Lahore that "Pakistan wants good relations with India and I am going to New Delhi with a message of peace".
It is the first ever visit by a Pakistan leader to India to attend the oath taking ceremony of a prime minister-designate.
63-year-old Modi, who led BJP to an astounding victory in the Lok Sabha polls, had last week extended an invitation to SAARC leaders including the Pakistan premier to attend his swearing-in ceremony.
Sources said it is expected that the Pakistan leader will also extend a formal invitation to his counterpart for a visit.
No breakthrough is expected during the visit but it will provide the two leaders an opportunity to cultivate personal relations which go a long a way in reducing tensions.