The visit is taking place when Pakistan's parliament is mulling a request by Saudi Arabia to provide troops, jets and warships to join its coalition fighting in Yemen against Shia Houthi rebels, allegedly supported by Iran.
An official of the foreign office said that Zarif was having a meeting with Pakistan's advisor on foreign affairs Sartaj Aziz.
Later the two will address a joint press conference.
The agenda of his visit has not been revealed but according to informed sources he will also meet Sharif.
Sharif has already taken an initiative to find a peaceful solution to the issue and visited Turkey last week.
After the visit, the Turkish president yesterday visited Tehran and held talks on the Yemen issue.
Sharif yesterday told parliament that he was waiting for a message from the Turkish leader after his visit to Tehran to take more steps for tackling the problem.
It is believed that Zarif will try to dispel the impression that his country was arming the Houthis who toppled the Yemen government and are now a threat to Saudi Arabia.
There is growing concern in the Muslim world that the Yemen crisis may deepen the sectarian divide and may endanger peace and stability in other countries, mostly having both Sunni and Shia populations.
Pakistan has a 20 per cent Shia population who are already under attack by extremist Sunnis groups like the Taliban and Lashkar-e-Jhangvi.
