Mattis landed at the Nur Khan Airbase in Rawalpindi where he was received by officials from Pakistan's defence and foreign ministries.
He will hold talks with civilian and military leaders including Pakistan Prime Minister Shahid Khaqan Abbasi and Army Chief General Qamar Javed Bajwa.
Relations between the US and Pakistan have plummeted after President Donald Trump announced his new Afghan and South Asia policy in August and directly blamed Islamabad for supporting the Afghan Taliban.
Mattis is on a five-day tour to Egypt, Jordan, Kuwait and Pakistan.
Visiting Islamabad about a week after the White House asked Pakistan to immediately charge and arrest 26/11 Mumbai attack mastermind Saeed, in the absence of which it warned of repercussions, Mattis said he needs to talk to leaders in the country and gain their understanding.
JuD chief Saeed, who has an American bounty of USD 10 million on his head, walked free on November 24 after a court ordered an end to his 10-month detention.
The Pakistan government decided against detaining him further in any other case, pushing back against India's efforts to bring the perpetrators of the 2008 Mumbai terror attack to justice.
"The first thing I'm going to do is do some listening, like I always do. My goal is to find common ground. I need to go, to sit down and listen to them, start there, start by listening," Mattis said on his way to Pakistan from Egypt.
Mattis is the first top American official to visit Pakistan after the completion of the first 100 days of the US' new South Asia strategy unveiled by Trump on August 21.
"We know we have some common ground. They have lost hundreds, thousands of their troops killed and wounded by terrorists. They have lost hundreds, thousands of their innocent people murdered and wounded by terrorists, so we know that there is common ground," he said, ahead of his visit.
Similarly, he said, there is common ground between Afghanistan and Pakistan, because there are terrorist groups that try to move back and forth, that do move back and forth in order to live in one country and attack the other.
"So we know there's common ground; it's how much more common ground can we find by listening to one another without being combative with one another, listening to others' perspective," Mattis said.
"But at the same time, as the General (Bajwa) has said, he wants no havens for terrorists anywhere, so we will work together and we'll find that common ground, if we have the will to. And then we'll work on how we address the problems where we can work together," he said.
He said he would not characterise everything over the past 16 years as failures.
"There is clearly an abundance of areas where we have to double down, and I am optimistic at this point that because of what our adversaries, our mutual enemies are doing, that we can find ways to work together," he said.
Mattis said both Pakistan and Afghanistan, the Pakistan military, the NATO-led militaries of 39 nations have all been in fights here. "We've got to find a way to work together".
He said there is "always an urgency to something" when 39 nations plus Afghanistan have their troops in the midst of a long war, where causalities are being taken.
"These are 39 of the most economically powerful, generally speaking, most long-term democratic institutions in the world. These are powerful nations morally, economically and diplomatically, and certainly militarily," he said.
Mattis said that it was essential in South Asia that all countries work together to restore the level of stability that allows for the economy to build.
"I mean, can you imagine a border between Pakistan and India where economic trade is not ongoing to the benefit of people on both sides of that border. We can not only imagine it, it's a reality right now," Mattis said.
"So we have got to work together on this, and there's a sense of urgency about that," he said.
"There's people who are living below the poverty line in both those countries. So as you look at this problem, if you'll expand the problem you can understand the sense of urgency to move beyond violence and get back to the normal order of things," he said.
"General Bajwa is a mature officer with a deep background in not just military matters, but South Asia security issues, and we have got to find a way to move beyond this constant fighting that's characterised the area," he said.
Media reports suggested that Mattis faces a difficult task ahead as Pakistan is not happy over repeated US allegations of safe havens for militants in the country.
The Express Tribune yesterday said the civilian-military leadership will adopt a "clear cut and decisive" stance during the visit of Mattis that "only cooperation will get cooperation".
(This story has not been edited by Business Standard staff and is auto-generated from a syndicated feed.)