Mattis flew into the war-torn city two weeks after Afghan President Ashraf Ghani unveiled a plan to open peace talks with the Taliban.
The insurgents have so far given no formal response to Kabul's offer for negotiations, but Mattis said some insurgent leaders have expressed an interest in the discussions.
"It may not be that the whole Taliban comes over in one fell swoop, that would be a bridge too far, but there are elements of the Taliban clearly interested in talking to the Afghan government," he told reporters aboard a military jet.
Ghani's peace plan includes eventually recognising the Taliban as a political party.
The Taliban last week described the Afghan government as "illegitimate" and its peace process efforts as "deceptive", in a statement calling for a boycott of an Islamic scholars' conference in Jakarta.
"Right now we want the Afghans to lead and to provide the substance of the reconciliation effort," Mattis said.
"What does that victory look like? It's a country whose own people and their own security forces handle law enforcement and any threats... certainly with international support for some years to come," he said.
The US has a renewed focus on Afghanistan after years of drawdowns under former president Barack Obama and talk by top US generals of "not winning" and of a "stalemate" in the seemingly intractable conflict.
"It's all working to achieve a political reconciliation, not a military victory," Mattis said.
"The victory will be a political reconciliation."
As part of the so-called South Asia Strategy, President Donald Trump last year ordered the increased bombing of Taliban targets -- including drug-making labs and training camps.
More than 3,000 additional US forces have also arrived in Afghanistan to boost the training and advising of local troops.
Approximately 14,000 American forces are now in Afghanistan, up from a low of about 8,500 when Obama left office.
Ghani's offer of peace talks comes as civilian casualties have soared in recent months, with the Taliban increasingly targeting towns and cities in response to Trump's new and more aggressive military policy.
The Taliban, Afghanistan's biggest militant group, claimed 472 attacks in January alone, the Washington-based terrorism research group TRAC said -- an astonishing number given that the traditional fighting season does not usually start until freezing temperatures have subsided in the spring.
Mattis said the jump in attacks on civilians was an indication that a pressured Taliban is unable to conduct broader, ground-taking operations.
The Afghan security forces were able to stop some attacks, Mattis said, though he noted he wanted to see them shift to a more "offensive mindset" in the coming months.
Mattis's surprise Afghan visit, his third as Pentagon chief, was kept under tight wraps after a security incident during his last trip in September, when insurgents shelled Kabul's airport hours after he arrived.
Mattis also said he had seen some changes in Pakistan's behaviour since Trump blasted the country last year for harbouring the Taliban.
"There are operations by the Pakistan military that are helping right now, ongoing as we speak," he said.
(This story has not been edited by Business Standard staff and is auto-generated from a syndicated feed.)