Trump told his Cabinet colleagues in the same meeting that he has ended $1.3 billion in aid to Pakistan because "this South Asian country houses enemies".
Earlier, South Carolina Senator Lindsay Graham, who is considered close to President Trump, told CNN in an interview that if Pakistan helped the US in bringing the Taliban to the table for negotiations, then the US would focus on counterterrorism and the IS.
Trump alleged that Pakistan has not been fair to the United States.
"We want to have a great relationship with Pakistan, but they house the enemy. They take care of the enemy. We just can't do that," Trump said.
Imran Khan was sworn in as Pakistan's Prime Minister in August last year.
The cricketer turned politician had said in January last year that meeting US President Trump would be a "bitter pill" to swallow should he become Pakistan's prime minister in elections later that year, but added "I would meet him."
Secretary of State Mike Pompeo had met Prime Minister Khan in Islamabad in September last year and pressed him to take "sustained and decisive measures" against terrorists threatening the regional peace and stability.
Just a month ago, Trump had written a letter to Khan, seeking Pakistan's help with stuttering Afghan peace talks and support in bringing the Taliban to the negotiating table to end the 17-year brutal war in the neighbouring country.