"We still have an overhang of a very large public debt...maybe in next 4-5 years that is the area where we will need to focus," Garg said at the FICCI Annual General Meeting here.
Rating agencies have raised concerns over the level of public debt and refrained from upgrading the sovereign rating of the country.
"Most credit rating agencies give a lot of weightage to the debt to GDP ratio of country, we focused more on the fiscal deficit presently. But going forward, that is the area where we will focus," he said.
On inflation, he said, "I think we have more or less conquered, I wouldn't say things may not go wrong, can go wrong sometimes, there are certain external factors like oil... which are not completely in our control. But I think we are reasonably assured that the inflation is less of a worry."
Headline inflation fell to a 17-month low of 2.33 per cent in November, primarily due to a favourable base effect and fall in prices of vegetables and cereals.
Garg also said India can become a $10 trillion economy by 2030 if some of the concerns are addressed.
"We can build a new economy whether you talk about 2025, 2022, 2030, a $10 trillion economy by 2030 or $5 trillion by 2025, all these are feasible, we can achieve it if we pay attention to all these major traits," he said.
He exhorted the industry to invest in the infrastructure sector where there is huge deficit.
Infrastructure is an area where more funds are needed and it also provides an investment opportunity for businesses, he said.
Speaking about Friday's board meeting of the Reserve Bank of India, Garg said the board discussed governance, liquidity and other important issues.
All these issues would be examined and discussion will take place between the Reserve Bank and the government, he said, adding that there is no agenda for the next board meeting as of now.
Asked if there was any discussion on interim dividend, he replied in the negative.
Addressing the event, Department of Industrial Policy and Promotion (DIPP) Secretary Ramesh Abhishek said the government will soon unveil a new industrial policy.