In a report marking the coming spring meeting, published just days before the scheduled Brexit date of Friday, the IMF looked at the impact of possible "no deal" scenarios.
In the worst-case situation, the fund assumes that a disorderly break between Britain and its largest trading partner would bring border disruption, raising import costs for businesses and households in Britain.
It estimates that the trade disruptions in that scenario would cause a decline in Britain's gross domestic product (GDP) of 1.4 percent in the first year, and 0.8 percent in the next.
The European Union would not be immune from the impact, although it would be less severe, with the bloc's GDP falling 0.2 and then 0.1 percent.
The IMF adds that the total impact would be a decline of 3.5 percent of British GDP between now and 2021 and 0.5 percent for the EU.
The fund notes however that it cannot predict all the effects of a "no deal" Brexit, or all the mitigating measures that might be taken.
The "no deal" scenarios assume that, in the absence of a new trade agreement, British exports to the EU revert to being subject to World Trade Organisation (WTO) rules.
This would see tariffs increase, while Britain would also lose access to trade agreements struck between the EU and other countries.
The IMF estimates are also based on published British plans to dramatically slash tariffs on imports from the EU.
It estimates an increase in GDP of 1.2 percent this year, rising to 1.4 percent in 2020. Previous projections unveiled at Davos predicted growth of 1.5 and 1.6 percent. The IMF report comes at the start of a decisive week, where EU leaders will decide at a Brussels summit on Wednesday whether to agree to London's request to delay Brexit day.
If they fail, or if Prime Minister Theresa May refuses to accept whatever they offer, Britain is on course to leave on Friday without a deal.