Prime Minister David Cameron had rejected the idea last week, warning that it would send out a 'terrible signal of Britain as a soft touch'.
Johnson defied warnings from Cameron to back the idea of allowing illegal immigrants to stay in the UK.
The ruling coalition has sought to take a tougher line on immigration, unveiling a raft of measures to curb benefit tourism and deport people in the UK illegally.
But Johnson said the failure of border agency staff and the police to find illegal immigrants and send them home meant there was already effectively an amnesty in place.
Appearing on his new monthly radio phone-in on LBC 97.3, the London Mayor said: "This is a chronic problem and if you look at what has happened in this country over the last 20 years, we have continuously failed to evict anybody."
"If you look at the number of people who are staying here illegally and you measure that against the number that are meant to be on planes, it is absolutely astonishingly small."
"The culture of human rights, the immense power of the ambulance-chasing lawyers who immediately come in and offer people protection against eviction, insert all sorts of delays into the procedure. It is blindingly difficult to get people on to planes.
According to media reports there are more than 500,000 illegal immigrants in the UK.
Johnson also said he could be tempted to run for mayor again - but only if hundreds of thousands of people line the streets demanding it.
The Mayor of London dodged questions about his ambitions to be Prime Minister when his term ends in 2016.
Instead he said he has been talking to friends about the possibility of breaking his promise to only serve eight years in City Hall.
