Covid masks thwarting facial recognition tech, will investigate: US agency

Even the best commercial facial recognition systems have error rates as high as 50% when trying to identify masked faces, says study

Having a tough time recognising your neighbours behind their pandemic masks? Computers are finding it more difficult, too.

A preliminary study published by a US agency on Monday found that even the best commercial facial recognition systems have error rates as high as 50% when trying to identify masked faces.

The mask problem is why Apple earlier this year made it easier for iPhone owners to unlock their phones without Face ID. It could also be thwarting attempts by authorities to identify individual people at Black Lives Matter protests and other gatherings.

The National Institute of Standards and Technology says it is launching an investigation to better understand how facial recognition performs on covered faces. Its preliminary study examined only those algorithms created before the pandemic, but its next step is to look at how accuracy could improve as commercial providers adapt their technology to an era when so many people are wearing masks.

Some companies, including those that work with law enforcement, have tried to tailor their face-scanning algorithms to focus on people's eyes and eyebrows.

NIST, which is a part of the Commerce Department, is working with the US Customs and Border Protection and the Department of Homeland Security's science office to study the problem.

It tested the software by drawing digital masks onto the faces in a trove of border crossing photographs, and then compared those photos against another database of unmasked people seeking visas and other immigration benefits.

The agency says it scanned 6.2 million images of about 1 million people using 89 algorithms supplied by tech firms and academic labs.

Under ideal conditions, NIST says the failure rate for the best facial recognition systems is only about 0.3%, though research has found significant disparities across race, gender and age. Add masks and the failure rate rises to 5% or worse. When confronted with masks, the agency says, many otherwise competent algorithms failed between 20% to 50% of the time.

Even before the coronavirus pandemic, some governments had sought technology to recognise people when they tried to conceal their faces.

Face masks had become a hallmark of protesters in Hong Kong, even at peaceful marches, to protect against tear gas and amid fears of retribution if they were publicly identified. The government banned face coverings at all public gatherings last year and warned of a potential six-month jail term for refusing a police officer's order to remove a mask.

Privacy activists, in turn, have looked for creative ways to camouflage themselves. In London, artists opposed to high-tech surveillance have painted their faces with geometric shapes in a way that's designed to scramble face detection systems.

Then came the coronavirus outbreak, when health experts around the world began strongly encouraging everyone to wear masks that cover the mouth and nose.

NIST's preliminary study says what masks people wear, and how they wear them, makes a difference to facial recognition systems. The results are mostly unsurprising: The more facial features that are covered, the harder it is to recognise the person beneath the mask.