How facial recognition is used to identify dead in Ukraine

Last month, a controversial facial recognition company, Clearview AI, announced that it had turned over its technology to the Ukrainian government.

The BBC has had access to evidence of how it is being used in over 1,000 cases to identify people living or dead.

It is reportage contains graphic descriptions that may disturb some readers.

A man lies motionless on the ground, his head bowed. His body is naked except for Calvin Klein boxer briefs. His eyes are surrounded by what appear to be bruises.

The body was found in Kharkiv, in eastern Ukraine. The BBC has seen photos of the man’s remains, but we don’t know the circumstances of the death.

There is clear evidence of head trauma. The man also had a tattoo on his left shoulder.

Ukrainian authorities did not know who the victim was and turned to a cutting-edge technology: facial recognition using artificial intelligence.

Clearview AI’s facial recognition system is the most famous — and controversial — in the world.

The company has collected billions of photos from social media, from sites like Facebook and Twitter, to create a massive database. Its president and founder, Hoan Ton-That, calls it a “face search engine”.

“It works like Google. But instead of putting words in, the user puts a picture of a face in the search field,” explains Ton-That.

The company faced several legal challenges. Facebook, YouTube, Google and Twitter have all sent notices to Clearview asking it to stop using images from its websites. The UK Information Commissioner’s Office even fined the company for not informing users about the collection of their personal photos.

In Ukraine

The Ukrainian government’s use of Clearview has raised questions about the implications of bringing this powerful tool to war.

Clearview is used extensively, albeit controversially, by authorities in the US. Ton-That says 3,200 government agencies have purchased or tried the technology.

After the Russian invasion of Ukraine, the company founder saw another application for his technology.

“We’ve seen images of prisoners of war, of people on the run, and we thought our technology could be useful for people identification and verification,” he says.

Ton-That made his search engine available to the Ukrainian government, and the offer was accepted.

Back in Kharkiv, authorities took a photograph of the dead man’s face: with his head held high and his sunken eyes turned towards the camera. They then verified the image against the Clearview database. The search turned up several photos of someone very similar to the dead man.

A photo had been taken on what appeared to be a hot day. The man was shirtless and had a tattoo on his left shoulder, just like the corpse. The search was successful and the authorities discovered the dead man’s name.

background

Using facial recognition to identify the dead is nothing new, and Clearview is not the only platform used for this purpose in Ukraine.

“We’ve been using this material for years,” says Aric Toler, director of investigations at Bellingcat, an organization specializing in investigative journalism.

In 2019, Bellingcat used facial recognition technology to help identify a Russian man who filmed the torture and murder of a prisoner in Syria.

The war in Ukraine is not the first time facial recognition technology has been used in a conflict.

But in Ukraine its use has been more widespread than in any previous conflict.

Toler told the BBC that he uses the FindClone facial recognition platform in Russia, which has been particularly useful in identifying dead Russian soldiers.

Like Clearview, FindClone combines images from the publicly available internet, including Russian social media pages. You can even find people who don’t have accounts on these sites.

“They may not have a social media profile, but their wives or girlfriends do. Or maybe they live in a small town with a large military base and have a lot of friends in their unit with accounts,” explains Toler.

This last point is critical to understanding the power of facial recognition technology.

This means that even a person who has never had a social media profile can be found. When appearing in a photo sent by a friend or simply being in the background of a random image on the internet, it is already in the database.

Even military or security personnel, who have little presence on the Internet, can be tracked.

a matter of accuracy

Critics of facial recognition point out that the technology is not infallible and that, in times of war, mistakes can have disastrous consequences.

Clearview is not just being used to identify dead bodies in Ukraine. The company also confirmed that it was being used by the Ukrainian government at checkpoints to help identify suspected enemies.

Clearview showed the BBC an email from a Ukrainian agency confirming that the system is being used to identify living people.

“This system gave us the opportunity to quickly confirm the accuracy of data on detained suspects,” reads the email from a Ukrainian official who declined to be named.

“While using Clearview, over a thousand queries were performed for proper verification and identification,” the email reads.

This worries some analysts.

Conor Healy is a facial recognition expert at IPVM, an organization that monitors security technology.

“It is important for Ukrainian forces to recognize that this is not a 100% accurate way to determine whether someone is your friend or enemy,” says Healy.

“This can’t be a life-and-death technology where you pass or fail, where you can be arrested or, God forbid, even killed. That’s not how it’s supposed to be used.”

Other analysts make even stronger warnings. Albert Fox Cahn of the surveillance group Surveillance Technology Oversight Project speaks of “a human rights catastrophe in the making”.

“When mistakes are made with facial recognition in peacetime, people are mistakenly arrested. When mistakes are made with this technology in a war zone, innocent people are killed,” Fox Cahn told Forbes magazine.

The BBC contacted the Ukrainian government to discuss the use of Clearview, but received no response.

Ton-That defends his company’s technology, saying it has a proven accuracy of over 99%.

However, this accuracy largely depends on the quality of the image, the position of the head, or whether the face is covered, for example, with a mask.

There’s also the issue of privacy, which has been problematic for Clearview in the US and Europe. The company pulls publicly available images from Facebook and Instagram to build its database.

But the company never obtained consent from the social media companies, or anyone else, to collect these images. You reading this article are almost certainly present in the database, although you probably didn’t give Clearview permission to use your image.

Last year, the UK Information Commissioner’s Office fined Clearview for failing to tell people it was collecting their personal photos on social media platforms.

Ton-That accepts that the legality of facial recognition technology is still being debated, but believes Clearview is operating within the law and says the technology is “misinterpreted”.

However, facial recognition technology clearly has dystopian applications. In November last year, the BBC reported that China was planning to use facial recognition technology against journalists.

Ton-That says Clearview would not allow such searches. He also guarantees that his company does not work with authoritarian governments and would not work with Russia.

However, Clearview’s technology can be applied in a military context.

Last year, for example, the company signed a contract with the Pentagon to explore the possibility of incorporating its technology into augmented reality glasses.

And Clearview is just one of several companies with military contracts to develop facial recognition artificial intelligence.

Privacy advocates also have another concern. Facial recognition technology could be useful to Ukrainian authorities in times of war. But will they return the technology to Clearview in peacetime?

“There are many examples of technologies being introduced in times of war and persisting in times of peace,” says Healy.

“I hope that’s not the approach they take.”