Facial recognition: Big Brother is watching you

Facial recognition technology, you might think, belongs in a dystopian nightmare. Imagine cameras tracking your every move, knowing exactly who you are, everything you've done, what you like and who you meet. But it's fiction no longer. Married with artificial intelligence (AI), the swarm of cameras perched on buildings and lamp posts the world over can now do all that and much, much more. And it won't be long before the technology is everywhere.

Of course, there are non-Orwellian uses of the technology. Your phone's camera can automatically pick out the faces of people you want to capture and make sure they're in focus. You can unlock your phone just by looking at it. Your face can open doors, with many companies now using security systems that check faces and unlock restricted areas to authorised employees. But with technology this powerful, it's inevitable that some of the biggest users are governments and security forces. If you've crossed a border recently, you may well have been obliged to stare into a camera comparing your visage to the face on the passport. That sounds reasonable. But state use goes much further than that.

Keeping an eye on you

UK police forces have started using facial recognition systems in the last four years, notes The Guardian. South Wales Police has trialled the technology to scan faces in crowds at major sporting events and check them against a "watch list" drawn from the 12.5 million faces on the police national database.

Subscribe to MoneyWeek

Subscribe to MoneyWeek today and get your first six magazine issues absolutely FREE

Get 6 issues free

Sign up to Money Morning

Don't miss the latest investment and personal finances news, market analysis, plus money-saving tips with our free twice-daily newsletter

Don't miss the latest investment and personal finances news, market analysis, plus money-saving tips with our free twice-daily newsletter

Sign up

In Malaysia, police are using facial recognition software built by Chinese startup Yitu Technology on their body cameras to scan and identify faces. And in Singapore, the government is aiming to install cameras on more than 100,000 lamp posts which may, it says, "be used for performing crowd analytics and supporting follow-up investigation in the event of a terror incident."

But nowhere has adopted the technology more enthusiastically than China. The Chinese government plans to be able to identify "anyone, anytime, anywhere in China within three seconds", says the South China Morning Post. Its spending on video surveillance makes up 46% of the $17.3bn global market, according to Radio Free Asia, a US-government backed broadcaster.

The most extreme case is in Xinjiang province, where the government has set up a network of cameras to monitor the minority Uighur population. Individuals who have come to the state's attention are watched around the clock. If they stray more than 300 metres from "safe areas" their home or place of work, for example the police are alerted. The Uighurs' homeland "has become an open-air prison", says lshat Hasan, president of the US-based Uyghur American Association exile group.

A boon for consumers?

But it's not all about re-education camps and oppressive surveillance. The potential consumer applications are endless. There is no need for a key ever, anywhere. Your car, house, work and hotel room can all be unlocked by just walking up to it; your face can be instantly and silently scanned. If you've opened a bank account with one of the new challenger banks you may have been asked to take a selfie and upload some photo identification (ID). Want to pay at a shop? Just look this way, and off you go, purchases in hand.

JetBlue Airways in the US is using facial recognition as part of its boarding process. Royal Caribbean is doing the same for cruises. Liverpool-based Human Recognition Systems sells a suite of software designed specifically for this purpose. One Chinese insurer, Ping An Insurance Group, is using the technology to screen customers for loans.

It is mainly used to verify identities, but it also serves as a high-tech lie detector to "analyse the faces of loan applicants in real time, searching for micro-expressions' that reveal their emotional and psychological state", says The Wall Street Journal. China's Tencent is trialling the technology to detect the faces of minors playing its video games: Beijing is concerned that children's excessive gaming is damaging their health.

Shops are signing up

Retailers are embracing the technology too. In Thailand, convenience store chain 7-Eleven is to use the technology in its 11,000 shops to spot loyalty scheme members, gauge customers' emotions and suggest products to them. In China, both Tencent and Ant Financial Services are installing screens at points of sale to process cashless transactions, says The Wall Street Journal. Wedome, a bakery chain, says using them has greatly increased efficiency, and that 70% of its customers choose to pay this way.

A few years ago, US retail behemoth Walmart tested the technology to spot shoplifters, says Fortune. It scanned the faces of everyone entering its shops and compared them against a database of offenders. Similar technology is now sold by California's FaceFirst to retail clients. Chief executive Peter Trepp says 40% of its business now comes from the retail sector, and he recently demonstrated at an industry conference how his company could profile customers, with details of the time they'd spent in the shop on past visits, how they moved through the store, and what they'd bought.

Germany's Cognitec makes systems for ID verification, border security and "VIP recognition" for high-end retailers and casinos so they can spot the big spenders and give them special treatment. It's also in the business of targeted advertising. As you pass a billboard your face is scanned, matched against your profiles, and an ad is displayed just for you. You may have an adblocker on your web browser, but how do you install one for the real world?

The backlash is building

One problem the technology has is public perception. We may have no objection to taking a selfie to open a bank account or to unlock our phones. But coupled with AI and so-called "big data" data sets whose size or type is beyond the ability of traditional relational databases to capture, manage and process many commercial applications verge on the sinister. Retail executives are salivating at the idea of scanning everyone who enters the shop, matching their faces to their public social media profiles or their browsing habits, or even measuring their mood to evaluate their satisfaction with their shopping experience; Walmart has submitted patents for such technology, says Business Insider. We may not object to shops matching faces to known shoplifters, but how many customers would really be totally comfortable with such an intrusive (or "optimised") shopping experience?

A backlash is already building. South Wales Police and the Met have been criticised by the UK's biometrics commissioner, reports The Guardian. San Francisco recently became the first major American city to ban the police and other public authorities from using facial recognition technology. And Chinese police had to apologise to a high-profile businesswoman after its AI system, which identifies jaywalkers and displays their identities on big public screens to shame them, picked out her face from an advert on the side of a bus.

The companies to look out for

Investing in the sector is tricky. There are plenty of startups involved, although none are yet listed on stockmarkets. In Silicon Valley alone, venture capitalists made an average total investment of $78.7m in the last three years, says Forbes. Names to keep on a watchlist include FaceFirst and Cognitech, while the UK is home to Human Recognition Systems. Sightcorp is a Dutch company spun out from the University of Amsterdam; keep in mind also France's Idemia.

The minnows are chasing a global facial recognition software market expanding by 20% a year and likely to be worth $9bn a year by 2022, estimates market research Future, a consultancy. It's no surprise, then, that Big Tech is dabbling in the sector, too. Facebook launched its DeepFace facial recognition system in 2014. Google's FaceNet programme came along a year later. These two companies have some of the biggest datasets of faces in the world, while Microsoft and Amazon both have huge databases of their own. Apple bought Israeli company RealFace in 2017. But many are struggling to exploit their products while staying on the right side of the ethical divide.

Amazon's product, for instance, is called Rekognition. But investors are concerned that its use in state surveillance could damage the company's reputation and its profits, reports The New York Times. Shareholders recently tabled motions at Amazon's annual meeting to prohibit sales to government agencies and to commission an independent report into whether the technology could threaten human rights and thus dent Amazon's profits.

China's big players

It's a similar story in China. Two of the biggest players there are SenseTime and Megvii, also known as Face++, of Beijing. Megvii recently secured $750m in funding, bringing the total raised since it was founded in 2011 to $1.4bn. SenseTime has raised $2.6bn since founding in 2014. Both are reported to be considering floating on the stockmarket later this year, reports the Financial Times.

Megvii's technology is being used by the Chinese authorities, while SenseTime recently sold out of a "smart policing" joint venture there. With backers including Fidelity International and Qualcomm, it has been suggested that it was keen to avoid putting off foreign backers. But that doesn't seem to have deterred the likes of Australian investment bank Macquarie, reported to have bought in to Megvii's latest funding round. Finally, other Chinese firms developing the technology include Anyvision, AliBaba and Tencent.