In the wealthy district of Gangnam in Seoul, immortalized in the viral pop hit “Gangnam Style,” there are currently 447 cosmetic surgery clinics packed into an area of about 40 square kilometers (5 square miles). This “beauty belt” neighborhood is the heart of what is often called the cosmetic surgery capital of the world. A Statista survey from 2020, quoted in the Telegraph, found that 25 percent of 19-29-year-old and 31 percent of 30-39-year-old women in South Korea had undergone some form of plastic surgery – meaning that a quarter to a third of adult women in a country of 51 million people have had their face’s biometric signature surgically modified, some more than once.
As facial recognition becomes increasingly prevalent across diverse use cases, it will inevitably begin to collide with other social and technological trends. The modification of physical features to achieve certain aesthetic effects is one of these. Nor is the issue isolated to one part of the world; social media, influencer culture, TikTok videos and other drivers of the visual zeitgeist mean there is increased pressure on people everywhere to look a certain way. The Telegraph reports that last year, South Korea relaxed its immigration rules to accommodate surgical tourists; the ministry of health and welfare has set a goal of attracting 110,000 or more by 2027. Enchanted by the porcelain-skinned dolls of K-pop, this subset of medical tourists comes from around the world to seek new faces.
According to a CVPR2020 workshop paper entitled “Plastic Surgery: An Obstacle for Deep Face Recognition?”, over the preceding five years, out of more than ten million annually reported plastic surgeries, some 40 percent were performed on the head or face. Rhinoplasty, blepharoplasty (eyelid surgery) and jawline contouring all change the geometric qualities of the facial biometric signature. According to the paper, “eyelid corrections and facelifts together cover two thirds of all plastic surgeries done on faces followed by nose corrections which make up almost one quarter.” Decreased costs make operations more affordable
and thus widespread – “therefore representing a significant problem for security controls.”
Trying to fool a facial recognition algorithm with lifelike silicone masks is one thing. But permanently altering one’s facial geometry can pose complicated hurdles to matching for identity verification and other uses of face biometrics.
“Both corrective as well as cosmetic surgeries alter the original facial information to a large extent thereby posing a great challenge for face recognition algorithms,” says a 2020 paper by Bouguila and Khochtali, published in the Journal of Stomatology, Oral and Maxillofacial Surgery.
Plastic surgery, say the authors, can be “misused by individuals who are trying to conceal their identity with the intent to commit fraud or evade law enforcement,” and furthermore “may allow anti-social elements to freely move around without any fear of being identified by any face recognition system.”
“As plastic surgeons, we have the power to change faces and consequently identities.”
Researchers use neural networks to address FRT and plastic surgery
A recent paper entitled Face Plastic Surgery Recognition Model Based on Neural Network and Meta-Learning Model, published in the Journal of Computer Science, proposes using artificial neural networks to solve the problem. Researchers from universities in the UAE, Jordan and Gaza say their “framework for face recognition before and after undergoing plastic surgery based on an artificial neural network” was motivated by the rise in the popularity and availability of cosmetic surgery.
The paper says that in experiments using their proposed artificial neural network with model-agnostic meta-learning (ANN-MAML), the learning model attained an accuracy of 90 percent in facial recognition using before-and-after rhinoplasty images, 91 percent on blepharoplasty surgery images, 94 percent on brow lift images, and 92 percent on face lift images.
The authors of the CVPR2020 paper, meanwhile, think better data sets are the answer.
“Diverse well-known face recognition approaches, e.g. based on PCA or LBP, have been benchmarked mostly on the web-collected IIITD plastic surgery face database,” says their paper. “Generally, significant performance drops were reported when comparing facial images taken before and after plastic surgeries.”
For their experiment, the authors introduce a new data set, the so-called Hochschule Darmstadt (HDA) plastic surgery database of facial images taken before and after surgery. “This database vastly complies with the quality requirements defined by the International Civil Aviation Organization (ICAO) for electronic travel documents and comprises face images of the five most frequently applied facial plastic surgeries,” the paper says.
In general, the fresh data set yields positive results.
“In contrast to the experimental setups of related studies, these data sets of high quality face images, which are referred to as HDA plastic surgery database and non-surgery database, respectively, allow for a clear isolation of the effects of plastic surgery on face recognition and facilitate comparability of results.”
“Experimental results on the newly created high-quality plastic surgery database also reveal that once distortions from low quality face images are ruled out, as it happens in ICAO compliant images, most plastic surgeries do not cause significant errors by deep face recognition systems.” For both the commercial and open source facial recognition algorithms tested against the HDA plastic surgery database, facial bone correction had the most severe impact on accuracy, followed by eyebrow corrections.
Injections not a big deal for FRT, but could still raise flags
Luckily, less invasive forms of facial modification may not pose the same problems as surgical transformations. In an article in the UAE-based National News, Dr. Maurizio Viel, a plastic surgeon at the Cornerstone Clinic in Dubai, says injectables and fillers like Botox do not typically reshape the fixed points on your face that FRT systems use for matching. But these procedures and their results can still cause confusion when matching individuals with their passport images, for instance.
Patrick Bixby, a professor at Arizona State University and the author of License to Travel: A Cultural History of the Passport, says that while modern passports are supplemented with identifiers such as fingerprint, iris, and face biometrics, “it is important to remember that it is our bodies, our physical appearance, that must match the document, not the other way around.”
“In other words, the documentation is the authoritative piece. Cosmetic surgery will place a burden on passport holders to update.”
Beauty filters a problem, but beauty standards a bigger one
The quest to meet beauty standards shaped by airbrushed advertising and Hollywood has also presented problems for face biometrics that go beyond the surgically inclined. Many may have been introduced to digital face filters through social apps like Snapchat or Zoom, but the exponential advancement of smartphone camera technology, and the integration of AI, means that digital options for facial beautification now come standard-issue.
Meanwhile, K-pop maintains its grip on the pop charts and the collective idea of what a face should be. A 2023 article in Forbes observes that, “much the same way watching a Korean drama may prompt viewers to crave kimchi and soju, close-ups of seemingly flawless K-drama actors and K-pop idols can promote the desire to improve skin quality and even realign some facial features.”
Another race, then, in the 21st century technological derby: deep learning facial recognition algorithms versus cheap, minimally invasive surgical face modification. There are always iris biometrics, the modification of which with implants is widely recognized as causing significant damage and potential vision loss.
Then again, there seems to be nothing we won’t change to look a little more perfect.
Article Topics
accuracy | biometric identification | biometric matching | facial recognition | plastic surgery
https://www.biometricupdate.com/202404/does-facial-recognition-work-if-surgery-alters-a-persons-face