Face recognition algorithms are becoming increasingly capable of extracting identity information from poor images. But new research into low-resolution face recognition highlights a stubborn operational problem: better algorithms can recover more from degraded imagery, but only when useful identity information remains.
Researchers at Johns Hopkins University tested a new approach to recognizing faces affected by conditions common in surveillance imagery, including blur, occlusion, poor contrast, atmospheric turbulence and low resolution. Their research identifies three underlying challenges.
First, useful identity information can be sparse, particularly in video where only some frames contain discriminative information. Second, gallery and probe images can come from very different quality domains: a high-resolution reference image contains texture, landmarks and fine facial details that may no longer be available in a degraded probe. Third, adapting recognition algorithms to low-resolution imagery can degrade information previously learned from higher-quality faces, a problem known as catastrophic forgetting.
The researchers were able to improve recognition across low-resolution benchmarks including BRIAR, IJB-S and TinyFace. But their failure analysis also demonstrated the limits of what better recognition can recover. Faces below roughly 8 by 8 pixels can contain too little texture or shape information for reliable matching, while extreme pose, heavy occlusion, atmospheric turbulence and motion blur continued to cause errors.
The findings illustrate a broader challenge for operational biometric systems: how do you determine that an image is too poor before sending it to the face matcher?
The answer is becoming more complicated as recognition algorithms improve. Image quality and recognition utility are related, but they are not the same thing. An image that a quality assessment algorithm considers poor may still contain enough identity information for a newer or stronger matcher to recognize successfully. Conversely, an image that passes conventional quality checks can still contribute to a false non-match.
A useful quality assessment algorithm must therefore do more than detect blur, pose, lighting or other defects. It must predict whether those defects have removed enough identity information to cause the recognition system to fail.
That is the problem NIST, DHS, UNHCR and developers of tools such as OFIQ and BQAT are trying to quantify.
NIST tests whether poor quality predicts recognition failure
NIST’s ongoing Face Analysis Technology Evaluation (FATE) Quality Summarization Track tests whether quality assessment algorithms, or QAAs, can predict false non-matches.
For its Entry-to-Visa evaluation, NIST compares airport arrival webcam images, which include operational challenges such as varying illumination and backgrounds, with high-quality visa-like application photographs.
NIST sets a recognition threshold at which a set of 15 accurate FRTE verification algorithms produces a 1 percent false non-match rate (FNMR). It then removes the 1 percent and 5 percent of images that each QAA considers lowest quality and recalculates the FNMR.
If a QAA perfectly predicted recognition failure, removing the lowest-quality 1 percent should take FNMR from 1 percent to zero. In practice, NIST finds that QAAs do not always assign low quality values to images involved in false non-matches.
NIST applies a similar methodology to Kiosk-to-Entry comparisons, where lower-quality kiosk images are compared with medium-quality airport arrival webcam images. This time, NIST removes the lowest-rated 20 percent of samples. Even after discarding far more images than the original 1 percent error rate, QAAs do not consistently identify all samples responsible for false non-matches.
The results demonstrate the gap: the images a QAA identifies as worst are not necessarily the images a face matcher fails to recognize.
DHS finds matchers succeeding on low-quality images
The Open Source Face Image Quality tool, or OFIQ, is one of the industry’s major attempts to standardize face quality assessment. Developed under Germany’s Federal Office for Information Security (BSI), it is the reference implementation for ISO/IEC 29794-5 and provides individual quality measures along with a Unified Quality Score (UQS).
A U.S. Department of Homeland Security study found the same quality-utility disconnect from another direction.
Researchers tested OFIQ against 16 commercial face recognition systems and concluded that UQS provided “extremely limited utility” for the DHS use cases examined.
Commercial matchers successfully recognized many images OFIQ rated as very low quality, and filtering images according to OFIQ scores did not substantially reduce recognition errors.
That has direct consequences for recapture. Asking a traveler or customer to take another photograph creates friction and operational cost, but DHS found that additional captures did not always improve image quality enough to justify the intervention. OFIQ showed more potential for selecting the best image when multiple captures were already available.
The findings illustrate the central problem: as recognition algorithms become better at extracting identity from degraded imagery, a low image-quality score does not necessarily mean low recognition utility.
UNHCR exposes the operational tradeoff
UNHCR’s much larger OFIQ evaluation shows what that disconnect can mean operationally.
The organization evaluated OFIQ against a demographically diverse collection of 32 million images, including two million mated pairs. The relationship between OFIQ scores and face match scores was only weak to moderate, although UQS was the strongest factor contributing to prediction, followed by the age of the photograph.
UNHCR found that a UQS threshold of 25 could still provide operational value because removing images at the lowest end of the quality distribution produced a disproportionate improvement.
But the tradeoff was substantial. About 23.7 percent of active enrollment images fell below UQS 25, while roughly 90 percent of those images could still be used effectively for face recognition.
A quality threshold can therefore remove disproportionately problematic images while simultaneously rejecting large numbers of images that a recognition system could successfully use.
A separate large-scale evaluation by eu-LISA and the European Commission’s Joint Research Centre provides a more encouraging data point. An as-yet unpublished evaluation of operational Visa Information System data found a correlation between higher UQS and lower FNMR.
That does not necessarily contradict the DHS findings. The evaluations involve different image sources, populations and recognition systems, so the relationship between quality and recognition failure may not transfer from one operational environment to another. DHS evaluated OFIQ against multiple commercial matchers in its use cases, while eu-LISA and JRC examined VIS data.
The divergence reinforces the broader issue: the predictive value of a quality score can itself depend on the images, population, matcher and operating environment against which it is tested.
For now, eu-LISA continues to use its own quality assessment tool for face biometric enrollment.
More measurements can describe quality, but can they predict failure?
OFIQ is not the only approach.
Biometix added face analysis based on OFIQ to its open-source Biometric Quality Assessment Tool (BQAT) in 2024. It subsequently expanded BQAT with measures including eye-color detection, gaze direction, brightness, contrast and dynamic range.
Those measures can provide a more detailed description of what is wrong with an image. But adding more measurements does not necessarily solve the prediction problem exposed by NIST, DHS and UNHCR.
A system may correctly identify that a face is poorly illuminated, off-angle or blurred. The harder question is whether those deficiencies have removed enough identity information to make the particular recognition algorithm processing it fail.
OFIQ 2 attempts to close the gaps
Those operational findings are feeding into the next generation of OFIQ.
OFIQ 2, being developed as the reference implementation for the revised ISO/IEC 29794-5 standard, will retain OFIQ’s 27 quality components and may add another. Potential improvements include better treatment of motion blur, gaze direction, background uniformity and expression neutrality, along with computational improvements and efforts to reduce demographic differentials.
The underlying algorithms may change as well. CenterFace is being considered to replace SSD for face detection, while PPFL could replace ADNet for landmark estimation. Candidate algorithms are being assessed for accuracy, computational complexity, demographic differentials and licensing. Alternatives to OFIQ 1’s Unified Quality Score algorithm are also under consideration.
Operational users are feeding experience back into development. Javier Galbally has explained how eu-LISA passes feedback to the OFIQ development team run by BSI and secunet and maintains a regularly updated webpage addressing frequently asked questions about the tool.
secunet plans to make a pre-release of OFIQ 2 available in the fourth quarter of 2026, providing more time for community review than the first release. A new demonstrator will follow, while webcam support and additional visualization capabilities are also in development.
But the evaluations of the current generation set a higher bar for OFIQ 2 than simply detecting more image defects. The key question will be whether the revised quality components and UQS improve the ability to predict recognition failure across different matchers and operational environments, rather than only providing a more detailed description of image quality.
The definition of “good enough” keeps moving
The research into low-resolution recognition and the quality-assessment studies describe two sides of the same problem.
Recognition algorithms are becoming better at extracting identity from increasingly degraded imagery. Quality algorithms are trying to determine when an image has become too degraded to use. But there may be no universal quality threshold separating those two states.
NIST shows that the images receiving the lowest quality scores are not necessarily the ones producing false non-matches. DHS shows commercial matchers successfully recognizing many images assigned very low OFIQ scores. UNHCR demonstrates the operational tradeoff of rejecting such images, while the different results reported by DHS and eu-LISA/JRC suggest that the predictive value of quality can vary with the matcher, dataset and operating environment.
Yet better recognition cannot eliminate the physical limit. The Johns Hopkins research found that severe degradation can eventually leave too little discriminative identity information for even an improved recognition algorithm to reliably extract.
The operational challenge, therefore, is no longer simply determining whether an image is good or bad. It is determining whether that image contains enough usable identity information for the recognition system that will actually process it.
Article Topics
Biometix | biometric data quality | biometric matching | Biometric Quality Assessment Tool (BQAT) | DHS | facial recognition | ISO 29794-5 | Open Source Face Image Quality (OFIQ)