Face recognition
Detection, verification, identification: three tasks with very different legal consequences, and why confusing them is expensive.
The three tasks
| Task | Question | Example |
|---|---|---|
| Detection | Is there a face here? | Camera focuses, face blurred out |
| Verification | Is this the person they claim to be? | Unlocking a phone, video identification |
| Identification | Who is this? | Searching a database |
Only the first avoids biometric data, and only if the result is not stored. The other two process biometric data within the meaning of Art. 9 GDPR.
How a comparison works technically
- 01
Detection
A detector finds faces in the image and outputs rectangles.
- 02
Alignment
Five keypoints rotate and scale the face into a standard pose. Without this step quality drops noticeably.
- 03
Embedding
A network produces a vector, usually 512 values, characterising the person.
- 04
Comparison
Cosine distance between two vectors, held against a threshold.
The resulting vector is itself biometric data. It is not anonymised even if the image was deleted: the same person can be recognised from it, and current methods can approximately reconstruct a facial image from it.
What has to be measured
- False acceptance and false rejection rates separately, not as one number.
- Both rates broken out by skin type, age and gender.
- The threshold fixed once and documented, not adjusted in operation.
- A non-biometric fallback that can be used without disadvantage.
Why identification differs from verification
Worked through at a very good false acceptance rate of 1e-6:
| Database size | P at least one false hit |
|---|---|
| 1,000 | 0.10 % |
| 100,000 | 9.5 % |
| 1,000,000 | 63.2 % |
| 10,000,000 | 99.995 % |
At a million entries, almost every other query returns a false hit although the system errs only once in a million per comparison. Which is exactly why the AI Act treats identification and verification as fundamentally different.
Unequal error distribution
Large-scale comparative tests, including the NIST FRVT series, consistently show differences in false acceptance rate across demographic groups, historically by up to two orders of magnitude. Newer models have narrowed but not removed the gap.
The practical consequence: an overall error rate is not sufficient evidence. Technical documentation under the AI Act must state per-group error rates, and operating without that breakdown cannot be shown to be non-discriminatory. See Bias in data.
The audit trail
Every identification query belongs in a log: time, reason, requesting office, result, decision taken, and whether a human reviewed it. Without that log neither a data subject request nor a supervisory inspection is possible. See Logging.
Related courses and sources
Gender Shades
The study showing how far face recognition misses depending on skin tone and gender. The trigger for today's rules.
For any discussion of face recognition: the study that triggered today's rules.