Unlocking a phone with your face feels convenient, but the same technology can also be used to identify people in crowds without their knowledge. Face recognition is one of the most powerful and most debated applications of AI. This article explains how it works and why society is arguing about it.
Detection, verification and identification
These terms are often confused.
- Face detection: finding where faces are in an image. It does not say who they are.
- Face verification (1-to-1): checking whether a face matches a claimed identity, such as unlocking your own phone.
- Face identification (1-to-many): searching a database to find who a face belongs to.
- Face analysis: estimating attributes such as age range or expression. Attempts to infer emotions or traits from faces are scientifically controversial.
Verification and identification have very different risk levels. Matching you to your own phone is usually low risk. Searching a large database for strangers is high risk.
How face recognition works
Step 1: detect and align
The system locates a face in the image and finds landmarks such as eyes, nose and mouth. It then rotates and scales the face to a standard position so that comparisons are fair.
Step 2: create an embedding
A deep neural network converts the aligned face into a vector of numbers, often 128 to 512 values, called a face embedding or template. The network is trained so that photos of the same person produce vectors that are close, while photos of different people produce vectors that are far apart.
Step 3: compare
To check a match, the system measures the distance (or cosine similarity) between two embeddings. If the distance is below a chosen threshold, it declares a match.
Step 4: decide with a threshold
The threshold controls the trade-off:
- A strict threshold reduces false matches (accepting the wrong person) but increases false non-matches (rejecting the right person).
- A loose threshold does the opposite.
The right setting depends on the use. Unlocking a phone differs from granting access to a secure facility.
Anti-spoofing
Systems must resist attacks using printed photos, screen replays or masks. Liveness detection uses cues such as blinking, depth sensors, infrared light or analysing texture to check that a real face is present.
Where it is used
- Phone and laptop unlocking
- Border control and airport gates
- Workplace or school attendance and access control
- Photo apps that group pictures by person
- Banking and identity verification (matching a selfie to an ID photo)
- Law enforcement investigations
- Finding missing people
- Retail analytics and security (more contentious)
A short code example
The open-source face_recognition library (built on dlib) shows the idea. Only use it on images where you have permission.
import face_recognition
known_image = face_recognition.load_image_file("person_a.jpg")
unknown_image = face_recognition.load_image_file("candidate.jpg")
known_enc = face_recognition.face_encodings(known_image)[0] # 128-number embedding
unknown_encs = face_recognition.face_encodings(unknown_image)
for enc in unknown_encs:
distance = face_recognition.face_distance([known_enc], enc)[0]
match = distance < 0.6 # common default threshold
print(f"distance={distance:.2f} match={match}")Lower distance means more similar. Production systems add liveness checks, quality checks, encryption and audit logs.
Why it is controversial
Accuracy differences
Independent evaluations, including large tests by national standards bodies, have found that error rates can differ across demographic groups depending on the algorithm, with some systems performing worse on women, darker skin tones or older and younger people. Newer algorithms have improved, but differences and variation between products remain, so testing on the relevant population is crucial.
Privacy and surveillance
Cameras combined with recognition can track people's movements in public spaces without consent. A face, unlike a password, cannot be changed if it is leaked. Biometric databases are attractive targets for attackers.
Consent and misuse
Photos scraped from the internet have been used to build databases without people's agreement. Systems can be misused for stalking, harassment or suppressing protest.
Wrongful identification
Mistaken matches have led to wrongful arrests in some reported cases. If a match is treated as proof rather than as a lead, harm can occur.
Chilling effect
Knowing you may be identified can discourage people from attending gatherings or expressing opinions.
Legal responses
Some cities, regions and countries restrict or regulate the use of face recognition, especially by public authorities or in public spaces, while others allow it with safeguards. Rules vary and are evolving.
Responsible use principles
- Have a legitimate, specific purpose and use the least intrusive method that works.
- Obtain informed consent where appropriate and give people alternatives, such as a PIN.
- Test for accuracy across groups and set thresholds suited to the risk.
- Never treat a match as proof. Require human review and corroborating evidence.
- Secure biometric data, encrypt it, limit retention and restrict access.
- Be transparent about when and why the system is used.
- Provide ways to challenge decisions and correct errors.
- Follow applicable laws and ethical guidelines.
Frequently Asked Questions
Is face recognition the same as face detection?
No. Detection only finds faces. Recognition determines identity or checks a match.
Does my phone send my face to the cloud?
Many phones store and process face data on the device in secure hardware. Policies differ between devices and apps, so check the settings.
Can face recognition be fooled?
Basic systems can be tricked by photos or masks, which is why liveness detection and multiple factors are used in secure applications.
Should schools and offices use it for attendance?
It raises privacy and consent issues, especially with children. Simpler alternatives such as cards or manual registers may be adequate and less intrusive.
Conclusion
Face recognition converts faces to embeddings and compares them with thresholds. It is useful for convenience and security, but it carries real risks around accuracy, privacy and misuse. Careful design, clear limits and legal safeguards are essential.