Face recognition
Also known as: face identification, person matching
Telling whose face is in a frame, as opposed to merely noticing that a face is present, which is a different and much lighter task.
Updated
What it means
Face recognition identifies a person. Face detection only reports that a face is somewhere in the frame, and the gap between the two is wide enough to matter legally as well as technically.
Detection is a routine computer vision task. Recognition builds a numerical description of a particular face and matches it against others, which is personal data by any reasonable reading of the term.
For editing, the attraction is obvious. Recognition is what would let a system keep the shots containing one particular child at a school sports day and skip the hundred containing everybody else.
How it works in practice
Matching works on embeddings rather than pictures. Each face becomes a vector, and two vectors close together are treated as the same person, which is the same idea behind Video embeddings applied to a much smaller crop.
Quality of the crop decides everything. A frontal, sharp, well lit face matches reliably, while a quarter profile at speed does not, so the frames fed to recognition are usually chosen deliberately rather than taken at random.
Action footage is a hostile case. Helmets, goggles, spray, motion blur and extreme angles remove most of what recognition depends on, so a chest mounted camera yields far fewer usable faces than a phone at a table.
What to watch out for
Accuracy is not even across people, and published error rates differ by skin tone, age and sex. Treating one threshold as equally safe for everyone in a frame is wrong, and the failures land on the same groups repeatedly.
The legal weight is the larger issue. Building a face template is processing biometric data in several jurisdictions, which brings consent and retention duties that plain Object detection does not.
That is why most editing pipelines deliberately stop at detection. Knowing a person is present is enough to weight a shot, and it avoids holding anything that identifies who the person was.
How RawClip handles it
Faces are detected so a shot with people in it can be weighted above an empty slope, but nobody is identified and no face template is ever stored. The analysis knows a person is there, not who they are.