Highlight detection model
Also known as: moment detection model, interestingness model
A model trained to answer one question about footage: which stretches would a viewer want to watch, and in what order of strength.
Updated
What it means
A highlight detection model predicts interest rather than content. Other models say what is in a frame; this one says how much a given stretch matters compared with the rest of the upload.
The output is a curve over time, and its peaks are the candidates for a cut. Everything downstream, including Smart trimming, works from that curve rather than from the footage directly.
Interest is learned from examples of what people kept and what they discarded, which means the model carries somebody else taste. That is both why it works on footage it has never seen and why it sometimes disagrees with the person who was there.
How it works in practice
Scores are relative within an upload, not absolute. The strongest moment of a dull session still scores highest in that session, so the model ranks rather than judges quality in any universal sense.
Several signals usually feed it. Motion, faces, speech and audio peaks all contribute, and the model learns how to weigh them against each other for the kind of activity in hand.
The target length then decides how far down the ranking to read. A short cut takes only the peaks; a longer Rough cut reaches into the middle of the curve and picks up context around them.
What to watch out for
Repetition defeats it. Six near identical park laps all score well individually, so the cut becomes a loop unless Duplicate shot detection intervenes before the selection is made.
It also undervalues the deliberate quiet shot. A long landscape wide carries no motion and no speech, scores low and gets dropped, along with the breathing space it would have given the sequence.
Training data bias shows up as genre blindness. A model trained mostly on team sports reads a slow technical climb as nothing happening, which is a judgement about its examples rather than about the footage.
How RawClip handles it
Every stretch of an upload is scored for how much a viewer would want it, and the cut is built from the top of that ranking down to the target length, which is why two uploads of the same activity come back different.