Automatic keyframe selection
Also known as: representative frame picking, thumbnail frame selection
Choosing the few frames that best represent a clip, used for thumbnails, preview strips and as the input a model actually looks at.
Updated
What it means
Automatic keyframe selection picks the handful of frames worth keeping out of the thousands in a clip. The chosen frames then stand in for the whole recording wherever something smaller than the video is needed.
They become the Thumbnail, the preview strip in a file browser, and in an analysis pipeline the frames a model is given instead of every frame on the card. One good choice therefore does a great deal of work.
Selection is not sampling at a fixed interval. A frame every two seconds hits plenty of blurred and half covered moments, so proper selection scores candidates and keeps the best one in each neighbourhood rather than the nearest one.
How it works in practice
The usual chain runs cheap checks first and expensive ones later. Frame sampling produces candidates, Blur detection throws out the soft ones, an exposure check drops the frames shooting straight into the sun, and what survives is ranked.
For a thumbnail the winner is a single frame. For analysis it is a spread across the clip, because three frames from the same second describe a recording far worse than three decent frames spread across it.
Spacing therefore matters as much as quality. Most implementations divide the clip into windows and allow only one winner per window, which guarantees coverage even when one stretch is far sharper than the rest.
What to watch out for
Optimising for sharpness alone produces dull results. The sharpest frame in a bike clip is often the rider standing still before the drop, while the moment anybody wants is slightly soft because it is moving fast.
Selection also inherits whatever the detectors miss. If no subject is recognised in a dark clip, the picker falls back on technical quality and returns a tidy frame of nothing at all.
That is how an edit ends up fronted by an empty slope. A cheap guard is to require that a chosen frame contains something, which is what Object detection is for at this stage.
How RawClip handles it
The frames chosen to represent each clip are also the frames the analysis reads, so a soft or empty pick wastes a look at that footage. Selection weighs sharpness, exposure and whether a subject is present, rather than sharpness on its own.