Vector similarity
Also known as: cosine similarity, embedding distance
Comparing two pieces of footage by how close their numerical descriptions sit, which is how a machine judges that two clips are alike.
Updated
What it means
Vector similarity is the arithmetic behind every comparison a model makes. Each clip or frame becomes a list of numbers, and closeness between two lists stands in for similarity of meaning.
The representation comes from somewhere else, usually Video embeddings, and this is just the measurement applied to it. Nothing about the measurement knows anything about video.
That is both its strength and its limit. It compares whatever the representation captured, so if the embedding ignored motion then two clips differing only in movement will measure as identical, and no threshold will ever separate them.
How it works in practice
The common use is grouping repeats. Clips whose vectors sit within a small distance of each other are treated as the same event, which is how a Near-duplicate clip is identified at all.
The second use is retrieval. A request is turned into a vector and the nearest clips are returned, which is the whole mechanism behind Semantic video search over a card of unlabelled files.
Distances are only meaningful within one representation. Comparing numbers produced by two different models is not a comparison of anything, so thresholds never transfer between them.
What to watch out for
There is no natural boundary between similar and different, only an ordering. Every threshold is a judgement, and the right one differs between grouping repeats and finding related footage.
Similarity also ignores quality entirely. The sharpest and the blurriest version of a moment sit close together, so the group has to be ranked afterwards by Quality scoring to pick a survivor.
High dimensional distances behave unintuitively as well. Almost everything is far from almost everything else, so small differences in threshold produce large swings in how much gets grouped, which makes the number genuinely hard to set by intuition.
How RawClip handles it
Clips are compared by their numerical descriptions rather than by filename or timestamp, which is what lets repeated attempts at the same trick be recognised as repeats and collapsed to the single strongest one before the cut is built. The same arithmetic is what finds comparable moments across a card that carries no labels beyond the dates the camera wrote.