Token
Also known as: model token, text unit
The smallest unit a model reads or writes, usually a fragment of a word, and the meter everything sent to a model is measured in.
Updated
What it means
A token is a piece of input or output smaller than a word. Common words are one token, unusual ones several, and images are converted into blocks that are counted the same way.
The concept only matters because it is the meter. Everything a model is given and everything it returns is counted in tokens, which is what makes a long video job a volume problem rather than a difficulty problem.
On video the split is lopsided. One frame is worth hundreds of tokens while the question is worth a few dozen, so Input token totals are dominated almost entirely by pictures.
How it works in practice
The practical lever on any video pipeline is how many frames get sent and at what size. Sampling and downscaling both reduce the total by large factors without much loss of judgement.
Answer length is the other half, and it is cheaper to control. Demanding a ranked list rather than prose keeps the Output token count small and the result parseable.
Audio is unusually good value by comparison. A soundtrack is cheap per second against frames, so it is normally worth sending in full rather than sampling it, which is how Audio event detection gets a complete signal without inflating the total.
What to watch out for
Token counts are not proportional to duration, which breaks naive planning. Ten minutes of a static shot and ten minutes of fast action sample to very different totals.
They also interact with the Context window in a way that surprises people. A sampling rate that fits a two minute clip overflows on an hour, so the rate has to scale with length.
Counting tokens is not the same as counting information. Hundreds of near identical frames inflate the total while adding nothing the model can use, and they displace frames that would have helped.
How RawClip handles it
Analysis is designed around how much has to be looked at rather than how long the footage runs, so frames are sampled and downscaled and answers are kept short. That is what makes examining a multi hour upload closely a practical thing to do.