Inference
Also known as: model prediction, running a model
Using a trained model to get an answer about new material, as distinct from training, which is the earlier work of building the model.
Updated
What it means
Inference is the model doing its job. Training happened once, somewhere else, on a large body of examples; inference is what runs every time footage is analysed, taking an input and returning an answer.
Everything an AI highlight generator does with a model is inference. No training happens while an upload is being processed, which is why the same footage analysed twice gives essentially the same result.
The distinction is practical rather than academic. Training needs enormous resources and happens rarely, while inference happens constantly, which is why the economics of analysing video is really about how many calls a job needs.
How it works in practice
Reducing the number of calls matters more than making each one faster. Sending a sample of frames instead of all of them, via Frame sampling, cuts the call count directly and by a large factor.
Deciding cheap questions arithmetically helps as much. Brightness and movement do not need a model at all, so the model is reserved for the questions nothing else can answer.
Calls are then grouped, because sending work in bulk keeps the hardware busy. That is the reasoning behind GPU inference and behind treating analysis as a queue rather than a conversation.
What to watch out for
Treating inference as free is the common planning error. Each call has a real duration, and a design that calls once per frame turns a ten minute clip into tens of thousands of calls and an unusable wait.
Results are also not perfectly stable. The same input can produce slightly different answers between runs and versions, so a pipeline assuming an exact repeat will see phantom changes in its own output.
Input preparation is easy to underestimate. Decoding, resizing and encoding frames for the model can take longer than the answers do, which makes Transcoding the real bottleneck in more pipelines than people expect.
How RawClip handles it
The analysis keeps the number of model calls low by measuring what can be measured and asking the model only about judgement, which is how a long upload can be examined without the wait becoming the product.