Examples
Worked examples
- Is an instance
Calling the GPT-4 API at temperature=0 to make outputs more reproducible
Counter-examples
Looks similar, but isn't
- Not an instance
Training the model on new data is not inference — it is training (or fine-tuning)
Editorial commentary
Inference is the process of generating outputs from an already-trained AI model in response to inputs at runtime, as distinct from training, which updates the model’s parameters. For a large language model, inference is the production of a completion from a prompt; it consumes compute, time, and energy per query, but does not change the underlying weights.
Inference compute is now its own governance question
Historically, disclosure and regulatory attention focused almost entirely on training compute (see compute (FLOPs estimate)), because that was where the bulk of a model’s compute cost sat. Newer “reasoning” model architectures deliberately spend substantially more compute at inference time — generating and evaluating intermediate reasoning steps before producing a final answer — shifting a meaningful share of total compute, cost, and energy use into the inference stage. This is sometimes called inference-time or test-time compute scaling, and it complicates any FLOPs-threshold regulation written with only training compute in mind: two models with identical training compute can now have very different inference-time compute profiles.
Why it matters for research
Reporting a model’s output without recording the exact inference conditions — model version, sampling/temperature settings, and for reasoning models, any inference-time “thinking budget” — is an incomplete disclosure, comparable to reporting an experimental result without recording instrument settings.
References
Also known as
Model inference · LLM inference
Machine-readable encodings
Use in your systems
<role vocab="credit"
vocab-identifier="https://casrai.org/dictionary/"
vocab-term="Inference"
vocab-term-identifier="https://casrai.org/dictionary/term/inference" />{
"@context": "https://schema.org",
"@type": "DefinedTerm",
"@id": "https://casrai.org/dictionary/term/inference",
"name": "Inference",
"identifier": "https://casrai.org/dictionary/term/inference",
"description": "The process of generating outputs from a trained AI model in response to inputs at runtime, distinct from training (which updates model parameters); for LLMs, inference is the production of completions from prompts.",
"inDefinedTermSet": "https://casrai.org/dictionary/domain/genai-disclosure#set",
"url": "https://casrai.org/dictionary/term/inference",
"alternateName": [
"Model inference",
"LLM inference"
],
"license": "https://creativecommons.org/licenses/by/4.0/",
"publisher": {
"@id": "https://casrai.org/#organization"
},
"author": {
"@id": "https://casrai.org/#editorial-team"
},
"datePublished": "2026-05-21T01:57:35",
"dateModified": "2026-09-05T17:30:34",
"inLanguage": "en-GB",
"isAccessibleForFree": true
}
