Examples
Worked examples
- Is an instance
An instruction-tuned model fine-tuned with 100k human preference comparisons over candidate responses.
- Is an instance
A DPO-trained model using preference data without an explicit reward-model step.
Counter-examples
Looks similar, but isn't
- Not an instance
Pure supervised fine-tuning on labelled instruction data (SFT, not RLHF).
- Not an instance
Pre-training next-token prediction on web text.
Editorial commentary
Reinforcement learning from human feedback (RLHF) is a fine-tuning technique that aligns a pretrained language model’s behaviour to human preferences by training a separate reward model on human comparisons between candidate outputs, then optimising the language model against that reward model using reinforcement learning (originally proximal policy optimisation, PPO). Its modern form for instruction-following chat models was established by Ouyang et al. (2022, InstructGPT), building on the earlier general RL-from-human-preferences framework of Christiano et al. (2017).
RLHF became the dominant alignment technique for chat-tuned LLMs released from 2022 onward, and remains widely used, but it is no longer the only technique in production use. Newer methods reformulate the same underlying goal (make the model prefer outputs a human would prefer) without a separate RL loop: Direct Preference Optimisation (DPO) fits the preference data directly via a classification-style loss; IPO and KTO are further variants addressing specific weaknesses in DPO’s assumptions. Constitutional AI‘s reinforcement phase is a related but distinct variant, RLAIF (RL from AI Feedback), which substitutes AI-generated preference judgments for the human comparisons RLHF depends on.
Why it matters for documentation and disclosure
Whether, and with what technique, a model was preference-tuned materially affects its behaviour — refusal patterns, verbosity, sycophancy toward the user’s stated view, and safety-relevant responses are all shaped by this stage far more than by pretraining alone. It belongs in a model’s fine-tune lineage record and, where relevant, in its model card: naming the base model without disclosing the alignment technique applied on top of it is an incomplete provenance record.
References
- Christiano et al., ‘Deep reinforcement learning from human preferences’ (NeurIPS 2017)
- Ouyang et al., ‘Training language models to follow instructions with human feedback’ (NeurIPS 2022)
Also known as
RLHF
Machine-readable encodings
Use in your systems
<role vocab="credit"
vocab-identifier="https://casrai.org/dictionary/"
vocab-term="RLHF (Reinforcement Learning from Human Feedback)"
vocab-term-identifier="https://casrai.org/dictionary/term/rlhf-reinforcement-learning-from-human-feedback" />{
"@context": "https://schema.org",
"@type": "DefinedTerm",
"@id": "https://casrai.org/dictionary/term/rlhf-reinforcement-learning-from-human-feedback",
"name": "RLHF (Reinforcement Learning from Human Feedback)",
"identifier": "https://casrai.org/dictionary/term/rlhf-reinforcement-learning-from-human-feedback",
"description": "A training methodology in which a language model is fine-tuned using a reward signal derived from human preferences over pairs (or larger sets) of candidate model outputs, typically by first training a reward model and then optimising the policy against it via PPO or a related algorithm.",
"inDefinedTermSet": "https://casrai.org/dictionary/domain/ai-ml-research-outputs#set",
"url": "https://casrai.org/dictionary/term/rlhf-reinforcement-learning-from-human-feedback",
"alternateName": [
"RLHF"
],
"license": "https://creativecommons.org/licenses/by/4.0/",
"publisher": {
"@id": "https://casrai.org/#organization"
},
"author": {
"@id": "https://casrai.org/#editorial-team"
},
"datePublished": "2026-05-21T02:22:51",
"dateModified": "2026-09-05T14:24:54",
"inLanguage": "en-GB",
"isAccessibleForFree": true
}
