AI glossary · Generative AI and LLMs
Reinforcement learning from human feedback (RLHF)
Reinforcement learning from human feedback (RLHF) trains an AI model to give better answers by learning from people’s ratings of its responses.
In more detail
People compare pairs of answers and pick the better one. Those choices train a scoring model, which then guides the AI toward answers people prefer, using reinforcement learning.
RLHF is a big part of what turned raw language models into helpful, polite chatbots. A side effect can be sycophancy: models learning to tell people what they want to hear.
Example
When a chatbot asks “Which response do you prefer?”, your choice may be used as this kind of feedback.
Related terms
Reinforcement learning (RL)Reinforcement learning (RL) trains an AI by trial and error: it takes actions, gets rewards or penalties, and learns which choices lead to the best results.AI alignmentAI alignment is the challenge of making an AI system’s goals and behaviour match what people actually intend and value.AI sycophancyAI sycophancy is when a chatbot tells you what you want to hear, agreeing with you or praising your ideas, even when that makes its answer less accurate.Fine-tuningFine-tuning is extra training that adapts an existing AI model to a specific task, style or field, using a smaller, focused set of examples.
Browse related tools
AI tools listed on BroUseAI in these categories.
