AI glossary · Ethics, safety and privacy
Red teaming
Red teaming is deliberately attacking an AI system, the way an adversary would, to find weaknesses before it is released.
Also called AI red teaming
In more detail
Red teams try jailbreaks, prompt injection, attempts to extract private data and requests for harmful help. What they find is used to strengthen guardrails.
Major AI labs red-team new models, often with outside experts, and some regulations expect it for the most capable systems.
Related terms
AI safetyAI safety is the work of making sure AI systems behave reliably and don’t cause harm, from everyday mistakes to serious misuse and long-term risks.AI jailbreakAn AI jailbreak is a prompt designed to trick a chatbot into ignoring its safety rules and producing content it is meant to refuse.GuardrailsGuardrails are checks and limits built into an AI product to stop harmful, off-topic or unauthorised outputs and actions.Frontier modelA frontier model is one of the most advanced AI models available at a given time, the kind that pushes the limits of what AI can do.
Browse related tools
AI tools listed on BroUseAI in these categories.
