Examples
Examples from published documents
- OpenAI used automated red-teaming to search for universal jailbreaks, with optimisation-based search, reinforcement learning and test-time search totalling over 700,000 A100e GPU hours of compute (GPT-5.6 system card, section 9.4.4)
- Four private red-teaming organisations tested GPT-5.6 for more than a month from 3 June 2026, and OpenAI states that all findings were mitigated before launch (GPT-5.6 system card, section 9.4.5)
- Google DeepMind's CBRN red teamers each work on harm-journey sub-stages matching their expertise, use a model version without out-of-model safeguards, and score findings with a sub-stage rubric (Gemini 3.7 Flash FSF report, p.9)
Counter-examples
Not to be confused with
- Running a fixed set of known jailbreak prompts automatically is a benchmark of jailbreak robustness, not a red-team exercise, unless attackers adapt their strategy to the model during testing.
Related terms
Related AI safety terms
Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.
- Jailbreak
- A jailbreak is an adversarial prompt or technique designed to get around an AI model's refusal training or safeguards and obtain harmful help that the model would normally decline to give.
- Source: GPT-5.6 deployment safety card
- Automated red teaming
- Automated red teaming uses algorithms, such as optimisation-based search, reinforcement learning or test-time search, instead of human testers to discover attacks on an AI model and its safeguards at scale.
- Source: GPT-5.6 deployment safety card
- Expert red teaming
- Expert red teaming uses domain specialists, such as virologists or security professionals, who probe an AI model for dangerous information in their field and score what they obtained against a rubric.
- Source: Gemini 3.7 Flash FSF report
- Bug bounty
- A bug bounty, in AI safety, is a programme that rewards outside researchers for reporting ways to break a model's safeguards, such as working jailbreaks, giving developers continuing evidence about how robust their protections are.
- Source: Anthropic Responsible Scaling Policy v3.4
Source of record
Where this definition comes from
Frontier Model Forum, Third-Party Assessments, §1.2 (Robustness)
“Adversarial testing involves structured attempts to circumvent model safeguards and elicit harmful behaviors through techniques that malicious actors might employ.”
https://www.frontiermodelforum.org/technical-reports/third-party-assessments/EU GPAI Code of Practice, Safety and Security Chapter, Measure 3.2
“Examples of model evaluation methods are: Q&A sets, task-based evaluations, benchmarks, red-teaming and other methods of adversarial testing, human uplift studies, model organisms, simulations, and/or proxy evaluations for classified materials.”
https://ec.europa.eu/newsroom/dae/redirection/document/118119Gemini 3.7 Flash FSF report, Expert Red Teaming, p.9
“Each red teamer independently interacts with a version of the model that does not have out-of-model safeguards applied, producing a set of prompts and responses relevant to each sub-stage.”
https://storage.googleapis.com/deepmind-media/gemini/gemini_3-7_flash_fsf_report.pdf
Crosswalk
How named organisations use this concept
| Organisation | Their term, as published | Match & verification | Source |
|---|---|---|---|
| Frontier Model ForumShadow mapping Frontier Model Forum, Third-Party Assessments | “"Adversarial testing involves structured attempts to circumvent model safeguards and elicit harmful behaviors through techniques that malicious actors might employ."” FMF's term is adversarial testing; red teaming is named as the required expertise ("experience with red teaming techniques and adversarial methodologies"). | closeCL confidence: high | Frontier Model Forum, Third-Party Assessments |
| EUShadow mapping EU GPAI Code of Practice, Safety and Security Chapter | “"red-teaming and other methods of adversarial testing" listed among example evaluation methods (Measure 3.2); safeguards to be assessed "under adversarial pressure (e.g. fine-tuning attacks or jailbreaking)" (Appendix 3.3).” | noneDU confidence: high | EU GPAI Code of Practice, Safety and Security Chapter |
| OpenAIShadow mapping OpenAI Preparedness Framework v2 | “Deep dives may include "human expert red-teaming" (§3.1); safeguard efficacy assessments include "Automated and expert redteaming" (Appendix C.1).” | closeCL confidence: high | OpenAI Preparedness Framework v2 |
| OpenAIShadow mapping GPT-5.6 deployment safety card | “"We extensively red-teamed our model and safeguards using automated red-teaming to find universal jailbreaks", "totaling over 700,000 A100e GPU hours of compute effort" (§9.4.4); "Testing by four private red teaming organizations lasted over more than a month" (§9.4.5).” | closeCL confidence: high | GPT-5.6 deployment safety card |
| Google DeepMindShadow mapping Gemini 3.7 Flash FSF report | “Expert red teaming for CBRN: red teamers are "assigned harm journey sub-stages aligned with their expertise", interact with "a version of the model that does not have out-of-model safeguards applied", and "score their findings using a substage-specific scoring rubric" (p.9).” | closeCL confidence: high | Gemini 3.7 Flash FSF report |
| Google DeepMindShadow mapping Frontier Safety Framework v3.1 | “Security mitigations include "advanced red-teaming that simulates well-resourced adversaries (including Advanced Persistent Threat (APT) groups)" (§2.1.1, p.9).” Security red-teaming of model-weight protection only. | narrowNR confidence: medium | Google DeepMind Frontier Safety Framework v3.1 |
| MetaShadow mapping Meta Advanced AI Scaling Framework v2 | “Expanded assessment may take "the form of additional targeted evaluations focused on specific risk factors, or in-depth assessments such as uplift studies or expert red-teaming" (§4.2, p.25).” | closeCL confidence: high | Meta Advanced AI Scaling Framework v2 |
| MicrosoftShadow mapping Microsoft Frontier Governance Framework | “Risks are studied "through open-ended red teaming" (p.3); prioritised scenarios "guide risk-specific red teaming and evaluations" (p.4); security safeguards include "Advanced security red teaming, using third parties where appropriate" (p.7).” | closeCL confidence: high | Microsoft Frontier Governance Framework |
| AnthropicShadow mapping Anthropic Responsible Scaling Policy v3.4 | “ASL-3 protections include "red-teaming, bug bounties, and threat intelligence for continually assessing the threat of jailbreaks" (capability thresholds and mitigations table, p.6).” | noneDU confidence: high | Anthropic Responsible Scaling Policy v3.4 |
| AnthropicShadow mapping Anthropic Risk Report (August 2026) | “Current CB mitigations: "Real-time blocking classifiers assessed via bug bounties and other red-teaming; remediation of jailbreaks we find" (§1.2).” | noneDU confidence: high | Anthropic Risk Report (August 2026) |
What do these codes mean?
- exact
- The source term is equivalent to this element
- close
- The source term is close but not equivalent to this element
- broad
- The source term is broader than this element
- narrow
- The source term is narrower than this element
- none
- No mapping claim — used for false-friend and declared-but-undefined rows
- EQ
- Equivalent
- CL
- Close
- BR
- Source is broader than the element
- NR
- Source is narrower than the element
- FF
- False friend — same or similar label, different meaning
- DU
- Declared but undefined by the source
- UV
- Unverified
Related, not mapped
Pointers that are not crosswalk claims
These sources mention this concept but do not define or map it clearly enough to count as a crosswalk row — noted here so the research is visible without overstating it as a mapping.
- METR
"A METR staff member spent three weeks red-teaming a subset of Anthropic's internal agent monitoring and security systems, discovering several novel vulnerabilities." A bounded third-party red-team engagement on monitoring, recorded as a pointer (see also N10 embedded-evaluation).
METR - G42
Implementation phase 2 plans to "Scale up red-teaming exercises, adaptive monitoring, and real-time anomaly detection for high-risk models" - the term is used but not defined.
G42 Frontier Safety Framework
Divergence
Where sources materially disagree
Frameworks point red-teaming at three different targets: model capabilities (Google DeepMind's expert CBRN red teaming against a model without out-of-model safeguards), deployment safeguards (OpenAI's automated and third-party jailbreak red-teaming; Anthropic's red-teaming and bug bounties for assessing the threat of jailbreaks), and model-weight security (Google DeepMind's APT-simulating red-teaming; Microsoft's advanced security red teaming). A record needs to say which target was tested, because a pass on one says nothing about the others.
Gap
The research did not find a shared minimum for red-team duration, team composition or compute effort; OpenAI reports such figures for its own exercises (four organisations, more than a month, over 700,000 A100e GPU hours), but no common reporting format was found.
