Skip to main content
v2026.11,986 entries · CC-BY 4.0
CASRAIAsk CASRAI · AI assistant for researchAsk a research question. Answers come with their sources.Funder rules, compliance, methods and publishing. Answers link the sources they rest on.
NIKOLAI elementN6 · Mitigations and securityProposednikolai-v0.3

Safeguard

Definition

A safeguard is a technical or procedural measure that reduces the risk of an AI model being misused or acting harmfully, such as refusal training, input and output classifiers, monitoring, access controls or protection of model weights. Frameworks group safeguards by the risk they target and where they apply.

Also called: Safety mitigation, Safety Measures, Required Safeguards, Preparedness Safeguards

NIKOLAI's proposed operational definition

NIKOLAI proposes to define a Safeguard record as: a technical or procedural measure intended to reduce misuse or misalignment risk, identified by type (e.g. safety training, filter, monitoring, access control), the risk domain it targets, and the deployment scope it applies to. This is an unsourced NIKOLAI editorial synthesis; SB 53 leaves the parent term "mitigations" undefined and METR frames the closest analogue narrowly around access controls, so NIKOLAI's definition is deliberately broader than any single source.

The operational definition is CASRAI's own proposal, not a definition any named organisation has agreed to. See what NIKOLAI is and is not, or browse the AI safety glossary.

Examples

Examples from published documents

  • Google DeepMind: "Deployment Mitigations: are safety measures we implement which are intended to counter the misuse or misaligned expression of critical capabilities in deployments. They may include safety post-training, input/output/chain-of-thought monitoring and analysis, account moderation, jailbreak detection and patching, user verification, and bug bounties." (Google DeepMind Frontier Safety Framework v3.1, glossary)
  • Anthropic's ASL-3 protections "include classifier guards at least as robust as our initial Constitutional Classifiers; access controls for trusted users with exemptions to classifier guards; red-teaming, bug bounties, and threat intelligence for continually assessing the threat of jailbreaks; and a number of noteworthy security controls." (Anthropic RSP v3.4, §1)
  • OpenAI considers "separate safeguards for two of the main ways in which risks can be realized": a malicious user and "a misaligned model, which autonomously causes the harm" (OpenAI Preparedness Framework v2, §4.1)

Counter-examples

Not to be confused with

  • A capability threshold says when stronger safeguards become necessary; it is not itself a safeguard (see capability-threshold).
  • A safety case is the argument that safeguards are adequate, not a safeguard (see safety-case).

Related terms

Related AI safety terms

Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.

Mitigation
A mitigation is any measure taken to reduce a risk. The EU GPAI Code of Practice splits AI mitigations into safety mitigations, which shape what a model does and who can use it, and security mitigations, which protect the model and its infrastructure.
Source: EU GPAI Code of Practice, Safety and Security Chapter
Guardrail
A guardrail is a runtime control placed around a model, such as input and output moderation that blocks or safely modifies harmful requests and responses. Amazon's framework calls these harmful content guardrails a "first and last line of defense".
Source: Amazon's Frontier Model Safety Framework
Usage policy
A usage policy is a developer's published rules on what people may and may not do with its models. Anthropic's RSP pairs it with measures to detect and act on violations quickly, and calls for an internal usage policy binding employees.
Source: Anthropic Responsible Scaling Policy v3.4, effective 8 Jul 2026
Refusal training
Refusal training is fine-tuning a model to decline requests that would help cause severe harm, or to give low-risk answers instead. OpenAI lists it as an example safeguard supporting the claim that malicious users cannot elicit a dangerous capability.
Source: OpenAI Preparedness Framework v2
Model-external safeguard
A model-external safeguard works outside the model's own behaviour, for example by blocking an output or detecting it with offline monitoring so the harm can be addressed later. Anthropic uses the term for this class of mitigation.
Source: Anthropic Risk Report, August 2026

Source of record

Where this definition comes from

Crosswalk

How named organisations use this concept

Every row below is a shadow mapping. A shadow row is CASRAI's own reading of a published document. No lab, evaluator or regulator named on a shadow row has declared, endorsed, or been consulted on it. That changes only when an organisation files its own Mapping Declaration.
OrganisationTheir term, as publishedMatch & verificationSource
AnthropicShadow mapping
Anthropic Risk Report, August 2026
“"Our models are subject to a variety of different safeguards, depending on their capability levels." Factors that vary include robustness, coverage and exemptions (§4.5); footnote 59: "our models and safeguards now vary along many dimensions, so we are no longer using this terminology" (of "ASL-3" and "ASL-2"). "Required Safeguards": "a default set of required safeguards that are expected to bring risk down to acceptable levels"”
The "Required Safeguards" definition is from an earlier Anthropic policy as quoted by the Frontier Model Forum (s3.2), not from the August 2026 Risk Report, which no longer uses ASL-era safeguard terminology (footnote 59).
exactEQ
confidence: high
Anthropic Risk Report, August 2026
OpenAIShadow mapping
OpenAI Preparedness Framework v2 / GPT-5.6 deployment safety
“"Safeguards Report" (§4.2); "Each capability threshold has a corresponding class of risk-specific safeguard guidelines under the Preparedness Framework." (§4.1); the GPT-5.6 card's "Preparedness Safeguards" are those deployed for "the two areas" rated High”exactEQ
confidence: high
OpenAI Preparedness Framework v2
Google DeepMindShadow mapping
Frontier Safety Framework v3.1
“"Deployment Mitigations: are safety measures we implement which are intended to counter the misuse or misaligned expression of critical capabilities in deployments." "Security Mitigations: ... intended to prevent the unauthorized modification or exfiltration of model weights"”exactEQ
confidence: high
Google DeepMind Frontier Safety Framework v3.1
xAIShadow mapping
Frontier AI Framework, 30 Jun 2026
“"Safety training: 'Training our models to recognize and decline harmful requests.'" "System prompts". "Filters: 'Applying classifiers to verify safety when a model is queried regarding topics of CRBN [sic] risks'"”
xAI's Frontier AI Framework (30 Jun 2026) carries PDF metadata reading "Privileged/Confidential DRAFT working FRAMEWORK DOC" with no xAI statement found disambiguating draft from final; treat this citation as provisional.
closeCL
confidence: medium
xAI Frontier AI Framework, 30 June 2026 (draft-labeled PDF metadata)
MetaShadow mapping
Meta Advanced AI Scaling Framework v2
“Table 1 measures ("Deploy with mitigations"); LlamaFirewall; Llama Defender program”closeCL
confidence: medium
Meta Advanced AI Scaling Framework v2
EUShadow mapping
EU GPAI Code of Practice, Safety and Security Chapter
“Commitment 5 "Safety mitigations" (training-data filtering, input/output monitoring and filtering, refusal fine-tuning, staged/gated access, "high-assurance quantitative safety guarantees", CoT-transparency techniques) and Commitment 6 "Security mitigations"; a model is "exempt from [Commitment 6] if the model's capabilities are inferior to the capabilities of at least one model for which the parameters are publicly available for download."”exactEQ
confidence: high
EU GPAI Code of Practice, Safety and Security Chapter
California SB 53Shadow mapping
California SB 53
“"Mitigations" undefined (framework topic (3))”
SB 53 requires labs to describe mitigations but never defines the term itself — the gap is the finding, recorded rather than dropped.
noneDU
confidence: low
California SB 53
METRShadow mapping
METR (metr.org)
“"Model Deployment Mitigations: Access and model-level measures applied to prevent the unauthorized use of a model's dangerous capabilities."”exactEQ
confidence: high
METR
What do these codes mean?
exact
The source term is equivalent to this element
close
The source term is close but not equivalent to this element
broad
The source term is broader than this element
narrow
The source term is narrower than this element
none
No mapping claim — used for false-friend and declared-but-undefined rows
EQ
Equivalent
CL
Close
BR
Source is broader than the element
NR
Source is narrower than the element
FF
False friend — same or similar label, different meaning
DU
Declared but undefined by the source
UV
Unverified

Related, not mapped

Pointers that are not crosswalk claims

These sources mention this concept but do not define or map it clearly enough to count as a crosswalk row — noted here so the research is visible without overstating it as a mapping.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →