Frontier AI Safety & Governance
A guide to frontier AI safety frameworks, regulation, third-party evaluation, and incident governance, for the compliance, policy, and governance professionals evaluating or adopting them.
Ask CASRAI · free to try
Ask about Frontier AI Safety & Governance
Ask your first 2 questions free below. Subscribers get 150 a day for $29 a month.
Ask CASRAI answers research-administration questions and cites the passages behind every claim. When our sources don't cover a question, it says so.
Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.
Works on this site and inside Claude, Cursor and the AI tools you already use.
Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays free to read, with no account and no card.
Frontier AI developers – Anthropic, OpenAI, Google DeepMind, xAI, Meta – each publish their own safety framework: a document setting out how they test models for dangerous capabilities before release, what thresholds trigger new safeguards, and what commitments they’ve made to evaluate, disclose, and respond to risk. Governments are moving in parallel: California’s SB 53, New York’s RAISE Act, and the EU’s AI Act and GPAI Code of Practice each impose their own transparency and reporting obligations on the same class of models. A growing evaluation ecosystem – METR, the UK AI Security Institute, the US Center for AI Standards and Innovation (CAISI) – sits between the two, testing frontier models independently and publishing what it finds.
This is CASRAI’s guide to that landscape: what each framework actually requires, how they compare to one another, who evaluates against them, and what a compliance, policy, or governance professional needs to know to work with any of them. It complements NIKOLAI, CASRAI’s frontier-AI-safety dictionary of elements, which crosswalks the vocabulary these frameworks use, element by element, against the real primary documents that define it – every reading labelled honestly as CASRAI’s own, never as an endorsement by the organisation it describes.
Safety framework fundamentals
Start here if you’re new to the space: what a Responsible Scaling Policy, Preparedness Framework, or Frontier Safety Framework actually is, the vocabulary they share (capability thresholds, safety cases, dangerous-capability evaluations), and how the major labs’ approaches compare side by side.
Regulation & standards
The binding and voluntary rules layered on top of labs’ own frameworks: California SB 53, New York’s RAISE Act, the EU AI Act and its GPAI Code of Practice, and the management-system standards (NIST AI RMF, ISO/IEC 42001) an organisation can build a compliance program – or certify – against.
Third-party evaluation & assurance
The independent evaluators testing frontier models from outside the labs that build them: METR, the UK AI Security Institute, the US CAISI, and the emerging standards (evaluator independence, embedded vs. arms-length access) that govern how that testing actually works.
Incident reporting & governance
What happens when something goes wrong: statutory incident-reporting deadlines under SB 53 and the RAISE Act, whistleblower protections for AI safety staff, and the internal governance – accountable decision-makers, board sign-off – frontier developers are building to meet these obligations.
Implementation & adoption
For the reader ready to build something: framework templates, AI risk assessment and risk registers, compliance checklists, and practical guidance for standing up an internal AI safety program – the closest content in this hub to what NIKOLAI itself is for.
Guides in this cluster
GPAI Systemic Risk: The EU AI Act Term Explained
What systemic risk means under the EU AI Act, the Article 51 classification test and 10^25 FLOPs presumption, and the Article 55 obligations it triggers once a model is classified.
The SB 53 Material-Change Trigger: When a Frontier AI Framework Must Be Updated
SB 53 requires a large frontier developer to publish its modified Frontier AI Framework, with a justification, within 30 days of a material modification. This guide covers what counts as material and the process for complying.
NIST AI RMF Generative AI Profile: What AI 600-1 Adds
NIST AI 600-1, the Generative AI Profile, is a separate July 2024 publication that extends the base AI RMF (NIST AI 100-1) for generative AI specifically — naming twelve risks unique to or exacerbated by GAI (confabulation, CBRN information or capabilities, data privacy, and more) and suggesting actions tagged to the RMF’s Govern/Map/Measure/Manage subcategories.
UK AI Safety Institute vs AI Security Institute: The 2025 Rename Explained
The UK government renamed its AI Safety Institute to the AI Security Institute on 14 February 2025. It is the same organisation under a new name, not two separate bodies — this guide explains what changed, why, and why the old name still turns up.
Building an Internal Audit Function for Frontier AI Safety
An internal AI-safety audit function checks, on a schedule, whether safety commitments are actually being met — distinct from incident response, which reacts when something goes wrong, and from third-party evaluation, which supplies outside credibility.
EU AI Act High-Risk System Compliance Checklist
A practical checklist against the EU AI Act high-risk system deadlines as amended by the 2026 AI Omnibus: Annex III (2 December 2027), Annex I (2 August 2028), and what providers and deployers have to complete before each.
NIST’s AI Agent Standards Initiative: What It Covers and Its Current Status
NIST’s Center for AI Standards and Innovation launched the AI Agent Standards Initiative in February 2026 to cover AI agent security, identity, and authorization — separate from the AI RMF. Here is what it actually covers and its current status.
Third-Party AI Evaluator Standards: Independence, Access, and Methodology
A cross-cutting guide to the standards questions that apply to every third-party AI evaluator: embedded vs. arms-length access, independence and conflict-of-interest, evaluation validity, red-team access agreements, publication rights, and retaliation protection — mapped to NIKOLAI’s N8 Transparency and Review track.
Building an AI Safety Framework with NIST’s Govern, Map, Measure, Manage Functions
A walkthrough of NIST AI RMF’s Govern, Map, Measure, and Manage functions as a methodology for building an AI safety framework from scratch, with a practical build sequence and verified subcategory detail.
Accountable Decision-Makers Under SB 53: What the Statute Actually Requires
SB 53 requires internal governance practices around deployment decisions, but the statute itself never names a required accountable-decision-maker role. Here is what it explicitly requires, what is implementation practice, and how NIKOLAIs N9 element proposes to fill the gap.
AI Safety vs. AI Security: What the Distinction Actually Means
A short definitional guide distinguishing AI safety (preventing an AI system from causing unintended harm through its own behavior or capabilities) from AI security (protecting AI systems from external attack or misuse, such as model weight theft, adversarial attacks, and data poisoning), grounded in how frontier labs and oversight institutions actually use the terms.
AI Red Teaming: Definition and How It Works in Frontier AI Safety
AI red teaming inside a frontier safety framework is a safeguard-testing commitment, not the AppSec/jailbreak service most search results describe — here is what it actually tests, and how it differs from a dangerous-capability evaluation.
Mapping Your AI Safety Program to the EU AI Act’s GPAI Code of Practice
Where an existing risk register, governance framework, and incident response program already satisfy the GPAI Code of Practice, and where they need extending.
The EU AI Act GPAI Code of Practice: What It Is and Who Signed It
What the EU AI Act’s GPAI Code of Practice actually requires — its three chapters, its presumption-of-conformity mechanism under Article 53(4), and which major AI labs have signed it.
Seoul Frontier AI Safety Commitments: The Signatory List and What They Pledged
The 20 companies that have signed the Seoul Frontier AI Safety Commitments, the full list including the February 2025 additions, and how the pledge relates to SB 53’s binding Frontier AI Framework requirement.
International AI Safety Report 2026: What It Found
What the Bengio-chaired International AI Safety Report 2026 found on frontier AI capabilities, malicious-use and control risks, and industry safety frameworks.
How to Grade a Frontier AI Safety Framework: The SaferAI Rubric
SaferAI, an independent nonprofit, scores frontier AI labs’ published safety frameworks against a four-dimension rubric. Here’s how the rubric works and how Anthropic, OpenAI, and Google DeepMind currently score.
UK AI Security Institute’s Frontier AI Trends Report: What Its Evaluations Have Found
AISI’s first Frontier AI Trends Report aggregates two years of evaluations across 30+ frontier models. Here is what it found on cyber, bio/chem, autonomy, and safeguards.
What Is a Frontier AI Framework? The SB 53 and RAISE Act Requirement, Explained
A Frontier AI Framework is a specific published document that SB 53 and the RAISE Act require large AI developers to maintain. Here is what each law requires, how the terms lined up in March 2026, and how it relates to a lab’s own voluntary safety policy.
AI Governance Framework Template: Councils, Risk Tiers, and Escalation Paths
The organizational layer an AI governance framework needs above a risk register: a governance council with defined authority, a risk-tiering methodology, escalation paths, and review cadence.
Building an Internal AI Safety Incident Response Program
What an internal AI safety incident response program needs structurally: detection channels, triage against a severity taxonomy, escalation to a named accountable decision-maker, and the SB 53 and RAISE Act external reporting clocks.
What Is AI Alignment? Definition and Why It Matters for Frontier AI Safety
AI alignment is the technical problem of making a model behavior match its developers intended goals and values. This guide defines alignment, distinguishes it from AI safety and AI security, and shows how frontier labs evaluate it in practice.
New York RAISE Act: What It Requires
New York’s RAISE Act requires large frontier AI developers to publish a Frontier AI Framework and report Critical Safety Incidents to DFS within 72 hours — a much tighter window than California SB 53’s 15 days.
Responsible AI Usage Policy Template: Acceptable-Use and Prohibited-Use Clauses
A practical guide to writing an internal AI acceptable-use policy: what acceptable-use and prohibited-use clauses typically cover, how it differs from a governance framework, and where it fits with SB 53 and EU AI Act obligations.
SB 53 Critical Safety Incident Reporting: What Counts, Deadlines, and Who to Notify
The statutory definition of a reportable incident under California SB 53, the 15-day versus 24-hour reporting clocks, who must be notified, what a compliant report must contain, and how the Attorney General enforces it.
CAISI: NIST’s Center for AI Standards and Innovation, Explained
CAISI is NIST’s Center for AI Standards and Innovation, renamed from the AI Safety Institute in June 2025. Its origin, its role inside NIST, its standards and evaluation mandate, and how it relates to the UK AI Security Institute.
AI Risk Assessment Framework and Risk Register: A Practical Starting Point
What a risk register actually contains (description, likelihood/impact, owner, mitigation, review cadence), how it maps to NIST’s AI RMF Govern-Map-Measure-Manage functions, and a practical starting template.
ISO/IEC 42001 Certification: Path, Timeline, and Annex A Controls
A practical guide to ISO/IEC 42001 certification: what it actually certifies (a management system, not a model), the real path from gap assessment through Stage 1 and Stage 2 audits, a realistic timeline, and what the nine Annex A control areas cover.
California SB 53 (Transparency in Frontier Artificial Intelligence Act): The Foundational Explainer
A plain-language breakdown of what California SB 53 actually requires: who counts as a frontier developer, the frontier AI framework and transparency report obligations, the 15-day critical safety incident reporting duty, whistleblower protections, and civil penalties.
What Is METR? How Its AI Safety Evaluations Work
An institutional profile of METR (Model Evaluation and Threat Research): what it is, what it evaluates, how its evaluations work, and its relationship to the AI labs it assesses.
Responsible Scaling Policy (RSP): What It Is and How the Major Labs Compare
What a Responsible Scaling Policy is, how capability thresholds and safeguard tiers work, and how Anthropic’s RSP compares to OpenAI’s Preparedness Framework and Google DeepMind’s Frontier Safety Framework.
Third-Party AI Auditing: What It Is and Who Does It
Third-party AI auditing means an independent party assessing an AI system, or the organization running it, against a compliance framework like ISO/IEC 42001 or the EU AI Act — not the same as using AI tools to automate financial audits.
CAISI and the UK AI Security Institute: How Pre-Deployment Testing Agreements Work
CAISI and the UK AI Security Institute both renamed themselves in 2025 and both run voluntary pre-deployment testing agreements with frontier AI labs. Here’s what those agreements actually cover, and what “early access” means in practice.
What Is a Frontier AI Model?
“Frontier AI model” is a capability class, not a specific product or lab. This guide walks through the three definitional approaches in active use — compute thresholds, capability thresholds, and relative state-of-the-art — and why the definition a framework uses is usually what triggers its obligations.
