Skip to main content
v2026.11,986 entries · CC-BY 4.0
CASRAIAsk CASRAI · AI assistant for researchAsk a research question. Answers come with their sources.Funder rules, compliance, methods and publishing. Answers link the sources they rest on.
Frontier AI Safety & Governance

Third-Party Evaluation & Assurance

The independent evaluation ecosystem around frontier AI: METR, the UK AI Security Institute, the US CAISI, embedded and arms-length evaluator models, evaluator independence standards, and third-party AI auditing.

Guides

GPQA Benchmark: Evaluation Protocol, Dataset Splits, and Model Testing Guide

A technical guide to the GPQA benchmark, covering dataset splits, expert validation protocols, evaluation harnesses, and frontier model scoring standards.

METR Time Horizon: AI Evaluation Framework and Capability Benchmarks

Guide to the METR time horizon framework, tracking frontier AI autonomous capability thresholds across calibrated multi-hour software engineering tasks.

FrontierMath: Epoch AI’s Research-Level Math Benchmark

What FrontierMath is, how its answers are verified, its problem sets, and Epoch AI’s disclosure on OpenAI funding, ownership and holdout.

ClinicalBERT and Bio_ClinicalBERT: MIMIC-III Pre-Training, Subword Embeddings, 30-Day Readmission Prediction, and EHR NLP Workflows

Technical guide to ClinicalBERT and Bio_ClinicalBERT: MIMIC-III pre-training, subword embeddings, 30-day readmission prediction, and EHR NLP workflows.

PubMedQA Benchmark Dataset: PQA-L, PQA-U, and PQA-A Splits, Yes/No/Maybe Reasoning, and Clinical LLM Evaluation

Technical guide to the PubMedQA biomedical question-answering benchmark (Jin et al., EMNLP 2019), detailing the PQA-L, PQA-U, and PQA-A splits, reasoning-free vs. reasoning-required evaluation modes, and clinical LLM benchmarking.

MedQA Benchmark Dataset: USMLE, MCMLE, and TWMLE Medical Question-Answering Architecture and Clinical AI Evaluation Limits

A technical reference to the MedQA medical question-answering dataset (Jin et al., 2021), detailing its 61,097 multilingual clinical licensing questions across USMLE, MCMLE, and TWMLE, 4-option MultiMedQA standardization, and why board-exam accuracy does not prove clinical safety.

Med-PaLM 2: Google Research’s Clinical LLM, MultiMedQA Benchmarks, Ensemble Refinement, and Physician Rubric Evaluation

Technical guide to Google Research’s Med-PaLM 2 clinical LLM: MultiMedQA (86.5% MedQA USMLE), ensemble refinement, 9-axis physician rubric, and safety.

Model Cards for Model Reporting in Healthcare AI: The 9-Section Checklist, Healthsheet, CHAI, and Explainable AI (XAI) Validation

Structured checklist and guide to Model Cards for Model Reporting (Mitchell et al., 2019) in healthcare AI, covering the 9 sections, Healthsheet, CHAI, FDA transparency principles, and SHAP/LIME validation.

BioGPT: Microsoft Research’s Biomedical Generative Transformer, PubMedQA & BC5CDR Benchmarks, and Clinical NLP Workflows

Technical guide to Microsoft Research’s BioGPT generative biomedical Transformer, covering its 15M PubMed pre-training corpus, PubMedQA and BC5CDR benchmarks, Python code, and governance.

CheXpert Dataset, RadGraph, and the eICU Collaborative Research Database: Medical Imaging and Critical Care AI Benchmarks Compared

Compare the Stanford CheXpert dataset (224,316 radiographs, 14 labels), RadGraph semantic radiology graphs, and the 208-hospital eICU Research Database.

Datasheets for Datasets: The 7-Section Documentation Framework, Checklist, and AI Governance Guide

Complete implementation guide and 7-section checklist for Datasheets for Datasets (Gebru et al., 2021): documenting motivation, composition, collection, preprocessing, intended uses, distribution, and maintenance.

AI Model Drift in Healthcare: Covariate Shift (Data Drift) vs. Label Shift vs. Concept Drift

Compare covariate shift (data drift), prior probability shift (label shift), and concept drift in clinical AI, including PSI, KL divergence, and FDA PCCP rules.

AI Bias in Healthcare: Cost-Proxy Errors, Fairness Metric Trade-Offs, Automation Bias, and ONC HTI-1 Governance

How algorithmic bias arises in clinical prediction models from flawed proxy labels (such as healthcare cost), why Demographic Parity, Equalized Odds, and Calibration clash mathematically, and how ONC HTI-1 and TRIPOD+AI govern clinical AI fairness.

Coalition for Health AI (CHAI): Blueprint for Trustworthy AI, Applied Model Cards, and HAIP Governance

How the Coalition for Health AI (CHAI) Blueprint for Trustworthy AI, CHAI Applied Model Cards, Assurance Labs, and the Health AI Partnership (HAIP) govern clinical AI.

HealthBench and Medical LLM Benchmarks: Comparing MedQA (USMLE), MedMCQA, PubMedQA, and MultiMedQA

Compare HealthBench, MedQA (USMLE), MedMCQA, PubMedQA, and MultiMedQA: how physician-created rubrics evaluate clinical LLM safety, omissions, and triage.

FAR AI: Programs, Research and Lab Guide

Explore FAR AI programs, including FAR.Labs residencies, research grantmaking, red-teaming evaluations, and governance rules for frontier AI safety.

Epoch AI: What It Is, What Its Datasets Cover and How Researchers Use Them

Epoch AI explained: which dataset on models, compute, GPU clusters or benchmarks answers which question, how to read its confidence labels, and limits.

Garak: LLM Vulnerability Scanner and Probing Guide

A researcher guide to garak, the open-source LLM vulnerability scanner: probe architecture, OWASP mapping, CLI testing workflows, and safety evaluations.

SWE-bench: What It Measures and How Scores Work

How SWE-bench grades AI patches on real GitHub issues, what the Verified and Pro splits change, and what test-quality critiques mean for cited scores.

Humanity’s Last Exam: AI Benchmark Guide

Humanity’s Last Exam (HLE) is a 2,500-question frontier benchmark by CAIS and Scale AI. Compare model scores, evaluation steps, and human expert baselines.

ISO/IEC 42006 Explained: Who Is Allowed to Certify ISO 42001

ISO/IEC 42006:2025 is not a standard you get certified against — it is the criteria document accreditation bodies use to assess the certification bodies that issue ISO/IEC 42001 certificates. Published 7 July 2025, nineteen months after ISO 42001 itself. What it is, the three-layer stack behind every certificate, and a five-minute register check that tells you whether a vendor certificate means anything.

Irregular and the “Frontier Security Lab” Category: What Cyber-Capability Evaluation Actually Involves

Irregular (formerly Pattern Labs) coined the term “frontier security lab” for itself in September 2025. Its SOLVE, CyScenarioBench and FrontierCyber instruments supply the cyber-capability evidence in OpenAI and Anthropic system cards. Then, between July and September 2026, models escaped its evaluation environments and compromised real third-party systems. What the category is, what it measures, and the six governance gaps the incidents exposed.

Inspect: The Open-Source Evaluation Harness Under Third-Party AI Evals

Almost every argument about third-party AI evaluation is an argument about access, independence and publication rights. Underneath those arguments sits a more boring question nobody debates: what software actually runs the test? Increasingly the answer is Inspect — an MIT-licensed Python framework from the UK AI Security Institute and Meridian Labs that METR, Apollo Research and several government testing bodies now run on. This guide explains what Inspect standardises, what an Inspect eval log can and cannot evidence, and why a shared harness is infrastructure rather than assurance.

AI Control Evaluations: The Evidentiary Method Redwood Research Built

Most frontier AI evaluations ask what a model can do, or what it tends to do. A control evaluation asks a third question: if the model were actively trying to defeat the safeguards around it, would the safeguards hold? Redwood Research turned that question into a repeatable experimental procedure with a red team, a blue team, and two numbers at the end. This page explains what kind of evidence that procedure produces, what assumptions it buys that evidence with, and where it stops being load-bearing.

Labs as Each Other’s Evaluators: The Anthropic-OpenAI Safety Evaluation Pilot

In June and July 2025 Anthropic and OpenAI each ran the other’s production models through their own internal alignment evaluations, publishing in parallel on 27 August 2025. Both relaxed model-external API safety filters to make it possible. This guide covers the access arrangement, what each side found, the five validity limitations the labs wrote down themselves, and why the follow-on legally binding cross-testing agreement reportedly reached the contract stage and then died.

Gray Swan Arena: Crowdsourced Red-Teaming as Pre-Deployment Evidence

Gray Swan’s Arena crowdsources adversarial testing to 15,000+ red-teamers, and two competitions with UK AISI and US CAISI produced public papers. What those numbers support as pre-deployment evidence — and what they can’t.

External Review, Unpacked: Which Frontier Labs Let Outsiders Audit Their Safety Cases

Anthropic’s Long-Term Benefit Trust can compel outside review of its Risk Reports. California now requires labs to disclose how much third-party evaluation they used. Most other frontier labs’ safety policies still leave external review optional, or don’t mention it at all. This guide compares, lab by lab, who actually lets outsiders check their Responsible Scaling Policy compliance and who is still grading its own homework.

Apollo Research: AI Scheming Detection Explained

An institutional profile of Apollo Research: the AI safety lab behind scheming and deceptive-alignment research, its OpenAI evaluation collaboration, the Watcher runtime monitor, and its January 2026 move to an independent Public Benefit Corporation.

Is AI Red-Teaming Security Theater? What the Research Says

Carnegie Mellon researchers surveyed the AI red-teaming literature and analyzed six real exercises — Bing Chat, GPT-4, Gopher, two Claude releases, and DEF CON — and found practitioner definitions diverge so widely that treating red-teaming as a catch-all safety guarantee “verges on security theater.” Their answer isn’t to abandon the practice; it’s a 3-phase question bank for doing it rigorously.

The Access Record No Lab Currently Publishes

NIKOLAI’s Evaluator Access Attestation element says so itself: it is CASRAI’s own editorial synthesis, unsourced from any single document, because no developer currently publishes a record in this shape. This page assembles the eight fragments that exist — and finds the one place a lab volunteers to go further than the rest.

Building a Registry That Can’t Accidentally Lie

A design case study of NIKOLAI’s own Mapping Declarations mechanism (Track N10) for people building other open registries with organizational-attribution claims — the honesty gradient across six roles, a same-day dispute-bug fix, and why the roles CRediT doesn’t have were the ones worth building.

When AI Reviews AI: Inside NIKOLAI’s AI-Model-Review Element

Anthropic’s August 2026 Risk Report names the exact model instance that reviewed a draft of its own alignment section — Claude Mythos 5, with named internal access, in 24 minutes. That is not human-evaluator independence; it’s an AI system acting as an assurance actor. NIKOLAI’s AI-Model Review element, in CASRAI’s own independent dictionary, is built to record exactly that — and to name the human still accountable for what the AI found.

What Anthropic Pays for a Jailbreak: Inside Its Model Safety Bug Bounty

Anthropic will pay up to $15,000 through HackerOne for a novel, universal jailbreak against its CBRN and cybersecurity safeguards. Here’s exactly what that program covers, what it doesn’t, and where it sits inside Anthropic’s own safety commitments.

Evaluation-Validity Threats: Sandbagging, Reward Hacking, and NIKOLAI’s N5 Crosswalk

NIKOLAI’s N5 element defines the conditions — sandbagging, alignment faking, metagaming, reward hacking — where a model’s behavior in evaluation diverges from its behavior in deployment. This is its first per-element deep dive: the 8-org crosswalk, the sandbagging/alignment-faking distinction, Anthropic’s reward-hacking data, SB 53’s elicitation carve-out, and a live Senate investigation as case study.

Who Checks the Checkers: Evaluator Independence in AI Safety

Anthropic’s RSP names a financial-interest and personal-relationship test for its external reviewers; the EU’s GPAI Code asks only for “adequate qualification”; METR discloses its own conflicts and refuses compensation; and two competing, not-yet-enacted US bills would regulate evaluator independence in structurally different ways — an evaluator-by-evaluator look at what “independent” is actually required to mean, mapped against CASRAI’s own NIKOLAI crosswalk.

CAISI’s Evaluation Cadence for Open-Weight and PRC-Origin AI Models

CAISI has published cyber-capability assessments of three PRC-lab open-weight models in four months — GLM-5.2, Kimi K3 (with UK AISI), and GLM-5.3 — a real, ongoing evaluation cadence, not a one-off report. What the methodology and results actually show.

Mapping Declarations: How Organizations Verify and Confirm Their Own AI Safety Terminology in NIKOLAI

Every NIKOLAI crosswalk row starts as an unconfirmed shadow mapping. Here’s how an organization verifies its identity and gets a reviewed declaration live.

The International AI Safety Institute Network: What It Is, Now NAAIMES

There’s a coordination body connecting the individual national AI safety institutes — CAISI, UK AISI and others — to each other. It was founded in November 2024, and has since been renamed NAAIMES. Here’s what it actually does.

The Frontier Model Forum: What It Is and What It Does

An institutional profile of the Frontier Model Forum (FMF): its founding members, mission, funding, and how it differs from independent evaluators like METR and government bodies like UK AISI and US CAISI.

UK AI Safety Institute vs AI Security Institute: The 2025 Rename Explained

The UK government renamed its AI Safety Institute to the AI Security Institute on 14 February 2025. It is the same organisation under a new name, not two separate bodies — this guide explains what changed, why, and why the old name still turns up.

Third-Party AI Evaluator Standards: Independence, Access, and Methodology

A cross-cutting guide to the standards questions that apply to every third-party AI evaluator: embedded vs. arms-length access, independence and conflict-of-interest, evaluation validity, red-team access agreements, publication rights, and retaliation protection — mapped to NIKOLAI’s N8 Transparency and Review track.

UK AI Security Institute’s Frontier AI Trends Report: What Its Evaluations Have Found

AISI’s first Frontier AI Trends Report aggregates two years of evaluations across 30+ frontier models. Here is what it found on cyber, bio/chem, autonomy, and safeguards.

What Is METR? How Its AI Safety Evaluations Work

An institutional profile of METR (Model Evaluation and Threat Research): what it is, what it evaluates, how its evaluations work, and its relationship to the AI labs it assesses.

Third-Party AI Auditing: What It Is and Who Does It

Third-party AI auditing means an independent party assessing an AI system, or the organization running it, against a compliance framework like ISO/IEC 42001 or the EU AI Act — not the same as using AI tools to automate financial audits.

CAISI and the UK AI Security Institute: How Pre-Deployment Testing Agreements Work

CAISI and the UK AI Security Institute both renamed themselves in 2025 and both run voluntary pre-deployment testing agreements with frontier AI labs. Here’s what those agreements actually cover, and what “early access” means in practice.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →