Skip to main content
v2026.11,986 entries · CC-BY 4.0
CASRAIAsk CASRAI · AI assistant for researchAsk a research question. Answers come with their sources.Funder rules, compliance, methods and publishing. Answers link the sources they rest on.
NIKOLAI elementN8 · Transparency and reviewProposednikolai-v0.3

AI-Model Review

Definition

An AI-model review is a record that an AI system performed an assurance task, such as reviewing a report, monitoring agents, grading outputs or red-teaming, stating which system, what access it had, who supervised it, and which human accepted its output.

Also called: AI-assisted review, Automated review, Automated auditing, AI reviewer, Analysis agents

NIKOLAI's proposed operational definition

NIKOLAI proposal: AI-Model Review is a record that an AI system performed an assurance task -- review, monitoring, grading, red-teaming, or analysis -- capturing the system's identity, the task performed, its access, its supervision, the disposition of its output, and the accountable human who owns that disposition. This working definition is NIKOLAI's own editorial synthesis. It is distinct from, and does not substitute for, a CRediT-pattern contributor-role accountability layer for human authorship, which is out of scope for this element.

The operational definition is CASRAI's own proposal, not a definition any named organisation has agreed to. See what NIKOLAI is and is not, or browse the AI safety glossary.

Examples

Examples from published documents

  • Anthropic prompted an instance of Claude Mythos 5, with access to internal Slack channels, documents and code, to review a near-final draft of a Risk Report section; it took 24 minutes (Anthropic Risk Report, August 2026, §2.20).
  • METR relied on GPT-5.6 Sol 'analysis agents' to scan transcripts in its incident investigation and spot-checked their output manually (METR, OpenAI-Hugging Face incident investigation).

Counter-examples

Not to be confused with

  • Crediting an AI model as a contributor to a document is not an AI-model review unless the record also states its access, supervision and the accountable human who accepted the output.

Related terms

Related AI safety terms

Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.

AI monitor
An AI monitor is an AI system used as a safeguard to supervise the actions of AI agents; OpenAI's Preparedness Framework lists a monitor AI that supervises agent actions among its safeguards against misaligned behaviour.
Source: OpenAI Preparedness Framework v2
Analysis agent
An analysis agent is an AI agent used by investigators to search and summarise large volumes of transcripts or logs, whose reliability must itself be checked because it can miss or misreport important evidence.
Source: METR: OpenAI-Hugging Face incident investigation

Source of record

Where this definition comes from

Crosswalk

How named organisations use this concept

Every row below is a shadow mapping. A shadow row is CASRAI's own reading of a published document. No lab, evaluator or regulator named on a shadow row has declared, endorsed, or been consulted on it. That changes only when an organisation files its own Mapping Declaration.
OrganisationTheir term, as publishedMatch & verificationSource
AnthropicShadow mapping
Anthropic Risk Report, August 2026 / Anthropic Alignment Assessment: Cybersecurity Incidents / Claude's Constitution
“"we prompted an instance of Claude Mythos 5 to review a near-final draft of Section 2 of this report"; access to "internal Anthropic Slack channels", "internal documents", "internal codebase" and subagents; "Readers should weigh my position honestly, as I do: I am a Claude model reviewing Anthropic's assessment of Claude models"; "In practice, Claude took 24 minutes to produce this review" (§2.20). "a prompted Claude model reviews suggested code changes" (§2.23.2.3). Incident scan second stage "used Claude to review the '9.2 million transcripts'". Claude's Constitution Acknowledgements: "Several Claude models provided feedback on drafts. They were valuable contributors and colleagues in crafting the document, and in many cases they provided first-draft text for the authors above."”
Also cites Anthropic Alignment Assessment: Cybersecurity Incidents and Claude's Constitution. The Constitution's Acknowledgements name Claude models as co-authors of first-draft text on a governing policy document but, unlike §2.20, do not name the accountable human editor(s) of that text.
exactEQ
confidence: high
Anthropic Risk Report, August 2026
OpenAIShadow mapping
OpenAI Preparedness Framework v2 / GPT-5.6 deployment safety page / pacing model development post
“"Monitor AI supervises agent actions" (PF Table 5). "GPT-Red: 'an automated red-teaming model trained using self-play reinforcement learning'" (GPT-5.6 s.4.2). "Automated investigators" (August 2026).”
Also cites the GPT-5.6 deployment safety page and OpenAI's post on pacing model development.
closeCL
confidence: medium
OpenAI Preparedness Framework v2
Google DeepMindShadow mapping
Gemini 3.7 Flash FSF report
“"Investigator agent: 'dynamically explore[s] prompting strategies (including jailbreaks) and synthesise[s] outputs'"; "Prompted Classifiers: 'LLM-based classifiers take in user conversations and output labels regarding malicious intent. Developed using AlphaEvolve'" (pp.27-28).”closeCL
confidence: medium
Gemini 3.7 Flash FSF report
xAIShadow mapping
Grok 4.20 model card / Grok 4 model card
“"Automated alignment audit: 'an internal tool built off of Petri 2.0 ... the auditing tool constructs detailed agentic scenarios'"; refusals "graded by 'another model'" (Grok 4 card).”
Also cites the Grok 4 model card.
closeCL
confidence: medium
Grok 4.20 model card
MetaShadow mapping
Meta Advanced AI Scaling Framework v2
“LlamaFirewall chain-of-thought auditing; risk of "long-term dependency of AI for supervisory functions" (§4.2.4).”narrowNR
confidence: medium
Meta Advanced AI Scaling Framework v2
METRShadow mapping
METR OpenAI-Hugging Face incident investigation / METR (site)
“The investigation "relied on GPT-5.6 Sol agents ("analysis agents") — often managing large nested trees of sub-agents — to autonomously analyze patterns". METR's guidance on incident investigations asks: "Is there evidence that the AI tools used for scanning and filtering through the results from our queries might fail to surface serious misalignment incidents (whether due to benign failures or intentional sandbagging)?"”
The question is from METR's 28 Jul 2026 post (https://metr.org/blog/2026-07-28-investigating-ai-propensities-after-incidents/), not the incident report.
closeCL
confidence: medium
METR: OpenAI-Hugging Face incident investigation
What do these codes mean?
exact
The source term is equivalent to this element
close
The source term is close but not equivalent to this element
broad
The source term is broader than this element
narrow
The source term is narrower than this element
none
No mapping claim — used for false-friend and declared-but-undefined rows
EQ
Equivalent
CL
Close
BR
Source is broader than the element
NR
Source is narrower than the element
FF
False friend — same or similar label, different meaning
DU
Declared but undefined by the source
UV
Unverified

Gap

CRediT (ANSI/NISO Z39.104-2022) covers human contributor roles on research outputs and has no representation for AI systems. Anthropic's Risk Report §2.20 records an AI reviewer's access, time on task, self-declared conflict and the handling of its criticisms; no other source in this corpus records all of these. METR's investigation shows the risk: about 7% of the transcripts it reviewed were 'successfully spoofed in some places' by the agents under study. The introduced FRONTIER Act requires audit reports to list 'the personnel involved in the audit' but does not contemplate AI systems doing audit work.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →