Examples
Examples from published documents
- Anthropic's policy commits to sharing a Risk Report with external reviewers within one week of submitting it to its Board and Long-Term Benefit Trust, and asking them for public commentary within 30 days of receipt (Anthropic RSP v3.4, §3.6.2).
- OpenAI gave the UK AI Security Institute early access to GPT-5.6 Sol to support independent, pre-deployment evaluation of cyber capabilities (OpenAI GPT-5.6 System Card, s.9.1.2.6).
- Signatories must have adequately qualified independent external evaluators run model evaluations unless the model is 'similarly safe or safer' or no qualified evaluator can be found after a public call (EU GPAI Code of Practice, Appendix 3.5).
Counter-examples
Not to be confused with
- Internal review by a team inside the developer is not external review, even when it is organisationally separate from the training team; the Frontier Model Forum accepts 'sufficiently independent' internal assessors only for open-ended assessment.
- A developer's own summary of an outside evaluator's findings is not the external review itself; the record should point to the evaluator's own report where one exists.
Related terms
Related AI safety terms
Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.
- Third-party assessment
- Third-party assessment is the industry term for an evaluation of a frontier model by an outside party to confirm developer evaluations or safety claims, test them with independent methods, or add specialist expertise the developer lacks.
- Source: FMF Third-Party Assessments
- Confirmatory assessment
- A confirmatory assessment is a third-party review that checks how a developer ran its own evaluations and whether its conclusions are supported, rather than applying new methods. Replication testing, methodology review and holistic review are its usual forms.
- Source: FMF Third-Party Assessments
- Government testing
- Government testing is evaluation of AI systems by a public testing authority, such as a national AI institute, which governments in the United States, United Kingdom, European Union and China rely on as a core part of their approach to AI risk.
- Source: AEF-1: Minimum Operating Conditions for Independent Third Party AI Evaluations (v1, 4 Dec 2025)
Source of record
Where this definition comes from
Anthropic Risk Report, August 2026, §1.3.5
“Version 3.2 of our RSP authorizes Anthropic's Long-Term Benefit Trust (LTBT) to request external review of risk reports, gives the LTBT the power to approve our selection of external reviewers ... every part of any unredacted risk report is reviewed by at least one such reviewer.”
https://www-cdn.anthropic.com/f61d49fa5596956a5dec75fea0e973bf6a6a8378/Redacted%20Risk%20Report%20August%202026%20.pdfAnthropic Responsible Scaling Policy v3.4, §3.6
“We will work toward a practice of seeking comprehensive, public external review on our Risk Reports. This means working with one or more third-party organizations that will receive private versions of our Risk Reports (unredacted or with minimal redactions, as discussed below) and publish comprehensive commentary on them.”
https://cdn.sanity.io/files/4zrzovbb/website/0bacdc8440ea96e62a8766d99ebe1d4eea6d5f3a.pdfEU GPAI Code of Practice, Safety and Security Chapter, Appendix 3.5
“In addition to internal model evaluations, Signatories will ensure that adequately qualified independent external evaluators conduct model evaluations pursuant to this Appendix 3, with regards to the systemic risk, unless: (1) the model is a similarly safe or safer model pursuant to Appendix 2.2; or (2) Signatories fail to appoint adequately qualified independent external evaluators, despite using early search efforts (such as through a public call open for 20 business days) and promptly notifying identified evaluators”
https://ec.europa.eu/newsroom/dae/redirection/document/118119METR, accountability element
“External scrutiny ensures that a company's safety claims can be independently validated by qualified experts.”
https://metr.org/
Crosswalk
How named organisations use this concept
| Organisation | Their term, as published | Match & verification | Source |
|---|---|---|---|
| AnthropicShadow mapping Anthropic Risk Report, August 2026 | “"Version 3.2 of our RSP authorizes Anthropic's Long-Term Benefit Trust (LTBT) to request external review of risk reports, gives the LTBT the power to approve our selection of external reviewers" ... "so long as every part of any unredacted risk report is reviewed by at least one such reviewer." "Since this change to our RSP, the LTBT has not requested an external review (nor has the RSP required that we conduct one), though we have continued to conduct pilot external reviews" (§1.3.5).” Also draws on the Advanced AI Framework's evaluator-review contents (AAF p.8). | exactEQ confidence: high | Anthropic Risk Report, August 2026 |
| OpenAIShadow mapping OpenAI Preparedness Framework v2 / Frontier Governance Framework | “"then when available and feasible, OpenAI will work with third-parties to independently evaluate models" (PF §5.2); "We may solicit and obtain input from external experts" (FGF §5).” Also cites Frontier Governance Framework §5. | closeCL confidence: medium | OpenAI Preparedness Framework v2 |
| Google DeepMindShadow mapping Gemini 3.7 Flash FSF report | “"External safety testing" by "specialist independent groups"; "The independent external evaluators explore frontier safety risk domains, however they do not directly comment on risk thresholds as contemplated under our FSF." (p.6)” | closeCL confidence: medium | Gemini 3.7 Flash FSF report |
| EUShadow mapping EU GPAI Code of Practice, Safety and Security Chapter | “"Signatories will ensure that adequately qualified independent external evaluators conduct model evaluations pursuant to this Appendix 3, with regards to the systemic risk", unless exempt as 'similarly safe or safer' or unable to find qualified evaluators 'despite using early search efforts.' Model Report must explain 'the choice of evaluator based on the qualification criteria' and whether independent evaluator input 'informed' the decision to proceed (Measures 7.2, 7.4).” | exactEQ confidence: high | EU GPAI Code of Practice, Safety and Security Chapter |
| California SB 53Shadow mapping California SB 53 | “Framework topic "(5) Using third parties to assess the potential for catastrophic risks and the effectiveness of mitigations of catastrophic risks"; transparency report summary of "(C) The extent to which third-party evaluators were involved."” | closeCL confidence: medium | California SB 53 |
| US Government (NIST CAISI) / UK AI Security Institute (AISI) partnershipShadow mapping NIST CAISI bulletin; DeepMind UK AISI partnership post | “CAISI: "CAISI will conduct pre-deployment evaluations and targeted research to better assess frontier AI capabilities." UK AISI with DeepMind: "partnered with the UK AISI since its inception in November 2023 to test our most capable models."” Also cites Google DeepMind's post on its partnership with the UK AI Security Institute. | closeCL confidence: medium | NIST CAISI bulletin |
| METRShadow mapping METR (site) | “"External scrutiny ensures that a company's safety claims can be independently validated by qualified experts."” | exactEQ confidence: high | METR |
| Frontier Model ForumShadow mapping FMF Third-Party Assessments technical report | “"Third-party assessment: 'Third-party assessments can be conducted on frontier models to confirm evaluations or claims on critical safety capabilities and mitigations.'" Functions: "Confirmation", "Robustness", "Supplementation".” | exactEQ confidence: high | FMF Third-Party Assessments |
| G42Shadow mapping G42 Frontier Safety Framework | “"G42 will engage in annual external audits to verify compliance with the Framework" (s.5).” | closeCL confidence: medium | G42 Frontier Safety Framework |
| AnthropicShadow mapping Policy on the AI Exponential (essay by Anthropic's CEO) | “"Models above a threshold of compute should undergo mandatory testing by a qualified third party for their level of risk in four specific areas: cybersecurity, biological weapons, loss of control of AI systems, and automated R&D." Delivery left as a choice between "a government agency (similar to the FAA)" or "a set of private organizations that are authorized and inspected by the government ... (a 'regulatory markets' approach)."” Proposes a distinct, two-track mandatory external-review architecture separate from the voluntary/embedded-evaluator model in 'We Must Pace the Frontier'; neither track specifies evaluator access level, publication rights or independence criteria -- a gap the essay itself does not fill. | closeCL confidence: medium | Policy on the AI Exponential (essay by Anthropic's CEO) |
| US Congress (H.R. 9925, FRONTIER Act, not enacted)Shadow mapping FRONTIER Act, 119th Congress | “"A very large frontier developer shall grant an IVO timely access upon request to unredacted materials, records, personnel, systems, and all other information reasonably necessary for conducting the assessments and ongoing monitoring required by this section"; "the IVO shall prepare and submit an assessment report not less frequently than once every 6 months" (Sec. 5). Separately, "a large frontier developer shall retain a third party to perform an independent audit of compliance with the frontier AI framework", annually, with "the signature of the lead auditor certifying the results of the audit" (Sec. 4(c)).” IVO access is 'upon request', not standing employee-like access, and the bill does not set publication rights for assessors beyond the redacted public copy. Not enacted. | closeCL confidence: medium | FRONTIER Act, 119th Congress (H.R. 9925, introduced, not enacted) |
| AnthropicShadow mapping Anthropic Responsible Scaling Policy v3.4 | “"We will work toward a practice of seeking comprehensive, public external review on our Risk Reports"; reviewers "publish comprehensive commentary" on "the quality of our reasoning, the validity of our risk assessments, the overall level of risk, and whether the redactions we've made for the public version are reasonable and appropriate"; a full external review is required "anytime a Risk Report covers highly capable models and is significantly redacted" (§3.6); "We will ask the external reviewers to provide public commentary on our report within 30 days of receipt" (§3.6.2).” Policy text; the Risk Report of August 2026 (§1.3.5) records that the LTBT had not yet requested a review and that reviews were still at pilot stage. | exactEQ confidence: high | Anthropic Responsible Scaling Policy v3.4 |
| Frontier Model ForumShadow mapping FMF Third-Party Assessments technical report | “"Third-party assessments can complement – but do not replace – internal safety processes and corporate governance mechanisms" (1.1); shared characteristics of effective assessments: "independence of judgment, technical rigor, clear methodology, appropriate expertise for the risks being evaluated, and actionable recommendations" (1.3).” Describes member-firm practice 'where appropriate'; no obligation to engage. | closeCL confidence: medium | FMF Third-Party Assessments |
What do these codes mean?
- exact
- The source term is equivalent to this element
- close
- The source term is close but not equivalent to this element
- broad
- The source term is broader than this element
- narrow
- The source term is narrower than this element
- none
- No mapping claim — used for false-friend and declared-but-undefined rows
- EQ
- Equivalent
- CL
- Close
- BR
- Source is broader than the element
- NR
- Source is narrower than the element
- FF
- False friend — same or similar label, different meaning
- DU
- Declared but undefined by the source
- UV
- Unverified
Related, not mapped
Pointers that are not crosswalk claims
These sources mention this concept but do not define or map it clearly enough to count as a crosswalk row — noted here so the research is visible without overstating it as a mapping.
- xAI
CASRAI's reading of the June 2026 framework is that it contains no commitment to external or third-party evaluation. Cards separately state "third-party testing shows that Grok 4's end-to-end offensive cyber capabilities remain below the level of a human professional" and that third-party evaluators "corroborated" Grok 4.6 cyber results -- a pointer, not a mapping.
xAI Frontier AI Framework, 30 Jun 2026 - Meta
External experts used in threat-modeling workshops, risk assessments "where appropriate" and red teaming "when appropriate" -- a pointer, not a mapping.
Meta Advanced AI Scaling Framework v2
Gap
California SB 53 requires a large frontier developer's framework to describe how it uses third parties to assess catastrophic risk, and its transparency report to summarise "the extent to which third-party evaluators were involved", but the statute sets no independence, qualification or access standard for those evaluators. The Frontier Model Forum report describes practice without a disclosure or publication-rights regime. Only the EU Code of Practice and the introduced FRONTIER Act tie external evaluation to stated qualification criteria.







