Examples
Examples from published documents
- The Model Report must include any available external evaluator reports, letting evaluators keep control of publication without implied endorsement by the developer (EU GPAI Code of Practice, Measure 7.4).
- METR's investigation report opens with the dates in scope and a redaction summary statement (METR, OpenAI-Hugging Face incident investigation).
Counter-examples
Not to be confused with
- A section of a developer's model card that summarises what an outside evaluator found is a developer summary, not the evaluator's report.
Related terms
Related AI safety terms
Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.
- Public commentary (risk report)
- Public commentary is an external reviewer's published assessment of a developer's risk report, covering the quality of its reasoning, the validity of its risk assessments, the overall risk level and whether redactions were reasonable.
- Source: Anthropic Responsible Scaling Policy v3.4
Source of record
Where this definition comes from
EU GPAI Code of Practice, Safety and Security Chapter, Measure 7.4(1)
“any available reports (e.g. via valid hyperlinks) from: (a) independent external evaluators involved in model evaluations pursuant to Appendix 3.5; and (b) security reviews undertaken by an independent external party pursuant to Appendix 4.5; to the extent that respects existing confidentiality (including commercial confidentiality) obligations and allows such external evaluators or parties to maintain control over the publication of their findings, without implicit endorsement by the Signatories of the content of such reports;”
https://ec.europa.eu/newsroom/dae/redirection/document/118119Anthropic Advanced AI Framework, p.8
“The evaluator should publish a review of the developer's most recent risk report, addressing the adequacy of the report's information; its analytical rigor; the appropriateness and materiality of any redactions made to the public version; and whether the evaluator disagrees with any of the report's key claims, especially its overall assessment of the level of risk for each Enumerated Risk category.”
https://www-cdn.anthropic.com/files/4zrzovbb/website/0a58d567024a8b448ff15158ebc3625328dfcc1f.pdfFRONTIER Act, 119th Congress (H.R. 9925, introduced, not enacted), Sec. 5(f)(2)
“(A) A description of the scope of the assessment, the time period covered, the materials reviewed, and the methods used. (B) A description of any limitations on the IVO's ability to thoroughly and adequately assess the adequacy of the very large frontier developer's frontier AI framework.”
https://www.govinfo.gov/content/pkg/BILLS-119hr9925ih/html/BILLS-119hr9925ih.htm
Crosswalk
How named organisations use this concept
| Organisation | Their term, as published | Match & verification | Source |
|---|---|---|---|
| EUShadow mapping EU GPAI Code of Practice, Safety and Security Chapter | “Measure 7.4 "External reports": the Model Report includes "any available reports" from independent external evaluators, allowing them "to maintain control over the publication of their findings, without implicit endorsement by the Signatories"; if none was involved, "a justification"; if one was, "an explanation of the choice of evaluator".” | exactEQ confidence: high | EU GPAI Code of Practice, Safety and Security Chapter |
| AnthropicShadow mapping Anthropic Advanced AI Framework / Responsible Scaling Policy v3.4 | “AAF: "The evaluator should publish a review of the developer's most recent risk report, addressing the adequacy of the report's information; its analytical rigor; the appropriateness and materiality of any redactions ...; and whether the evaluator disagrees with any of the report's key claims" (p.8). RSP §3.6: reviewers "publish comprehensive commentary" covering "the quality of our reasoning, the validity of our risk assessments, the overall level of risk, and whether the redactions we've made for the public version are reasonable and appropriate."” Also cites Anthropic's Responsible Scaling Policy v3.4. | exactEQ confidence: high | Anthropic Advanced AI Framework |
| US Congress (H.R. 9925, FRONTIER Act, not enacted)Shadow mapping FRONTIER Act, 119th Congress | “IVO "assessment report" contents: "(A) A description of the scope of the assessment, the time period covered, the materials reviewed, and the methods used. (B) A description of any limitations on the IVO's ability to thoroughly and adequately assess" the framework, findings, "corrective actions recommended", and "The signature of the IVO's lead assessment partner or equivalent responsible official"; the IVO then gives "an opinion" on the developer's response (Sec. 5(f)-(g)).” Not enacted. Reports go to the Under Secretary; a summary and redacted copy are published. | exactEQ confidence: high | FRONTIER Act, 119th Congress (H.R. 9925, introduced, not enacted) |
| METRShadow mapping METR OpenAI-Hugging Face incident investigation | “Report opens with "Dates in scope" and a "Redaction summary statement: Except where explicitly noted in this post, OpenAI redacted no additional information that was important to our conclusions."” A worked instance of the scope and redaction fields. | closeCL confidence: high | METR: OpenAI-Hugging Face incident investigation |
| AI Evaluator Forum (AEF-1)Shadow mapping AEF-1: Minimum Operating Conditions for Independent Third Party AI Evaluations | “"Evaluators demonstrate adherence by completing the checklist and publishing it alongside their results" (AEF-1 page); "3.4 Editorial control Requirement: The evaluator retained editorial control over how they present the results of their evaluation".” Also cites https://aievaluatorforum.org/initiatives/minimum-operating-conditions for the checklist sentence. | closeCL confidence: high | AEF-1: Minimum Operating Conditions for Independent Third Party AI Evaluations (v1, 4 Dec 2025) |
| California SB 53Shadow mapping California SB 53 | “Transparency reports of large frontier developers include summaries of "(C) The extent to which third-party evaluators were involved" (22757.12(c)(2)(C)).” A developer-written summary of involvement, not the evaluator's own report. | broadBR confidence: medium | California SB 53 |
What do these codes mean?
- exact
- The source term is equivalent to this element
- close
- The source term is close but not equivalent to this element
- broad
- The source term is broader than this element
- narrow
- The source term is narrower than this element
- none
- No mapping claim — used for false-friend and declared-but-undefined rows
- EQ
- Equivalent
- CL
- Close
- BR
- Source is broader than the element
- NR
- Source is narrower than the element
- FF
- False friend — same or similar label, different meaning
- DU
- Declared but undefined by the source
- UV
- Unverified
Related, not mapped
Pointers that are not crosswalk claims
These sources mention this concept but do not define or map it clearly enough to count as a crosswalk row — noted here so the research is visible without overstating it as a mapping.
- Google DeepMind
"The independent external evaluators explore frontier safety risk domains, however they do not directly comment on risk thresholds as contemplated under our FSF." (p.6) -- findings reported inside the developer's own document; a pointer, not a mapping.
Gemini 3.7 Flash FSF report
Divergence
Where sources materially disagree
Sources differ on who controls the text. The EU Code of Practice and Anthropic's policy leave publication control with the evaluator; in METR's cross-developer pilot, participants gave written approval for attributed text while METR kept editorial control of the rest; SB 53 asks only for the developer's own summary of evaluator involvement.
Gap
No common report template exists. AEF-1 provides an operating-conditions checklist, and the FRONTIER Act lists report contents, but no source defines a machine-readable format linking a report to the model version, access terms and redactions it covers.
