Skip to main content
v2026.11,858 entries · CC-BY 4.0
NIKOLAI elementN8 · Transparency and reviewProposednikolai-v0.1

Evaluator Access Attestation

NIKOLAI proposal: an Evaluator Access Attestation is a record, jointly attested by the developer and the evaluator, of what access an evaluator was granted and denied -- model versions, safeguard state, reasoning traces, weights, internal documents, personnel, systems -- for how long, and under what confidentiality and data-handling terms. This is NIKOLAI's own editorial synthesis, unsourced from any single document; no developer currently publishes a record in this shape.

This is CASRAI's own proposed definition, not a definition any named organisation has agreed to. See what NIKOLAI is and is not.

Source of record

Where this definition comes from

  • EU GPAI Code of Practice, Safety and Security Chapter, Appendix 3.4/3.5; Appendix 2.2

    adequate access, information, time, and other resources, including access to model activations, gradients, logits (or other forms of raw model outputs), chains-of-thought, and/or other technical details, and access to the model version(s) with the fewest safety mitigations implemented (such as a helpful-only model version, if it exists); indicative time floor 'at least 20 business days is appropriate for most systemic risks and model evaluation methods.'

    https://ec.europa.eu/newsroom/dae/redirection/document/118119
  • METR, engagement terms

    Each participant provided: Access to their most capable internal model(s) at the time of assessment, including raw chains of thought. Rate limits of at least 4M input tokens per minute, 1M output tokens per minute, and 1K requests per minute; zero data retention.

    https://metr.org/
  • Executive Order 14409

    provide the Federal Government with access to covered frontier models, subject to appropriate confidentiality, cybersecurity, insider-risk, and intellectual-property protection, use, and nondisclosure requirements, for a period of up to 30 days.

    https://www.whitehouse.gov/presidential-actions/2026/06/promoting-advanced-artificial-intelligence-innovation-and-security/

Crosswalk

How named organisations use this concept

Every row below is a shadow mapping. It is CASRAI's own reading of a published document. No lab, evaluator or regulator named here has declared, endorsed, or been consulted on this mapping. That will change only when an organisation files its own Mapping Declaration — see the non-endorsement policy.
OrganisationTheir term, as publishedMatchSource
Anthropic
Amodei "We Must Pace the Frontier" / alignment-assessment cybersecurity brief / Advanced AI Framework / Risk Report
"ongoing, employee-like access to a team of embedded third-party evaluators"; access "mostly comparable to what internal risk assessment teams have" (office desks, badges, laptops). METR agreement: "wide-ranging access, including to transcripts beyond the window in which the incidents occurred, and to Anthropic employees, who will be permitted to share confidential information. Our initial agreement runs for eight weeks, with the option to extend by mutual agreement." AAF evaluator access: "access to an unredacted version of the developer's most recent risk report and system cards, access to the developer's most capable models, and the opportunity to ask and receive reasonable responses." Researchers with classifier exemptions must "attest to a list of Anthropic-provided security requirements."
Also cites {ALA}, {AAF}, {RR}.
close
confidence: medium
We Must Pace the Frontier (Amodei essay)
OpenAI
Sam Altman post / GPT-5.6 deployment safety page / METR incident investigation
Altman: "independent evaluators with employee-like access" is "a great idea" and "OpenAI will do the same." GPT-5.6: UK AISI "extensive grey box access" (s.9.4.6); SecureBio "evaluated two pre-release checkpoints of GPT-5.6 Sol and a railfree version." Incident investigation: "OpenAI attested that the transcripts we reviewed were unredacted."
Also cites {G56}, {METRHF}.
close
confidence: medium
Sam Altman post (X)
xAI
Grok 4.6 model card / Grok 4.20 model card / Elon Musk post
"We additionally provided an unrestricted configuration of Grok 4.6 to third-party evaluators" (§7); Grok 4.20 early snapshot. Musk: "Peer review of AI by competitors is the right way to start this off."
Also cites {G420}, {MUSK}.
narrow
confidence: medium
Grok 4.6 model card
Meta
Meta Advanced AI Scaling Framework v2
Preparedness reports describe "details about elicitation, time and resources spent, and access given to internal and external evaluators" (§2.2.1). A disclosure field, not an attestation.close
confidence: medium
Meta Advanced AI Scaling Framework v2
EU
EU GPAI Code of Practice, Safety and Security Chapter
Appendix 3.4/3.5: independent external evaluators get "adequate access, information, time, and other resources", including "access to model activations, gradients, logits (or other forms of raw model outputs), chains-of-thought, and/or other technical details, and access to the model version(s) with the fewest safety mitigations implemented"; indicative time floor "at least 20 business days"; qualification requires domain expertise, security protocols and a confidentiality agreement. Post-market monitoring (Measure 3.5) separately guarantees "adequate free access" to the most capable deployed version, its CoT and its least-mitigated version, unless "similarly safe or safer" (Appendix 2.2).exact
confidence: high
EU GPAI Code of Practice, Safety and Security Chapter
US Government (Executive Order 14409 / NIST CAISI)
Executive Order 14409 / NIST CAISI bulletin
"provide the Federal Government with access to covered frontier models, subject to appropriate confidentiality, cybersecurity, insider-risk, and intellectual-property protection, use, and nondisclosure requirements, for a period of up to 30 days." CAISI agreement terms are not public (gap).
Also cites {CAISI}.
close
confidence: medium
Executive Order 14409
METR
METR (site)
"Each participant provided: Access to their most capable internal model(s) at the time of assessment, including raw chains of thought." Rate limits "of at least 4M input tokens per minute, 1M output tokens per minute, and 1K requests per minute"; zero data retention. "We believe that there should be full transparency about the terms of the engagement, including at least redaction terms, access provided, time and personnel provided, and agreed upon scope of the investigation."exact
confidence: high
METR
Frontier Model Forum
FMF Third-Party Assessments technical report
"Appropriate access: balances information needs with security considerations ... providing only the minimum information necessary"; "Security readiness: ... third-party assessors must demonstrate their capacity to responsibly handle privileged access before receiving it." No numbered access levels.close
confidence: medium
FMF Third-Party Assessments
AI Evaluator Forum (AEF-1, discovery sweep)
Discovery sweep
Proposed access levels "AL1 black-box; AL2 grey-box; AL3 white-box" (arXiv 2601.11916); AEF-1 condition "Sufficient Access and Resources" (discovery).
Unverified discovery-stage material, not a primary-source developer commitment.
close
confidence: low
AI Evaluator Forum (AEF-1) / arXiv 2601.11916 discovery sweep

Related, not mapped

Pointers that are not crosswalk claims

These sources mention this concept but do not define or map it clearly enough to count as a crosswalk row — noted here so the research is visible without overstating it as a mapping.

Gap

*Gap to record:* METR's call for "full transparency about the terms of the engagement" is the only source that treats access terms as a publishable record. Amodei's commitment adds "access they did or did not receive" as publishable content. No template exists.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →