Skip to main content
v2026.11,986 entries · CC-BY 4.0
CASRAIAsk CASRAI · AI assistant for researchAsk a research question. Answers come with their sources.Funder rules, compliance, methods and publishing. Answers link the sources they rest on.
NIKOLAI elementN8 · Transparency and reviewProposednikolai-v0.3

Evaluator Access Attestation

Definition

An evaluator access attestation is a statement, confirmed by both the AI developer and the outside evaluator, of what access the evaluator received and was denied: model versions, safeguards, reasoning traces, documents, staff and systems, for how long, and under what confidentiality terms.

Also called: Evaluator access terms, Terms of engagement, Access and resources disclosure, Sufficient Access and Resources, Appropriate access

NIKOLAI's proposed operational definition

NIKOLAI proposal: an Evaluator Access Attestation is a record, jointly attested by the developer and the evaluator, of what access an evaluator was granted and denied -- model versions, safeguard state, reasoning traces, weights, internal documents, personnel, systems -- for how long, and under what confidentiality and data-handling terms. This is NIKOLAI's own editorial synthesis, unsourced from any single document; no developer currently publishes a record in this shape.

The operational definition is CASRAI's own proposal, not a definition any named organisation has agreed to. See what NIKOLAI is and is not, or browse the AI safety glossary.

Examples

Examples from published documents

  • OpenAI reports that UK AISI received 'extensive grey box access': the chain-of-thought of the safety reasoning monitor, exact policy wording and real-time classifier-label feedback (OpenAI GPT-5.6 System Card, s.9.4.6).
  • Participants in METR's cross-developer risk report provided their most capable internal models, including raw chains of thought, and non-public information on internal AI use (METR Frontier Risk Report, Feb-Mar 2026).
  • The Model Report must describe 'the access and other resources provided to' internal evaluation teams and independent external evaluators (EU GPAI Code of Practice, Measure 7.3(1)(g)).

Counter-examples

Not to be confused with

  • A developer's one-sided list of 'access given to evaluators' in its own report is a disclosure, not an attestation: nothing records whether the evaluator agrees the list is complete.
  • The pre-release sharing window (how many days before release a tester gets the model) is one attribute of access, recorded separately.

Related terms

Related AI safety terms

Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.

Embedded evaluator
An embedded evaluator is a third-party evaluator given ongoing, employee-like access inside an AI developer, such as office desks, badges, laptops and internal tools, to verify safety practices and report incidents, rather than a time-limited engagement.
Source: We Must Pace the Frontier (essay by Anthropic's CEO)
Pre-deployment testing
Pre-deployment testing is evaluation of an AI model by an outside party, such as a government AI institute, before public release, under an arrangement that gives the tester early access to the unreleased model.
Source: OpenAI GPT-5.6 System Card
Appropriate access
Appropriate access is the level of model access, information and time an outside assessor needs for a given assessment, balanced against security, so that the developer shares only the minimum information necessary.
Source: FMF Third-Party Assessments
Security readiness
Security readiness is an assessor's demonstrated ability to handle privileged access safely, through data-handling protocols, confidentiality agreements and access controls, before a developer grants it access to models or sensitive information.
Source: FMF Third-Party Assessments

Source of record

Where this definition comes from

Crosswalk

How named organisations use this concept

Every row below is a shadow mapping. A shadow row is CASRAI's own reading of a published document. No lab, evaluator or regulator named on a shadow row has declared, endorsed, or been consulted on it. That changes only when an organisation files its own Mapping Declaration.
OrganisationTheir term, as publishedMatch & verificationSource
AnthropicShadow mapping
We Must Pace the Frontier (essay by Anthropic's CEO) / Anthropic Alignment Assessment: Cybersecurity Incidents / Advanced AI Framework / Risk Report
“Essay: "Each frontier AI company commits to giving ongoing, employee-like access to a team of embedded third-party evaluators"; "Desks in our offices, access badges, and company laptops. Access to workspaces, tools, and permissions mostly comparable to what internal risk assessment teams have." METR agreement: "wide-ranging access, including to transcripts beyond the window in which the incidents occurred, and to Anthropic employees, who will be permitted to share confidential information. Our initial agreement runs for eight weeks". AAF: "access to an unredacted version of the developer's most recent risk report and system cards, access to the developer's most capable models, and the opportunity to ask and receive reasonable responses".”
Also cites Anthropic Alignment Assessment: Cybersecurity Incidents, the Advanced AI Framework and the Risk Report.
closeCL
confidence: medium
We Must Pace the Frontier (essay by Anthropic's CEO)
OpenAIShadow mapping
Post on X by OpenAI's CEO / OpenAI GPT-5.6 System Card / METR incident investigation
“Post: "Committing to having independent evaluators with employee-like access is a great idea, and we will do the same." GPT-5.6 System Card: "UK AISI was provided with extensive grey box access: access to chain-of-thought of the safety reasoning monitor, exact policy wording, and real-time feedback on classifier labels that would not be accessible to real-world attackers" (s.9.4.6); SecureBio "evaluated two pre-release checkpoints of GPT-5.6 Sol and a railfree version of GPT-5.6 Sol" (s.9.1.1.8). METR investigation: "OpenAI attested that the transcripts we reviewed were unredacted."”
Also cites the GPT-5.6 deployment safety card and METR's OpenAI/Hugging Face incident investigation.
closeCL
confidence: medium
Post on X by OpenAI's CEO
xAIShadow mapping
Grok 4.6 model card / Grok 4.20 model card / post on X by xAI's founder
“"We additionally provided an unrestricted configuration of Grok 4.6 to third-party evaluators, who corroborated the results of our internal evaluations and testing on cyber capabilities" (§7); Grok 4.20: early snapshot to third-party evaluators. Post: "Peer review of AI by competitors is the right way to start this off."”
Also cites the Grok 4.20 model card and a post on X by xAI's founder.
narrowNR
confidence: medium
Grok 4.6 model card
MetaShadow mapping
Meta Advanced AI Scaling Framework v2
“Preparedness reports describe "details about elicitation, time and resources spent, and access given to internal and external evaluators" (§2.2.1). A disclosure field, not an attestation.”closeCL
confidence: medium
Meta Advanced AI Scaling Framework v2
EUShadow mapping
EU GPAI Code of Practice, Safety and Security Chapter
“Appendix 3.5: "Signatories will provide independent external evaluators with adequate access, information, time, and other resources (pursuant to Appendix 3.4)". Appendix 3.4 defines that as "adequate access to the model ... including, as appropriate, access to model activations, gradients, logits (or other forms of raw model outputs), chains-of-thought, and/or other technical details, and access to the model version(s) with the fewest safety mitigations implemented (such as a helpful-only model version, if it exists)" and time: "For example, a period of at least 20 business days is appropriate for most systemic risks and model evaluation methods". Qualification requires domain expertise, information-security protocols and agreement to protect confidential information. Measure 7.3(1)(g): the Model Report gives "a description of the access and other resources provided to" internal teams and independent external evaluators. Measure 3.5 (post-market monitoring): "adequate free access" to the most capable deployed version, its chains-of-thought and the least-mitigated version, provided "through an API, on-premise access (including transport), access via Signatory-provided hardware, or by making the model parameters publicly available for download, as appropriate."”exactEQ
confidence: high
EU GPAI Code of Practice, Safety and Security Chapter
US Government (Executive Order 14409 / NIST CAISI)Shadow mapping
Executive Order 14409 / NIST CAISI bulletin
“"provide the Federal Government with access to covered frontier models, subject to appropriate confidentiality, cybersecurity, insider-risk, and intellectual-property protection, use, and nondisclosure requirements, for a period of up to 30 days." The terms of agreements made by CAISSI (formerly CAISI) are not public (gap).”
Also cites the NIST CAISI bulletin.
closeCL
confidence: medium
Executive Order 14409
METRShadow mapping
METR Frontier Risk Report (Feb-Mar 2026) / METR incident-investigation post (28 Jul 2026)
“"Each participant provided: Access to their most capable internal model(s) at the time of assessment, including raw chains of thought." Requested: "Rate limits of at least 4M input tokens per minute, 1M output tokens per minute, and 1K requests per minute" and "Zero data retention (ZDR), if possible". 28 Jul 2026 post: "we believe that there should be full transparency about the terms of the engagement, including at least redaction terms, access provided, time and personnel provided, and agreed upon scope of the investigation."”
Also cites METR's 28 Jul 2026 post (https://metr.org/blog/2026-07-28-investigating-ai-propensities-after-incidents/).
exactEQ
confidence: high
METR Frontier Risk Report (February-March 2026)
Frontier Model ForumShadow mapping
FMF Third-Party Assessments technical report
“"Appropriate access: balances information needs with security considerations ... providing only the minimum information necessary"; "Security readiness: ... third-party assessors must demonstrate their capacity to responsibly handle privileged access before receiving it." No numbered access levels.”closeCL
confidence: medium
FMF Third-Party Assessments
AI Evaluator Forum (AEF-1)Shadow mapping
AEF-1: Minimum Operating Conditions for Independent Third Party AI Evaluations / arXiv 2601.11916
“Proposed access levels "AL1 (black-box model access and minimal information), AL2 (grey-box model access and substantial information), and AL3 (white-box model access and comprehensive information)" (arXiv 2601.11916, abstract); AEF-1 principle "Sufficient Access and Resources": "1.1 Technical Access Requirement: The evaluator secured sufficient technical access to assess the specific system characteristics being evaluated."”
Verified against the AEF-1 PDF and the arXiv 2601.11916 abstract (fetched 13 Nov 2026). The arXiv paper is an academic proposal, not a developer commitment; see the model-access-level element for the taxonomy.
closeCL
confidence: medium
AEF-1: Minimum Operating Conditions for Independent Third Party AI Evaluations (v1, 4 Dec 2025)
What do these codes mean?
exact
The source term is equivalent to this element
close
The source term is close but not equivalent to this element
broad
The source term is broader than this element
narrow
The source term is narrower than this element
none
No mapping claim — used for false-friend and declared-but-undefined rows
EQ
Equivalent
CL
Close
BR
Source is broader than the element
NR
Source is narrower than the element
FF
False friend — same or similar label, different meaning
DU
Declared but undefined by the source
UV
Unverified

Related, not mapped

Pointers that are not crosswalk claims

These sources mention this concept but do not define or map it clearly enough to count as a crosswalk row — noted here so the research is visible without overstating it as a mapping.

Gap

No developer, evaluator or regulator publishes access terms in a common record format. METR calls for 'full transparency about the terms of the engagement', the essay by Anthropic's CEO says reviewers should be able to publish 'the access they received or didn't receive', and AEF-1 supplies a disclosure checklist, but none defines fields that both parties confirm.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →