Skip to main content
v2026.11,986 entries · CC-BY 4.0
NIKOLAI elementN8 · Transparency and reviewProposednikolai-v0.3

Evaluator Safe Harbour

Definition

An evaluator safe harbour is a developer's commitment not to take legal or technical action against an outside evaluator for testing its AI model within an agreed scope or for publishing the findings, usually conditioned on responsible disclosure.

Also called: Safe harbor, Non-retaliation clause, Legal safe harbor for evaluators, Researcher safe harbour

NIKOLAI's proposed operational definition

NIKOLAI proposal: Evaluator Safe Harbour is a rule recording a developer's commitment not to take legal or technical action against an outside evaluator for testing within an agreed scope or for publishing findings, together with the conditions the evaluator must meet (scope limits, data handling, responsible disclosure, timelines) and any statutory immunity that applies. This is NIKOLAI's own editorial synthesis.

The operational definition is CASRAI's own proposal, not a definition any named organisation has agreed to. See what NIKOLAI is and is not, or browse the AI safety glossary.

Examples

Examples from published documents

  • Signatories will not retaliate against independent external evaluators for testing or publishing findings, provided they follow a disclosure procedure that caps publication delay at 30 business days (EU GPAI Code of Practice, Measure 3.5).
  • AEF-1 recommends that the provider grant legal safe harbour for evaluator actions within the agreed scope of the evaluation (AEF-1, condition 1.5).

Counter-examples

Not to be confused with

  • Whistleblower protection for a developer's own employees is not an evaluator safe harbour; it covers staff reporting risks, not outside testers.
  • Immunity from third-party lawsuits for a licensed verifier protects against claims by others, not against action by the developer being evaluated.

Related terms

Related AI safety terms

Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.

Responsible vulnerability disclosure
Responsible vulnerability disclosure is the practice of first reporting a newly found weakness in an AI system, such as a safeguard bypass or prompt-injection method, to the provider and other affected providers before making it public.
Source: AEF-1: Minimum Operating Conditions for Independent Third Party AI Evaluations (v1, 4 Dec 2025)
Non-retaliation (evaluators)
Non-retaliation is a developer's undertaking not to take any legal or technical retaliation against independent external evaluators as a consequence of their testing or their publication of findings, subject to stated conditions.
Source: EU GPAI Code of Practice, Safety and Security Chapter

Source of record

Where this definition comes from

Crosswalk

How named organisations use this concept

Every row below is a shadow mapping. A shadow row is CASRAI's own reading of a published document. No lab, evaluator or regulator named on a shadow row has declared, endorsed, or been consulted on it. That changes only when an organisation files its own Mapping Declaration.
OrganisationTheir term, as publishedMatch & verificationSource
EUShadow mapping
EU GPAI Code of Practice, Safety and Security Chapter
“"Signatories will not take any legal or technical retaliation against the independent external evaluators as a consequence of their testing and/or publication of findings as long as the evaluators" meet five conditions: no intentional disruption of model availability; no misuse of sensitive or confidential user data; no activities posing "a significant risk to public safety and security"; no use of findings "to threaten Signatories"; and adherence to a responsible vulnerability disclosure procedure under which publication cannot be delayed or blocked "for more than 30 business days" (Measure 3.5).”
Applies to post-market monitoring access for independent external evaluators.
exactEQ
confidence: high
EU GPAI Code of Practice, Safety and Security Chapter
AI Evaluator Forum (AEF-1)Shadow mapping
AEF-1: Minimum Operating Conditions for Independent Third Party AI Evaluations
“"1.5 Safe harbor Recommendation: The system provider provided legal safe harbor for actions by the evaluator that are within the agreed-upon scope of the evaluation."”
A recommendation, not a requirement, within AEF-1's 'Sufficient Access and Resources' principle.
exactEQ
confidence: high
AEF-1: Minimum Operating Conditions for Independent Third Party AI Evaluations (v1, 4 Dec 2025)
What do these codes mean?
exact
The source term is equivalent to this element
close
The source term is close but not equivalent to this element
broad
The source term is broader than this element
narrow
The source term is narrower than this element
none
No mapping claim — used for false-friend and declared-but-undefined rows
EQ
Equivalent
CL
Close
BR
Source is broader than the element
NR
Source is narrower than the element
FF
False friend — same or similar label, different meaning
DU
Declared but undefined by the source
UV
Unverified

Related, not mapped

Pointers that are not crosswalk claims

These sources mention this concept but do not define or map it clearly enough to count as a crosswalk row — noted here so the research is visible without overstating it as a mapping.

  • US Congress (H.R. 9925, FRONTIER Act, not enacted)

    "An IVO shall be immune from suit and liability under Federal and State law with respect to all claims for loss caused by, arising out of, relating to, or resulting from the materialization of a catastrophic risk of a frontier model assessed by the IVO if the IVO was licensed by the Under Secretary under this section at the time of the assessment" (Sec. 5(q)) -- liability immunity for licensed verifiers, not protection from the developer; a pointer, not a mapping. Not enacted.

    FRONTIER Act, 119th Congress (H.R. 9925, introduced, not enacted)

Gap

The research did not find a standing evaluator safe-harbour commitment in the lab frameworks reviewed; AEF-1 treats safe harbour as a per-engagement recommendation. California SB 53 protects covered employees who report risks, not outside evaluators.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →