CASRAI Dictionary

Tag: AI Act GPAI

  • AI Literacy Obligation Article 4: Staff Training Checklist

    The AI literacy obligation Article 4 of the EU Artificial Intelligence Act has applied since 2 February 2025, yet many research institutions still treat it as a future item rather than a live requirement. Article 4 states that providers and deployers of AI systems “shall take measures to ensure, to their best extent, a sufficient level of AI literacy of their staff and other persons dealing with the operation and use of AI systems on their behalf.” For universities, funders, and research institutes running admissions tools, grant-screening software, or generative AI assistants, that duty is already in force. This article sets out a practical checklist for what “sufficient” means in a research setting, who it covers, and how to evidence it.

    What Article 4 actually requires

    Article 4 does not prescribe a fixed curriculum, test, or certificate. It requires institutions to calibrate AI literacy measures against staff technical knowledge, experience, education, training, and the context in which AI systems are used. The obligation applies to both “providers” (organisations that develop and place an AI system on the market) and “deployers” (organisations using an AI system for a professional purpose).

    A research institution can be either, sometimes simultaneously. A department that builds a bespoke research-data classification model is a provider of that system; the same university using an off-the-shelf tool for applicant screening or meeting transcription is a deployer. The AI Act scope includes a narrow exemption for systems developed and used exclusively for scientific research before market placement — but once a research tool is deployed operationally, that exemption falls away and Article 4 applies in full.

    The statutory definition of AI literacy, in Article 3(56), is: skills, knowledge and understanding that allow providers, deployers and affected persons to make an informed deployment of AI systems, and gain awareness of the opportunities and risks it can cause. The Commission’s AI Office has confirmed it will not impose a single mandatory format.

    Who counts as “staff” under the AI literacy obligation

    This is where most institutions under-scope their programmes. The Commission’s guidance clarifies that “staff and other persons” is broader than payroll headcount — it extends to anyone acting on the institution’s behalf, including contractors and service providers. In a research setting that typically means:

    • Researchers and PIs using generative AI tools for literature review, drafting, or data analysis
    • Research administrators and grants officers using AI-assisted screening or compliance-checking tools
    • HR and admissions staff using AI-based applicant or candidate screening systems
    • IT and research-computing staff configuring or maintaining institutional AI deployments
    • External contractors or visiting researchers granted access to institutional AI systems
    • Board and senior leadership, who need enough literacy to assess institutional AI risk

    Article 4 does not require every group to receive identical training: a developer configuring a high-risk system needs materially more depth than an administrator using a transcription tool. The table below illustrates how role and system risk map onto training depth.

    Personnel category Typical AI Act role Indicative literacy depth
    Data scientists building institutional AI tools Provider Technical: limitations, bias, risk mitigation, documentation
    Grants/admissions staff using screening AI Deployer Operational: output interpretation, human oversight, escalation
    Researchers using generative AI for drafting/analysis Deployer (or exempt if pre-market research use) General awareness: hallucination risk, confidentiality
    Contractors/visiting staff with system access “Other persons” acting on the institution’s behalf Baseline awareness proportionate to access level
    Governing board/senior leadership Deployer (oversight capacity) Strategic: risk appetite, resourcing, regulatory exposure

    Building a defensible AI literacy programme

    The Commission’s AI Literacy Q&A sets out a four-step minimum approach research institutions can adapt directly:

    • Establish a general understanding of AI within the organisation — what AI is used, where, and why
    • Determine the institution’s role for each system: provider, deployer, or both
    • Assess the risk level of each AI system in use, including any high-risk systems under Chapter III
    • Build literacy actions proportionate to staff knowledge gaps and the context of use

    Relying solely on a vendor’s instructions for use is explicitly insufficient — this mirrors the separate Article 26 duty on deployers of high-risk systems to ensure staff are “sufficiently trained” to exercise human oversight.

    What is Article 4 of the EU AI Act?

    Article 4 is the AI Act provision requiring providers and deployers to ensure a “sufficient level” of AI literacy among staff and other persons operating AI systems on their behalf. It has applied since 2 February 2025, ahead of most other AI Act obligations, and covers technical knowledge, training context, and the persons affected by the system.

    What is the definition of AI literacy in the EU AI Act?

    Article 3(56) defines AI literacy as the skills, knowledge and understanding that let providers, deployers and affected persons make informed decisions about deploying AI systems, while remaining aware of the opportunities, risks, and potential harms those systems can cause in their specific context of use.

    Who at a research institution needs AI literacy training under Article 4?

    Anyone dealing with an AI system’s operation or output on the institution’s behalf, not just IT or data-science staff. This includes researchers, administrators, HR and admissions teams, contractors, and senior leadership — with training depth proportionate to each group’s technical role and the risk level of the system they use.

    Does Article 4 require a certificate or formal training records?

    No. The AI Office has confirmed there is no mandated test or certificate for Article 4 compliance. Institutions should instead keep an internal record of training delivered, attendance, and content — evidence that becomes essential if a national market surveillance authority later reviews compliance.

    Documenting compliance: what evidence to keep

    Because Article 4 sets no certification standard, the practical question is evidential: what would a research institution show a national market surveillance authority if asked? A defensible record typically includes:

    • A written AI inventory identifying each system in use, its provider/deployer classification, and risk tier
    • Training content and delivery records (dates, attendance, format) mapped to each staff category
    • A documented rationale for why the chosen literacy measures were “sufficient” for each role and system
    • An AI use policy communicated to staff, contractors, and other relevant persons
    • A review cycle — literacy measures should be revisited as systems and risk profiles change

    National market surveillance authorities take over enforcement of Article 4 from 2 August 2026, when the AI Act’s general application and high-risk provisions take effect. Enforcement is meant to be proportionate — gravity, intent, and negligence are all considered — but an incident with no evidence of staff training is likely to weigh against an institution.

    Implications for research institutions and GPAI tools

    Research institutions increasingly deploy general-purpose AI (GPAI) tools — large language model assistants embedded in research-writing or literature-search workflows — rather than narrow, purpose-built systems. The AI Act GPAI provisions (Chapter V, applying to GPAI model providers from 2 August 2025) sit alongside, not instead of, the Article 4 duty on deployers: using a GPAI-based writing assistant still makes an institution a deployer under Article 4, regardless of the model provider’s own transparency obligations.

    One development worth tracking: the European Commission’s 2025 Digital Omnibus proposal would shift part of the Article 4 burden from individual organisations toward Member States and the Commission itself. That proposal has not been adopted, so the current text of Article 4 remains binding on institutions as providers or deployers — institutions should not defer action for a change that may not materialise as proposed.

    Sector signals reinforce the direction of travel: UK courts have separately expected legal professionals to demonstrate AI competence in submissions, and UK financial regulators have referenced the Senior Managers Regime in connection with AI risk oversight — evidence that “sufficient AI literacy” is becoming an expectation beyond the AI Act’s direct territorial reach.

    What research institutions should do next

    Institutions without a formal AI literacy programme should start with an AI system inventory, classify each system by provider/deployer role and risk tier, then build tiered training from that analysis rather than buying a generic module. Article 4 rewards documented judgement over box-ticking: the institutions best placed for enforcement from August 2026 will be those that can show a reasoned, evidenced, role-differentiated approach. Research-administration functions, which typically already own institutional policy documentation, are well positioned to lead this work alongside data protection, IT governance, and research integrity offices.

  • EU AI Act Research Exemption: What Article 2(6) Actually Covers

    A run of academic literature published since mid-2025 — an editorial in GRUR International, a peer-reviewed analysis in Nature’s npj Digital Medicine, and a widely cited Swedish doctoral paper — has converged on the same conclusion: the EU AI Act research exemption is far narrower, and far less certain, than most research offices assume. Regulation (EU) 2024/1689 does carve scientific research and development out of scope, but that carve-out is built from two separate provisions with different wording, different triggers, and different failure points. For institutions running AI-assisted studies, clinical trials, or general-purpose model development, misreading where the exemption ends is now a live compliance risk.

    What Article 2(6) actually says

    Article 2(6) of the AI Act states that the Regulation “does not apply to AI systems or AI models, including their output, specifically developed and put into service for the sole purpose of scientific research and development.” Two conditions must both be met: the system or model must be developed for scientific research, and it must be put into service — first used for its intended purpose — exclusively for that research. Recital 25 is the only interpretive gloss the legislative text offers, and it does not define “scientific research and development” further.

    Critically, Article 2(6) exempts systems that are put into service for research, but it does not extend to systems that are placed on the market. That distinction — put into service versus placed on the market, defined respectively in Articles 3(10) and 3(9) — is where the exemption’s practical limits begin.

    Two exemptions, not one: Article 2(6) vs Article 2(8)

    Law professor Michèle Finck’s October 2025 editorial “In Search of the Lost Research Exemption” (GRUR International, Vol. 74, Issue 10) makes the point that is most often missed: the AI Act contains two distinct research exemptions, not one. Article 2(6) is narrow and limited to scientific research; Article 2(8) is broader and covers any research, testing or development activity, scientific or not, but only up to the point of market placement or service.

    Provision What it exempts Key limit
    Article 2(6) AI systems/models developed and put into service solely for scientific research and development Not limited to pre-market stage, but strictly tied to “sole purpose” of research — loses protection once put into service for any other use
    Article 2(8) Any research, testing or development activity (not limited to science) regarding AI systems or models Applies only prior to placing on the market or putting into service; explicitly excludes real-world testing

    Finck argues that this dual structure creates an “interpretative conundrum”: if Article 2(8) only ever covers activity that happens before market placement, and market placement is already the trigger for the Act’s obligations regardless of the exemption, the provision risks adding little independent legal value — precisely the ambiguity that gives the “lost” exemption its name.

    Where the research exemption stops applying

    The Nature-published analysis by Meszaros and colleagues (npj Digital Medicine, 2026) sets out a conceptual framework built around a single regulatory threshold: placement on the market or putting into service. Everything on the research side of that line can be exempt; everything on the other side is regulated. Three scenarios repeatedly cross that line.

    Commercialisation and dual-purpose systems

    A system loses its exemption the moment it is not developed for the sole purpose of research. Finck highlights that Horizon Europe-style collaborations, where a university partners with an industrial co-investigator who intends to commercialise the output once the exploratory phase ends, sit in exactly this grey zone. Whether “commercial purpose” is assessed objectively (does a commercial partner exist) or subjectively (did the researchers intend commercialisation) remains unresolved in the text itself.

    Post-market deployment and real-world testing

    Article 2(8) states plainly that “testing in real-world conditions shall not be covered by that exclusion.” A model tested only in a closed lab environment can remain exempt; the same model tested on live users, patients, or public-facing systems generally cannot, unless it proceeds through the Act’s dedicated real-world testing and regulatory sandbox framework (Articles 57–61). Colonna’s 2024 analysis for the DiVA repository similarly stresses that the exemption was never intended to cover deployment-stage activity dressed up as “ongoing research.”

    GPAI models and systemic-risk obligations

    Because Article 2(6) explicitly names “AI models” alongside “AI systems,” a general-purpose AI (GPAI) model built and used solely for research is exempt. That exemption evaporates once a provider places the model on the Union market — including releasing a checkpoint for downstream use beyond pure research. From that point, Title VIII’s GPAI obligations under Article 53 (technical documentation, copyright-compliance summaries) apply, and models presumed to carry systemic risk — those trained with cumulative compute above 10^25 FLOPs — face the additional Article 55 duties regardless of open-source licensing. A separate, unconditional exclusion exists for military, defence and national-security AI under Article 2(3); that provision is absolute and is not contingent on “sole purpose,” unlike the research exemptions.

    Frequently asked questions

    What is Article 2(6) of the EU AI Act?

    Article 2(6) excludes AI systems and AI models — including their output — from the AI Act when they are specifically developed and put into service for the sole purpose of scientific research and development. It does not, however, exempt systems that have been placed on the market.

    Does the AI Act research exemption cover real-world testing?

    No. Article 2(8) states explicitly that testing in real-world conditions is not covered by the research exclusion. Researchers deploying systems outside a controlled setting generally need to use the Act’s regulatory sandbox and real-world testing framework instead.

    Are GPAI models exempt from the AI Act during research?

    Yes, while a general-purpose AI model is developed and used solely for research it falls outside scope. Once placed on the market, Title VIII obligations attach, with stricter Article 55 duties for models presumed to carry systemic risk above the 10^25 FLOPs training-compute threshold.

    Can university-industry collaborations rely on the research exemption?

    Only where the sole purpose remains scientific research. Per Finck’s 2025 analysis, Horizon Europe-style projects involving a commercial partner intending future exploitation risk losing Article 2(6)/2(8) protection once a profit-oriented purpose is established.

    What this means for research institutions and publishers

    Research administration offices — the ARMA, EARMA and INORMS community that oversees institutional compliance — now have a practical due-diligence question to add to AI-enabled research proposals: at what point does this project’s AI system move from “developed for research” to “put into service” or “placed on the market”? That question matters most for:

    • Clinical and biomedical AI tools that progress from retrospective lab validation to prospective real-world testing on patients.
    • Multi-partner Horizon Europe consortia where an industrial partner holds commercialisation rights from the outset.
    • Open-source model releases on code and model-sharing platforms, which several commentators — including the arXiv paper “Beware! The AI Act Can Also Apply to Your AI Research” — flag as a possible trigger for “placing on the market.”
    • Foundational research (for example, in AI explainability or causal reasoning) whose downstream applications are not yet known at the outset, which Finck notes may struggle to meet the “sole purpose” test even where no commercial partner is currently involved.

    Institutions with dedicated research administration functions are best placed to build this threshold assessment into ethics review and grant-agreement workflows now, rather than retrofitting compliance once a system reaches deployment.

    Looking ahead

    The AI Act’s general provisions, including Article 2’s scope rules, have applied since 2 February 2025; GPAI obligations followed on 2 August 2025; most remaining obligations, including high-risk system requirements under Annex III, become applicable from 2 August 2026. Every commentator reviewed here — Finck, Meszaros et al., and Colonna — reaches the same practical conclusion: the European Commission’s promised guidance on the research exemptions has not yet resolved the “sole purpose,” commercial-intent, and real-world-testing ambiguities in the text. Until that guidance lands, institutions should treat the exemption as a narrow, conditional safe harbour rather than a blanket shield, and document the specific research purpose, funding structure, and deployment plan for every AI system that currently relies on it.

  • AI in Grant Peer Review: How ERC, NIH, UKRI and NHMRC Draw the Line

    Four major funders have now published, or are actively revising, formal rules on AI in grant peer review, and the details differ enough that a reviewer moving between panels could unknowingly breach one funder’s terms while complying with another’s. In March 2026 the European Research Council (ERC) issued new guidelines on AI use in evaluation; the US National Institutes of Health (NIH) tightened its stance on AI-drafted applications from September 2025; UK Research and Innovation (UKRI) maintains a stricter blanket ban that peers expect to loosen; and Australia’s National Health and Medical Research Council (NHMRC) introduces a revised generative-AI policy from 28 April 2026. Research offices drafting or updating reviewer agreements need to track all four.

    How ERC, NIH, UKRI and NHMRC draw the line

    Each funder separates permitted “AI-assisted” support from prohibited “AI-generated” evaluation, but the exact boundary — and the effective date — varies.

    Funder Rule effective AI-assisted (permitted) AI-generated (prohibited)
    ERC 24 March 2026 Language polishing of a reviewer’s own report; general (non-proposal) information searches Summarising proposals, assessing scientific merit, drafting evaluations, uploading any proposal content to external AI systems
    NIH Applications submitted from 25 September 2025 Limited administrative tasks in application preparation Reviewers using generative AI to analyse applications or formulate critiques; applications “substantially developed by AI” are treated as non-original and not reviewed
    UKRI Current policy; Research Funding Policy Group review pending None yet formally sanctioned for reviewers — even AI-assisted grammar checks are currently disallowed Any generative AI use by reviewers or panellists in assessing applications
    NHMRC 28 April 2026 Generative AI to refine clarity or grammar of a reviewer’s own comments Using AI to evaluate, critique or score applications

    A fifth data point is worth noting: the US-based Foundation for Food & Agriculture Research (FFAR) went further still in November 2025, prohibiting reviewers from using AI tools in any capacity during peer review — including refinement of their own comments — on confidentiality grounds. That makes FFAR the strictest outlier against which UKRI’s current position, and NHMRC’s narrower allowance, can be benchmarked.

    • Confidentiality is the universal red line. Every policy reviewed prohibits uploading proposal text, applicant data or reviewer notes into public or third-party AI tools.
    • Non-delegation is the second constant. Scientific merit assessment must remain a human judgement in all four jurisdictions, regardless of how permissive the language-polishing allowance is.
    • UKRI is currently the most conservative of the four, with a sector-wide Research Funding Policy Group review expected to permit limited generative AI use in processing (not scoring) applications while keeping final decisions human-made.

    AI-assisted vs AI-generated: common questions

    Research offices repeatedly ask the same handful of questions when briefing reviewers. The answers below are grounded in the funder documents referenced above.

    What is the difference between AI-assisted and AI-generated peer review?

    AI-assisted review means a human reviewer uses a tool only for mechanical tasks — grammar, clarity, formatting of their own text — while retaining full intellectual authorship of the assessment. AI-generated review means the AI performs part of the evaluative task itself, such as summarising a proposal, scoring merit, or drafting critique content, which every funder surveyed here prohibits.

    Has NIH banned AI in grant peer review?

    Yes. NIH prohibits scientific peer reviewers from using generative AI tools to analyse applications or formulate critiques, a position it has held since June 2023. From 25 September 2025, NIH also treats applications substantially developed by AI as non-original, removing them from review rather than scoring them on merit.

    Can UKRI reviewers use AI to check grammar in their assessments?

    Not currently. UKRI’s existing policy forbids reviewers and panellists from using generative AI for any part of assessment, including language or grammar correction — a stricter line than ERC or NHMRC. A sector-wide funder policy group is expected to revisit this, but any change would still require human-made final decisions.

    When does the NHMRC generative AI policy take effect?

    NHMRC’s revised Policy on Use of Generative Artificial Intelligence in Grant Review takes effect from 28 April 2026. It permits peer reviewers to use generative AI to refine the clarity or grammar of their own comments, but explicitly prohibits using AI to evaluate, critique or score applications.

    Practical reviewer-agreement language for research offices

    Research offices administering panels — whether for an internal seed-fund competition, a co-funded international call, or as a delegated peer-review manager for an external funder — need reviewer agreements that anticipate divergence between funder rules. Three drafting principles reduce risk:

    • Name the prohibited actions explicitly, not just the tool category. A clause banning “AI tools” is weaker than one banning “uploading proposal content, applicant identifiers, or draft scores to any AI system, whether or not the funder’s own policy names that system.”
    • State the confidentiality obligation independently of the AI-use clause. General-purpose AI (GPAI) providers regulated under the EU AI Act’s GPAI obligations, in force since August 2025, may process submitted inputs for model improvement unless expressly excluded, so agreements should require reviewers to confirm no proposal content has been shared with any third-party system, GPAI-regulated or not.
    • Require disclosure, not just prohibition. A short attestation line — “I have not used generative AI to draft, summarise or score any part of this review, and any AI assistance used was limited to language editing of my own original text” — gives research integrity offices an auditable record if a dispute arises.

    Where a funder (such as NHMRC from April 2026) permits limited AI-assisted editing, research offices should still require reviewers to disclose which tool was used and confirm no proposal content was entered into it. This keeps institutional practice defensible even where funder rules differ from one call to the next.

    Implications and outlook

    For institutions running multi-funder portfolios, the practical challenge is less about any single funder’s rule and more about reviewer confusion across simultaneous panels. A reviewer serving both an ERC panel and a UKRI-funded call in the same month operates under materially different AI permissions for the same underlying task. Research offices should treat funder AI policies as living documents — ERC’s and NHMRC’s 2026 updates both followed roughly a year or more after their organisations’ initial public positions on AI, suggesting further revision is likely as reviewer behaviour and AI capability both evolve.

    The direction of travel across all four funders is convergence on two non-negotiables — confidentiality of proposal content and non-delegation of scientific judgement — even as the permitted margin for administrative AI assistance slowly widens. Research offices that build reviewer agreements around those two constants, rather than around any single funder’s current wording, will need fewer rewrites as UKRI’s pending policy shift and any subsequent NIH or ERC revisions land through 2026 and beyond.

    For related terminology used across funder and publisher AI-governance documents, see the CASRAI research dictionary, and for broader institutional process guidance visit the research administration resource hub.