Tag: ai safety institute network

  • UK AI Safety Institute vs 4 Global Peers

    The UK AI Safety Institute — renamed the AI Security Institute (AISI) in 2025 — is a research directorate of the Department for Science, Innovation and Technology that evaluates frontier AI systems and funds external safety research, distinguishing it from the US CAISI’s standards focus, Japan’s non-R&D coordination role, Canada’s CIFAR-administered grants, and the EU AI Office’s regulatory enforcement mandate. For institutions weighing where to seek collaboration or funding for AI safety evaluation work, these differences in remit, funding scale, and academic-access routes are decisive.

    The UK AI Safety Institute is one node in a wider “International Network of AI Safety Institutes,” launched at the AI Seoul Summit in May 2024, bringing together technical bodies from the UK, US, Japan, Canada, the EU and other jurisdictions to coordinate — but not centralise — frontier AI risk assessment.

    What is the UK AI Safety Institute (AISI) today?

    The AI Security Institute is a directorate of the UK’s Department for Science, Innovation and Technology (DSIT), established in November 2023 following the Bletchley Park AI Safety Summit. Its mission, per GOV.UK, is “to minimise surprise to the UK and humanity from rapid and unexpected advances in AI.”

    A UK Parliament written statement of February 2025 confirmed the rebrand from “AI Safety Institute” to “AI Security Institute,” sharpening its focus on national-security-relevant risks such as cyber, chemical and biological misuse, alongside broader model evaluation work. The rename matters for researchers: many external directories still index the institute under its original name, which can misdirect funding enquiries.

    AISI holds pre-release testing access agreements with Anthropic, Google and OpenAI, and maintains Inspect, an open-source evaluation platform that lets companies, governments and academic teams run standardised AI safety tests without a bespoke agreement with AISI itself.

    How do the five institutes’ remits compare?

    All five bodies share a broad goal of understanding advanced-AI risk, but their statutory and operational remits diverge sharply — from hands-on evaluation to pure regulation.

    • UK AI Security Institute (AISI): evaluates frontier models, runs foundational safety research and grant programmes, and facilitates international information exchange.
    • US Center for AI Standards and Innovation (CAISI): sits inside NIST; focuses on testing, standards and national-security assessment. Renamed from “US AI Safety Institute” in 2025, mirroring the UK’s shift.
    • Japan AI Safety Institute (J-AISI): explicitly states it is not an R&D organisation; it consolidates evaluation methods and standards from industry and academia as a coordination hub.
    • Canada AI Safety Institute (CAISI): advances AI safety science with international partners, focused on synthetic-content risk and systems that could undermine human oversight.
    • EU AI Office: sits within the European Commission with an enforcement mandate — it supervises general-purpose AI models under the EU AI Act, the world’s first comprehensive statutory AI framework.

    Only the EU AI Office carries binding regulatory enforcement powers; the other four are advisory, evaluative and research-funding bodies without statutory power to compel compliance.

    Institute Parent body Core remit Direct academic funding
    UK AI Security Institute DSIT Evaluation, foundational safety research Yes — grants £50,000–£500,000
    US CAISI NIST / Dept. of Commerce Standards, national-security testing Limited — collaborative, not grant-led
    Japan J-AISI Government-affiliated hub Coordination, standards consolidation No — information-sharing role only
    Canada CAISI Innovation, Science and Economic Development Canada Safety science, synthetic-content risk Yes — via CIFAR-administered CAISI Research Program
    EU AI Office European Commission AI Act enforcement, GPAI supervision Indirect — via Horizon Europe / Digital Europe

    How is each institute funded?

    Funding scale is the clearest differentiator for institutions assessing where a grant application or evaluation partnership is likeliest to land.

    The UK AISI traces its funding to a £100 million initial investment behind the Frontier AI Taskforce, its 2023 predecessor body, and now runs the Alignment Project, a global research fund backed by more than £15 million. Grants under its Challenge Fund and Systemic AI Safety Grants typically range from £50,000 to £500,000 per award, open to UK and international applicants.

    The US CAISI operates on a comparatively modest footing: an initial budget of roughly $10 million for the 2024/25 fiscal year, with legislative proposals since floated to raise annual funding into the $67–155 million range — proposals, not yet appropriated funding.

    Canada’s AI Safety Institute is funded at CA$50 million over five years, of which CA$27 million has been channelled to the Canadian Institute for Advanced Research (CIFAR) to run the CAISI Research Program. Canada has also committed CA$1 million to the UK’s Alignment Project through CIFAR — a direct funding link institutions can leverage for joint bids.

    Japan’s J-AISI has not published a standalone budget, consistent with its coordination-only remit rather than direct grant-making. The EU AI Office likewise discloses no ring-fenced budget of its own; EU AI research funding flows through Horizon Europe and the Digital Europe Programme, together worth well over €1 billion annually, of which only a fraction is Office-directed academic work.

    How does each body engage external academic researchers?

    Engagement models range from direct grant-making to purely consultative input, which changes what “collaboration” actually means for a university research office.

    • UK AISI: direct grants to academic institutions and non-profits in the UK and internationally through the Challenge Fund, Systemic AI Safety Grants and the Alignment Project.
    • US CAISI: collaborative research relationships with universities to develop guidelines and voluntary standards, rather than large competitive grant rounds.
    • Japan J-AISI: partnership and information-sharing with academia and industry, consolidating findings rather than commissioning new funded research.
    • Canada CAISI: funding via CIFAR’s Catalyst Projects and Solution Networks, with awards up to CA$70,000 per year for up to two years, plus ties to Canada’s three national AI institutes — Amii, Mila and the Vector Institute.
    • EU AI Office: consultative input via the AI Board and a scientific panel of independent experts shaping codes of practice for general-purpose AI models, rather than a competitive grants pipeline.

    For a research administration office, this means the UK and Canadian institutes are the two realistic direct-funding routes today; the US, Japanese and EU bodies are better approached as standards-setting or advisory partners than as grant sources.

    Common questions

    Is the UK AI Safety Institute still called that?

    No. The UK AI Safety Institute was renamed the AI Security Institute in 2025, confirmed in a UK Parliament written statement, and the US counterpart was simultaneously renamed the Center for AI Standards and Innovation (CAISI). Both retain their original evaluation and research functions under the new names.

    What is the International Network of AI Safety Institutes?

    It is a coordination body launched at the AI Seoul Summit in May 2024, joining institutes from the UK, US, Japan, Canada, the EU and other governments. Its first formal meeting took place in November 2024, and it exists to align evaluation methods, not to centralise funding or enforcement power.

    How can an academic team apply for UK AISI funding?

    UK-based and international researchers can apply through AISI’s Challenge Fund or Systemic AI Safety Grants, with typical awards between £50,000 and £500,000, or through the cross-national Alignment Project, which pools UK and partner-government contributions, including Canada’s CA$1 million pledge via CIFAR.

    Does the EU AI Office fund academic AI safety research directly?

    Not directly. The EU AI Office is primarily a regulatory and enforcement body for the EU AI Act; academic AI research funding in the EU runs through Horizon Europe and the Digital Europe Programme, with the Office instead offering academics a consultative seat via its scientific panel and the AI Board.

    What this means for institutions seeking partnerships

    Research administration offices scoping AI safety evaluation collaborations should match their proposal to the right model rather than assuming one “AI Safety Institute” template applies globally. A UK or Canadian bid should target a named grant scheme with a defined award range; a US, Japanese or EU approach should be framed as standards-development or advisory input instead.

    Because only the UK and Canadian institutes run competitive, named academic grant programmes — and already share a funding link through CIFAR and the Alignment Project — joint UK–Canada bids are, as of mid-2026, the most concrete route into public frontier-AI-safety funding for external academic groups. The EU AI Office’s enforcement powers will likely reshape this landscape as AI Act obligations mature, but its funding role stays structurally indirect for now. Institutions should track each institute’s funding cycle separately rather than treat the international network as one funding body.

  • AI Safety Institute Network: How Cross-Border Testing Works

    The AI safety institute network is a group of ten government-backed bodies — currently Australia, Canada, the EU, France, Japan, Kenya, the Republic of Korea, Singapore, the UK and the US — that coordinate technical methods for evaluating advanced AI models before and after deployment. It is not one institute; it is a forum for shared testing standards, joint exercises and information-sharing between national evaluators, formally renamed in February 2026 to reflect a wider science-of-evaluation mandate.

    The AI safety institute network traces to the May 2024 Seoul AI Summit, where the UK, US, Japan, Singapore, the EU and several other governments agreed to establish an international forum for AI safety science. Since then it has run joint model-testing pilots, absorbed a new member (Kenya), and changed its own name — developments that most explainer content published in 2024–25 has not caught up with.

    What is the AI safety institute network?

    The network is a technical forum, not a regulator. Each participating jurisdiction runs its own domestic AI safety or security institute — a government office, typically sitting inside a science or digital ministry — and the network is the channel through which those offices align methods rather than rules.

    As of February 2026 its formal name is the International Network for Advanced AI Measurement, Evaluation and Science, having operated since its November 2024 founding as the “International Network of AI Safety Institutes.” The rename, announced on the UK AI Security Institute’s blog, signals a shift from a safety-branding forum toward a standing scientific body focused on measurement methodology.

    • Established: November 2024, following the Seoul Statement of Intent signed at the May 2024 AI Seoul Summit
    • Current members: Australia, Canada, the European Union, France, Japan, Kenya, the Republic of Korea, Singapore, the United Kingdom and the United States
    • 2026 Network Coordinator: the United Kingdom, via the AI Security Institute (AISI)
    • Mandate: build internationally recognised methods for measuring and evaluating advanced AI capabilities and risks

    How does cross-border AI model testing actually work?

    Cross-border testing runs on bilateral agreements and joint exercises rather than a single binding treaty. The clearest example is the UK–US Memorandum of Understanding signed on 1 April 2024, under which the two institutes committed to a shared approach to model evaluations, at least one joint test of a publicly accessible model, technical-research collaboration and staff exchanges.

    Member institutes have already run several joint exercises. A pilot evaluation of an open foundation model tested interoperability between two toolkits — the UK’s open-source Inspect framework and Singapore’s Moonshot platform — checking whether results from different infrastructure could be compared directly.

    • A multilingual evaluation led jointly by Singapore, Japan and the UK translated and validated benchmark questions into ten languages to test privacy, crime, intellectual-property and jailbreak-resistance risks across languages
    • A joint exercise on “agentic” AI systems examined risks including sensitive-information leakage, fraud and cybersecurity in models capable of autonomous planning and task execution
    • Information-sharing covers model limitations, capabilities and documented safety incidents, subject to each institute’s domestic law

    No member institute has ceded sovereign testing authority to the network. Each government still decides what to evaluate and publish; the network’s value is methodological convergence, not enforcement.

    What changed when the network renamed itself in 2026?

    At a technical meeting held alongside the NeurIPS conference in San Diego in late 2025, members reported growing consensus on evaluation principles and surfaced open methodological questions in a February 2026 blog post — evidence the network has moved from launch-phase diplomacy to a working scientific programme.

    Consensus areas reported include: evaluations must state clear objectives before they run; results must be transparent and reproducible; testing parameters should match the evaluation’s purpose (best-case elicitation versus realistic use); and reports should separate individual tests and stay legible to non-specialist policymakers.

    Unresolved questions include how much methodological detail evaluators can publish without enabling replication of dangerous capabilities, how flexible reporting templates should be, and how to evaluate deployed AI systems — not just underlying models. The network’s next meeting is at the India AI Impact Summit, where members plan to test approaches against real-world use cases.

    How do the national institutes compare?

    Funding, remit and regulatory posture vary sharply between members — “the AI safety institute” is a misleading singular, since at least six materially different institutional models operate under one network.

    Jurisdiction Institute Parent body Reported funding Primary posture
    United Kingdom AI Security Institute (AISI, formerly AI Safety Institute) Department for Science, Innovation and Technology £100 million secured until 2030 Voluntary evaluations, R&D, 2026 Network Coordinator
    United States Center for AI Standards and Innovation (CAISI, formerly US AISI) NIST $10 million for FY2024/25 Voluntary evaluations and standards R&D
    European Union EU AI Office DG Connect €46.5 million setup funding Binding regulation — enforces the EU AI Act
    Singapore Digital Trust Centre / AI Verify Foundation IMDA $37 million setup funding Testing tooling and content-assurance standards
    Japan Japan AI Safety Institute IPA Not separately disclosed Evaluation methodology and standards research
    Canada Canadian AI Safety Institute ISED Not separately disclosed R&D and international coordination

    The EU is the outlier: its AI Office is the only member with statutory enforcement power, administering the EU AI Act’s obligations on providers of general-purpose models with systemic risk. Every other member’s work is voluntary — which is why joint testing protocols, not legislation, are the network’s main coordination tool.

    Where can academic researchers plug into pre-deployment evaluation work?

    Pre-deployment evaluation is not a closed government function. Several member institutes have built explicit, funded routes for academic researchers rather than treating testing as an internal civil-service exercise.

    • Open evaluation tooling: the UK AISI’s Inspect framework is published as open-source software, so university research groups can build and run their own model evaluations using the same infrastructure member institutes use, rather than reverse-engineering proprietary test harnesses
    • Grant-funded safety research: AISI’s Systemic Safety Grants programme has funded academic work on systemic AI risks, alongside The Alignment Project — a multi-funder grants and compute programme backed by AISI, UKRI, Schmidt Sciences and several frontier AI developers supporting alignment research beyond government labs
    • Conference-linked workshops: the network’s 2025 technical meeting was held alongside NeurIPS to exchange methods with academic and industry researchers, not as a closed intergovernmental session
    • Multilingual and cross-cultural evaluation gaps: the network has flagged that evaluation science for non-English, non-Western contexts is underdeveloped — a direct, citable entry point for university linguistics and computational social science groups

    For institutional leaders, the practical implication is that AI safety evaluation now sits alongside conventional funder compliance work: grant terms from bodies co-funding alignment research carry reporting and reproducibility expectations that research administration teams already apply to other externally funded programmes.

    Answer-first questions on the AI safety institute network

    Which countries have AI safety institutes?

    Ten members currently participate in the International Network for Advanced AI Measurement, Evaluation and Science: Australia, Canada, the European Union, France, Japan, Kenya, the Republic of Korea, Singapore, the United Kingdom and the United States, each running its own domestic evaluation body.

    Did the AI Safety Institute change its name?

    Yes. In 2025 the UK’s AI Safety Institute was renamed the AI Security Institute, and its US counterpart became the Center for AI Standards and Innovation (CAISI). In February 2026 the international network itself was renamed from the “International Network of AI Safety Institutes” to the “International Network for Advanced AI Measurement, Evaluation and Science.”

    Are the network’s testing protocols legally binding?

    No single binding treaty governs the whole network. Cooperation runs through bilateral instruments such as the UK–US Memorandum of Understanding and voluntary joint exercises; the EU AI Office is the sole member with statutory enforcement power, via the EU AI Act.

    Who coordinates the network in 2026?

    The United Kingdom, through its AI Security Institute, holds the 2026 Network Coordinator role and has committed to converting the group’s evaluation-science consensus into detailed best-practice documentation before the network’s next meeting at the India AI Impact Summit.

    What happens next

    The network’s trajectory — from a 2024 diplomatic statement to a 2026 renamed scientific body with a rotating coordinator and a published consensus-and-open-questions agenda — suggests consolidation around evaluation methodology rather than a drift into national silos. Kenya’s addition also signals a deliberate widening beyond the original G7-plus-Singapore core.

    For universities, funders and publishers tracking AI governance, the signal to watch is the best-practice documentation the UK has committed to during its 2026 coordinator term, and whether evaluating deployed AI systems — not just underlying models — gets resolved before the India AI Impact Summit.