CASRAI Dictionary

Tag: ai safety institute

  • Foundation Model Taskforce: How It Became AISI

    The Foundation Model Taskforce was the UK government’s April 2023 vehicle for AI capability-building; it was renamed the Frontier AI Taskforce in September 2023, became the AI Safety Institute in November 2023, and was renamed again as the AI Security Institute in February 2025. Each rename shifted both remit and budget line, and each shift changed who research funders and universities need to engage with on AI policy.

    The Foundation Model Taskforce is the name the Department for Science, Innovation and Technology (DSIT) gave, in April 2023, to a government unit tasked with building UK capability in large-scale AI systems and evaluating their risks. It no longer exists under that name — but its institutional descendant, the AI Security Institute, still shapes the funding and evaluation landscape that UK research offices operate in today.

    What was the UK’s Foundation Model Taskforce?

    The Foundation Model Taskforce was announced on 24 April 2023, when DSIT and the Prime Minister’s Office committed an initial £100 million to what the government called “an expert taskforce to help the UK build and adopt the next generation of safe AI.” The taskforce was explicitly modelled on the Vaccine Taskforce’s fast-moving, mission-led structure rather than a standard civil-service directorate.

    Tech investor Ian Hogarth was appointed to chair the taskforce on 18 June 2023. Its founding brief covered three strands: building sovereign UK capability in foundation models, procuring pilots for public-service applications, and researching model safety. That third strand — safety research — is the only one that survived into the institution’s later names.

    Why did it become the Frontier AI Taskforce, then the AI Safety Institute?

    On 7 September 2023, the government renamed the unit the Frontier AI Taskforce, a change confirmed in a written statement to Parliament (HCWS1054, 19 September 2023). The rename narrowed the public framing from “building” foundation models to “evaluating” frontier ones — models at the most capable end of the spectrum.

    Two months later, at the AI Safety Summit hosted at Bletchley Park in November 2023, the Frontier AI Taskforce was formally established as a permanent body: the AI Safety Institute. Its stated mission, per its GOV.UK organisation page, was “to minimise surprise to the UK and humanity from rapid and unexpected advances in AI.” Ian Hogarth continued as chair. Capability-building and public-sector procurement — the taskforce’s original first two strands — were absorbed back into DSIT proper, leaving the Institute a narrower, research-and-evaluation-only mandate.

    The UK’s move was not isolated. At the AI Seoul Summit in May 2024, ten jurisdictions — the UK, the US, the EU, Japan, Singapore, South Korea, Canada, France, Kenya and Australia — signed the Seoul Statement of Intent, launching the International Network of AI Safety Institutes to align testing methodologies. The network’s first in-person meeting followed in San Francisco on 20–21 November 2024.

    Why did the AI Safety Institute become the AI Security Institute?

    In February 2025, Technology Secretary Peter Kyle announced at the Munich Security Conference that the AI Safety Institute would be renamed the AI Security Institute (AISI). The change was not cosmetic. The Institute’s remit was redrawn to focus on “serious AI risks with security implications” — chemical, biological and cyber misuse, and AI-enabled crime, including fraud and child sexual abuse material — and a new criminal-misuse research team was created within it.

    Notably, the rebranded Institute’s remit explicitly excludes bias and freedom-of-speech concerns, which had sat within the AI Safety Institute’s broader original scope. The US made a parallel move at almost the same moment: its NIST-housed AI Safety Institute became the Center for AI Standards and Innovation (CAISI), reflecting a wider Anglo-American pivot from precautionary “safety” framing toward narrower “security” framing.

    Name Established Chair / lead Core focus
    Foundation Model Taskforce April 2023 Ian Hogarth (from June 2023) Capability-building, procurement, safety research
    Frontier AI Taskforce September 2023 Ian Hogarth Evaluation of frontier-model risk
    AI Safety Institute November 2023 (Bletchley Park) Ian Hogarth Safety research and model evaluation
    AI Security Institute (AISI) February 2025 (Munich) DSIT-appointed leadership Cyber, chem-bio, and criminal-misuse risk; excludes bias/free-speech remit

    What does this lineage mean for UK research funders and universities?

    For research administration teams, the practical consequence of four names in under two years is that the government counterpart for AI-safety dialogue has changed shape repeatedly, and its funding role has narrowed each time. Three points matter for institutional engagement.

    • AISI is not the Alan Turing Institute. The two are frequently conflated in university communications. AISI is a DSIT research unit with a security-and-evaluation mandate; the Turing Institute is the UK’s separately constituted national institute for data science and AI, with a broader research-council relationship. Research offices briefing academic leadership should not treat the two as interchangeable stakeholders.
    • Funder co-delivery already exists. EPSRC and Innovate UK — both part of UK Research and Innovation — partnered with the AI Safety Institute to deliver the Systemic Safety Grants programme: an initial £4 million tranche (of £8.5 million earmarked across later phases), funding around 20 projects of up to £200,000 each, with proposals due 26 November 2024 and awards confirmed by February 2025. This is the clearest precedent for how AISI-adjacent funding reaches universities: through UKRI council co-delivery, not direct AISI grant lines.
    • AISI holds no statutory or regulatory power. Unlike the Information Commissioner’s Office or a licensing regulator, AISI cannot compel model access, issue penalties, or set binding standards — its leverage is voluntary pre-deployment testing agreements with frontier labs. Universities citing AISI in governance or ethics documentation should describe it accurately as an evaluation and research body, not a regulator.

    The direction of travel — from open-ended “safety” to bounded “security” — also signals where new funding calls are likely to concentrate: cyber-misuse and biosecurity-adjacent AI research, rather than the bias, fairness and societal-impact work that sat inside the Institute’s original remit. Research offices tracking DSIT and UKRI calls should expect that broader societal-AI-risk work will increasingly need to find funders outside the AISI lineage altogether, most likely through EPSRC’s standard responsive-mode routes or ESRC-led programmes.

    Common questions on the UK’s AI governance machinery

    Did the UK change the name of the AI Security Institute?

    Yes. The UK’s AI Safety Institute was renamed the AI Security Institute in February 2025, announced by Technology Secretary Peter Kyle at the Munich Security Conference. The change narrowed the Institute’s remit to security-relevant risks — cyber, chemical, biological and criminal misuse — while dropping bias and free-speech concerns from scope.

    Who funds the AI Safety Institute?

    AISI is funded through DSIT’s departmental budget, originally drawn from the £100 million committed to the Foundation Model Taskforce in April 2023. Some research activity is co-delivered with UK Research and Innovation councils, such as the EPSRC/Innovate UK-backed Systemic Safety Grants programme, rather than funded solely through AISI’s own budget line.

    Who is the head of the AI Safety Institute UK?

    Ian Hogarth, a technology investor and entrepreneur, chaired the organisation continuously from June 2023 through its Foundation Model Taskforce, Frontier AI Taskforce and AI Safety Institute phases. Its advisory board has included figures such as Turing Award laureate Yoshua Bengio, reflecting an international scientific-advisory model rather than a purely civil-service structure.

    What is the UK AI Security Institute?

    The AI Security Institute is a research organisation within DSIT that evaluates the security risks of advanced AI models, rather than a regulator. Per its own mission statement, it exists “to equip governments with a scientific understanding of the risks posed by advanced AI,” working through voluntary testing arrangements with frontier AI developers.

    Institutions building AI governance guidance should track AISI’s evolving remit alongside — not instead of — the Alan Turing Institute, UKRI’s research-council funding calls, and any forthcoming statutory AI legislation, since none of these bodies currently holds the full picture on its own. The machinery will very likely change name again; the underlying funding relationships, not the current label, are what research offices should document and monitor.

  • UK AI Safety Institute vs 4 Global Peers

    The UK AI Safety Institute — renamed the AI Security Institute (AISI) in 2025 — is a research directorate of the Department for Science, Innovation and Technology that evaluates frontier AI systems and funds external safety research, distinguishing it from the US CAISI’s standards focus, Japan’s non-R&D coordination role, Canada’s CIFAR-administered grants, and the EU AI Office’s regulatory enforcement mandate. For institutions weighing where to seek collaboration or funding for AI safety evaluation work, these differences in remit, funding scale, and academic-access routes are decisive.

    The UK AI Safety Institute is one node in a wider “International Network of AI Safety Institutes,” launched at the AI Seoul Summit in May 2024, bringing together technical bodies from the UK, US, Japan, Canada, the EU and other jurisdictions to coordinate — but not centralise — frontier AI risk assessment.

    What is the UK AI Safety Institute (AISI) today?

    The AI Security Institute is a directorate of the UK’s Department for Science, Innovation and Technology (DSIT), established in November 2023 following the Bletchley Park AI Safety Summit. Its mission, per GOV.UK, is “to minimise surprise to the UK and humanity from rapid and unexpected advances in AI.”

    A UK Parliament written statement of February 2025 confirmed the rebrand from “AI Safety Institute” to “AI Security Institute,” sharpening its focus on national-security-relevant risks such as cyber, chemical and biological misuse, alongside broader model evaluation work. The rename matters for researchers: many external directories still index the institute under its original name, which can misdirect funding enquiries.

    AISI holds pre-release testing access agreements with Anthropic, Google and OpenAI, and maintains Inspect, an open-source evaluation platform that lets companies, governments and academic teams run standardised AI safety tests without a bespoke agreement with AISI itself.

    How do the five institutes’ remits compare?

    All five bodies share a broad goal of understanding advanced-AI risk, but their statutory and operational remits diverge sharply — from hands-on evaluation to pure regulation.

    • UK AI Security Institute (AISI): evaluates frontier models, runs foundational safety research and grant programmes, and facilitates international information exchange.
    • US Center for AI Standards and Innovation (CAISI): sits inside NIST; focuses on testing, standards and national-security assessment. Renamed from “US AI Safety Institute” in 2025, mirroring the UK’s shift.
    • Japan AI Safety Institute (J-AISI): explicitly states it is not an R&D organisation; it consolidates evaluation methods and standards from industry and academia as a coordination hub.
    • Canada AI Safety Institute (CAISI): advances AI safety science with international partners, focused on synthetic-content risk and systems that could undermine human oversight.
    • EU AI Office: sits within the European Commission with an enforcement mandate — it supervises general-purpose AI models under the EU AI Act, the world’s first comprehensive statutory AI framework.

    Only the EU AI Office carries binding regulatory enforcement powers; the other four are advisory, evaluative and research-funding bodies without statutory power to compel compliance.

    Institute Parent body Core remit Direct academic funding
    UK AI Security Institute DSIT Evaluation, foundational safety research Yes — grants £50,000–£500,000
    US CAISI NIST / Dept. of Commerce Standards, national-security testing Limited — collaborative, not grant-led
    Japan J-AISI Government-affiliated hub Coordination, standards consolidation No — information-sharing role only
    Canada CAISI Innovation, Science and Economic Development Canada Safety science, synthetic-content risk Yes — via CIFAR-administered CAISI Research Program
    EU AI Office European Commission AI Act enforcement, GPAI supervision Indirect — via Horizon Europe / Digital Europe

    How is each institute funded?

    Funding scale is the clearest differentiator for institutions assessing where a grant application or evaluation partnership is likeliest to land.

    The UK AISI traces its funding to a £100 million initial investment behind the Frontier AI Taskforce, its 2023 predecessor body, and now runs the Alignment Project, a global research fund backed by more than £15 million. Grants under its Challenge Fund and Systemic AI Safety Grants typically range from £50,000 to £500,000 per award, open to UK and international applicants.

    The US CAISI operates on a comparatively modest footing: an initial budget of roughly $10 million for the 2024/25 fiscal year, with legislative proposals since floated to raise annual funding into the $67–155 million range — proposals, not yet appropriated funding.

    Canada’s AI Safety Institute is funded at CA$50 million over five years, of which CA$27 million has been channelled to the Canadian Institute for Advanced Research (CIFAR) to run the CAISI Research Program. Canada has also committed CA$1 million to the UK’s Alignment Project through CIFAR — a direct funding link institutions can leverage for joint bids.

    Japan’s J-AISI has not published a standalone budget, consistent with its coordination-only remit rather than direct grant-making. The EU AI Office likewise discloses no ring-fenced budget of its own; EU AI research funding flows through Horizon Europe and the Digital Europe Programme, together worth well over €1 billion annually, of which only a fraction is Office-directed academic work.

    How does each body engage external academic researchers?

    Engagement models range from direct grant-making to purely consultative input, which changes what “collaboration” actually means for a university research office.

    • UK AISI: direct grants to academic institutions and non-profits in the UK and internationally through the Challenge Fund, Systemic AI Safety Grants and the Alignment Project.
    • US CAISI: collaborative research relationships with universities to develop guidelines and voluntary standards, rather than large competitive grant rounds.
    • Japan J-AISI: partnership and information-sharing with academia and industry, consolidating findings rather than commissioning new funded research.
    • Canada CAISI: funding via CIFAR’s Catalyst Projects and Solution Networks, with awards up to CA$70,000 per year for up to two years, plus ties to Canada’s three national AI institutes — Amii, Mila and the Vector Institute.
    • EU AI Office: consultative input via the AI Board and a scientific panel of independent experts shaping codes of practice for general-purpose AI models, rather than a competitive grants pipeline.

    For a research administration office, this means the UK and Canadian institutes are the two realistic direct-funding routes today; the US, Japanese and EU bodies are better approached as standards-setting or advisory partners than as grant sources.

    Common questions

    Is the UK AI Safety Institute still called that?

    No. The UK AI Safety Institute was renamed the AI Security Institute in 2025, confirmed in a UK Parliament written statement, and the US counterpart was simultaneously renamed the Center for AI Standards and Innovation (CAISI). Both retain their original evaluation and research functions under the new names.

    What is the International Network of AI Safety Institutes?

    It is a coordination body launched at the AI Seoul Summit in May 2024, joining institutes from the UK, US, Japan, Canada, the EU and other governments. Its first formal meeting took place in November 2024, and it exists to align evaluation methods, not to centralise funding or enforcement power.

    How can an academic team apply for UK AISI funding?

    UK-based and international researchers can apply through AISI’s Challenge Fund or Systemic AI Safety Grants, with typical awards between £50,000 and £500,000, or through the cross-national Alignment Project, which pools UK and partner-government contributions, including Canada’s CA$1 million pledge via CIFAR.

    Does the EU AI Office fund academic AI safety research directly?

    Not directly. The EU AI Office is primarily a regulatory and enforcement body for the EU AI Act; academic AI research funding in the EU runs through Horizon Europe and the Digital Europe Programme, with the Office instead offering academics a consultative seat via its scientific panel and the AI Board.

    What this means for institutions seeking partnerships

    Research administration offices scoping AI safety evaluation collaborations should match their proposal to the right model rather than assuming one “AI Safety Institute” template applies globally. A UK or Canadian bid should target a named grant scheme with a defined award range; a US, Japanese or EU approach should be framed as standards-development or advisory input instead.

    Because only the UK and Canadian institutes run competitive, named academic grant programmes — and already share a funding link through CIFAR and the Alignment Project — joint UK–Canada bids are, as of mid-2026, the most concrete route into public frontier-AI-safety funding for external academic groups. The EU AI Office’s enforcement powers will likely reshape this landscape as AI Act obligations mature, but its funding role stays structurally indirect for now. Institutions should track each institute’s funding cycle separately rather than treat the international network as one funding body.