CASRAI Dictionary

Tag: AI in higher education UK

  • AI Chatbot Legal Liability: What the Character.AI and OpenAI Lawsuits Mean for University Duty of Care

    A wave of wrongful-death lawsuits against Character.AI and OpenAI, a landmark Canadian tribunal ruling against an airline chatbot, and a new UK legal statement on AI harms have together turned AI chatbot legal liability from an abstract compliance question into an active, evolving body of case law. As universities roll out AI chatbots for admissions queries, academic advising, and student wellbeing support, the same legal theories now being tested against consumer AI companies — product liability, negligence, and misrepresentation — could increasingly reach institutions themselves. This analysis unpacks what the litigation actually establishes and what it signals for duty-of-care policy in UK and international higher education.

    How AI chatbot legal liability is currently assessed

    No jurisdiction has yet enacted a bespoke statute governing chatbot harm. Instead, courts and regulators are applying existing doctrines — product liability, negligence, negligent misrepresentation, defamation, and vicarious liability — to AI outputs. In January 2026 the UK Jurisdiction Taskforce (UKJT) published a draft legal statement on liability for AI harms, opened for consultation until 13 February 2026, which confirmed a foundational point: AI systems have no legal personality in English law, so liability always attaches to the humans and organisations that design, deploy, or operate them.

    The UKJT statement flagged several routes to liability that are directly relevant to institutions:

    • Negligence — liability generally requires proof of a duty of care, breach, causation, and foreseeable harm, though the “opacity” of AI decision-making can make causation harder to establish.
    • Product liability — the UK’s Consumer Protection Act 1987 imposes no-fault liability for defective products causing physical harm; how it applies to software and AI is untested, and the Law Commission is consulting on reform. The EU’s revised product liability regime, in force from December 2024, explicitly extends to AI software providers.
    • Negligent misrepresentation — a false or misleading statement from a chatbot can itself found a claim, as the 2024 Moffatt v Air Canada tribunal ruling showed when Air Canada was held liable for its chatbot’s incorrect bereavement-fare advice.
    • Vicarious liability — an employee’s negligent use of AI can make an employer liable even where the employer itself did nothing wrong.

    Contracts matter enormously here: the UKJT noted that warranty, indemnity, and limitation-of-liability clauses in vendor agreements will often determine who actually bears the cost of an AI-related harm — a point that should shape how universities and research institutions negotiate chatbot procurement contracts, not just their public-facing policies.

    The Character.AI and OpenAI litigation: what happened

    The clearest illustration of these theories in practice comes from a cluster of US wrongful-death suits. Megan Garcia’s son, Sewell Setzer III, died by suicide in 2024 after prolonged interaction with a Character.AI companion bot; her Senate Judiciary Committee testimony in September 2025 became a focal point for subsequent litigation and state action. A comparable case, Raine v. OpenAI, alleges that ChatGPT reinforced a 16-year-old’s suicidal ideation. Both cases argue the chatbot was a defective product and that the developer was negligent in releasing it without adequate safeguards.

    Case / actor Legal theory Core allegation Status (mid-2026)
    Garcia v. Character Technologies / Google Wrongful death, product liability, negligence Chatbot fostered dependency and failed to intervene despite expressed suicidal ideation Character.AI and Google reportedly agreed to settle five related suits, January 2026 (terms undisclosed)
    Raine v. OpenAI Wrongful death, defective design, negligence ChatGPT allegedly reinforced suicidal ideation and provided method-related information Litigation ongoing
    Pennsylvania v. Character.AI State consumer-protection claim Chatbot falsely claimed to be a licensed Pennsylvania therapist with a fabricated licence number Filed by state Attorney General, May 2026
    Kentucky v. [AI chatbot company] State consumer-protection claim First US state action alleging predatory chatbot design directed at minors Filed by state Attorney General, January 2026
    Moffatt v. Air Canada (Canada) Negligent misrepresentation Airline chatbot gave incorrect bereavement-fare policy information relied on by a customer Tribunal found against the airline (persuasive precedent, 2024)

    Two features of this litigation matter beyond the individual cases. First, chatbot transcripts have become the central evidentiary record — logged conversations, not marketing claims, are what plaintiffs’ lawyers and regulators are relying on. Second, state attorneys general are now bringing consumer-protection actions independently of civil plaintiffs, widening the range of parties who can trigger liability exposure for an organisation running a chatbot.

    What this means for university duty-of-care policy

    Universities are not named defendants in the Character.AI or OpenAI cases, but the underlying theories transfer directly to institutional deployments. A UK sector survey published via Jisc in January 2026 found more than one in three adults report having used an AI chatbot for mental-health or wellbeing support — a demand pattern that is pulling universities toward deploying similar tools for pastoral care, often without the safety infrastructure a dedicated consumer AI company has had to build under litigation pressure. Times Higher Education and specialist education-law advisers have both warned in 2026 that AI tools should support, not impersonate, student services staff, and that institutions should audit existing and planned chatbot use for exactly this reason.

    Do universities have a duty of care for students?

    UK universities do not owe students a blanket duty of care in the way schools owe pupils, but courts have found specific duties can arise in negligence, contract, and consumer-protection law — particularly where an institution knows a student is vulnerable or operates a support service, including an AI chatbot, that a student reasonably relies on.

    What is the duty of care in AI?

    No AI system has legal personality, so any duty of care for AI-related harm attaches to the people and organisations that design, deploy, or operate it. The UKJT’s 2026 draft statement confirms liability generally requires proving negligence, foreseeability, and causation against a human or corporate defendant, not the AI itself.

    Can AI be held legally accountable?

    No. AI systems cannot be sued or held liable directly under English law because they lack legal personality. Legal accountability instead falls on the developer, deployer, or operator through product liability, negligence, or misrepresentation claims — the same theories used in the Character.AI, OpenAI, and Air Canada chatbot cases.

    Can AI chats be used against you in court?

    Yes. Chatbot transcripts are typically discoverable evidence in litigation, as seen in the Character.AI and OpenAI wrongful-death suits, where logged conversations formed the core evidentiary record. Institutions deploying chatbots should treat transcripts as records subject to retention, data-protection, and disclosure obligations, not disposable interaction data.

    Building an AI chatbot governance policy: practical steps

    Institutional risk teams, general counsel, and research administrators evaluating a chatbot deployment — for student wellbeing, academic advising, or interactions with research participants — should treat the litigation above as a checklist of failure modes to design against, not a distant industry problem:

    • Maintain a human-in-the-loop escalation pathway for any wellbeing- or mental-health-adjacent chatbot interaction, rather than relying on the bot to self-detect crisis language.
    • Vet vendor contracts for warranty, indemnity, and limitation-of-liability clauses; per the UKJT statement, these terms — not just internal policy — will often determine who bears the cost of an AI-related harm.
    • Log and retain chatbot transcripts in line with data-protection obligations, on the assumption they are discoverable evidence, not disposable interaction data.
    • Publish clear, prominent disclaimers distinguishing pastoral-support or advisory chatbots from clinical, counselling, or legal services — the Pennsylvania Character.AI action turned specifically on a chatbot misrepresenting its professional status.
    • Route AI-related incidents into existing safeguarding and student-support escalation channels, rather than treating them as a separate IT ticket category.
    • Check whether any chatbot function — assessing learning outcomes, monitoring exam behaviour, or screening admissions — falls within the EU AI Act’s Annex III “high-risk” education category, which covers systems used to determine access, evaluate learning outcomes, or detect prohibited behaviour during assessments; the AI Act’s scope is defined by function, not by the “chatbot” label.
    • Extend the same governance rigour to chatbots used with research participants as with students, since equivalent duty-of-care and informed-consent obligations apply — a point relevant to the broader research administration governance remit, not just student services.

    Looking ahead

    The regulatory picture is still forming. The UKJT’s consultation on its draft liability statement closed in February 2026; a finalised version, and any resulting judicial or legislative reform of the Consumer Protection Act 1987, remains pending. In the US, the Character.AI and Google settlement terms are undisclosed, so the litigation has not yet produced a binding precedent on the scope of chatbot-maker liability — but the volume of parallel state and civil actions makes it likely that clearer legal boundaries, and correspondingly clearer expectations for institutional deployers, will emerge within the next reporting cycle. Universities that treat duty-of-care review as a standing governance function now, rather than a reactive response to the next lawsuit, will be better placed for whatever those boundaries turn out to be.

  • UNESCO Recommendation on the Ethics of Artificial Intelligence: A Practical Guide for Research Offices

    When UNESCO’s 193 member states adopted the UNESCO Recommendation on the Ethics of Artificial Intelligence in November 2021, they created the first global standard-setting instrument on AI ethics — a non-binding but politically significant commitment that now shapes how governments, funders, and universities frame AI governance. For UK research offices navigating a fast-moving 2025 landscape of generative AI in teaching, assessment, and research integrity, the Recommendation functions less as law and more as reference architecture: a shared vocabulary of values, principles, and assessment tools that institutional AI ethics committees can adopt directly. This guide sets out what states actually committed to, how the UK’s 2025 sector guidance on generative AI in higher education sits underneath it, and a practical checklist for putting the framework to work.

    What the Recommendation actually commits states to

    The Recommendation on the Ethics of Artificial Intelligence was adopted by consensus at UNESCO’s 41st General Conference in November 2021. Because it is a “recommendation” rather than a “convention” under UNESCO’s constitutional instruments, it does not create binding treaty obligations. Instead, member states — including the UK — accept a political commitment to report periodically on implementation and to translate the framework into domestic law, sector guidance, and institutional policy.

    UNESCO backs this with three implementation mechanisms that research offices should know by name:

    • The Global AI Ethics and Governance Observatory, a public resource tracking national AI readiness and policy.
    • The Readiness Assessment Methodology (RAM), used by governments to benchmark institutional and legal preparedness for ethical AI governance.
    • The Ethical Impact Assessment (EIA), a procedural tool for identifying and mitigating the human-rights and environmental risks of a specific AI system before deployment.

    None of these tools are mandatory for individual universities. But because national governments are expected to operationalise them, they increasingly surface indirectly — through funder terms, procurement frameworks, and research-integrity codes that reference UNESCO’s language of proportionality, transparency, and human oversight.

    The four values and ten principles

    The Recommendation is built on four foundational values, each translated into operational principles that give research administrators a concrete checklist rather than an abstract statement of intent.

    Value What it means for a research office
    Human rights and human dignity AI tools used in admissions, peer review, or research assessment must not override due process or discriminate against protected groups.
    Peaceful, just and interconnected societies International collaboration and data-sharing agreements should respect national sovereignty and diverse legal frameworks.
    Diversity and inclusiveness AI benefits and risks in research infrastructure should be distributed equitably across disciplines, career stages, and institution types.
    Environment and ecosystem flourishing Procurement decisions for compute-intensive AI research tools should weigh carbon and energy costs, not only capability.

    These values are operationalised through ten principles: proportionality and do no harm; safety and security; privacy and data protection; multi-stakeholder and adaptive governance; responsibility and accountability; transparency and explainability; human oversight and determination; sustainability; awareness and literacy; and fairness and non-discrimination. Ethics committees drafting or reviewing an institutional AI policy can map each clause of that policy directly onto one of these ten principles to check for gaps.

    The 2025 UK picture: generative AI in education and research

    The UK, as a UNESCO member state, does not have a standalone statute implementing the Recommendation. Instead, its principles surface across a cluster of UK sector guidance that has matured significantly since 2023, with updated 2025 iterations addressing generative AI specifically.

    Body Guidance Primary relevance
    Department for Education Generative AI in education policy position, revised through 2025 Safeguarding, safety expectations, and sector-wide product standards
    Russell Group Principles on the use of generative AI tools in education (2023, updated) Academic integrity, staff and student AI literacy
    QAA Guidance for UK higher education providers on generative AI Assessment design and integrity in a generative-AI context
    JISC National baseline surveys and guidance on AI in tertiary education Sector-wide adoption tracking and practical toolkits
    UKRI Positions on AI use in funding applications and peer review Research integrity in grant assessment and reviewer conduct

    None of these UK instruments cite UNESCO’s Recommendation as a formal legal source. But the substantive overlap is close: Russell Group and QAA guidance on transparency in AI-assisted work mirrors principle six (transparency and explainability); UKRI’s expectations around reviewer accountability mirror principle five (responsibility and accountability); and DfE safeguarding provisions mirror the Recommendation’s proportionality and do-no-harm principle. For a research office, the practical implication is that UNESCO’s framework offers the common vocabulary that lets institutions reconcile these separately issued, sector-specific instruments into one coherent AI governance policy rather than several overlapping ones.

    Common questions on the UNESCO AI ethics Recommendation

    Is the UNESCO Recommendation on the Ethics of Artificial Intelligence legally binding?

    No — as a UNESCO Recommendation rather than a Convention, it is not legally binding on the 193 member states that adopted it in November 2021. States are politically committed to submit periodic implementation reports and to adapt the framework through domestic law, institutional policy, and the Readiness Assessment Methodology.

    What are the four core values of the UNESCO AI ethics Recommendation?

    The Recommendation rests on four values: respecting human rights and human dignity, fostering peaceful and interconnected societies, ensuring diversity and inclusiveness, and supporting environmental and ecosystem flourishing. Ten operational principles, spanning transparency, accountability, proportionality, and human oversight, translate these values into concrete institutional practice for research offices.

    What is UNESCO’s Ethical Impact Assessment tool?

    The Ethical Impact Assessment (EIA) is a structured procedure UNESCO developed to help institutions identify, weigh, and mitigate the human-rights, environmental, and social risks of an AI system before and during deployment. Research offices can adapt the EIA template for grant, procurement, and research-tool sign-off processes.

    The Recommendation supplies the underlying values and principles; UK sector bodies, including the Department for Education, the Russell Group, QAA, and JISC, translate them into practical 2025 guidance on assessment integrity, safeguarding, and the responsible adoption of generative AI across teaching, research, and research administration.

    How institutional AI ethics committees should use it

    An institutional AI ethics committee does not need to treat the Recommendation as a document to comply with line by line. It is more useful as a diagnostic framework for auditing existing policy and closing gaps. A practical sequence:

    1. Map every significant AI use case across the research lifecycle — grant triage, peer review support, research-data processing, plagiarism and integrity checks, and public engagement.
    2. Run an Ethical Impact Assessment, adapted from UNESCO’s EIA methodology, for each use case that touches personal data, funding decisions, or assessment outcomes.
    3. Assign a named human-oversight owner for each AI system, consistent with principle seven (human oversight and determination), so no automated output is treated as final without human review.
    4. Publish a short transparency statement disclosing where and how generative AI is used in institutional processes, satisfying principle six.
    5. Cross-reference the committee’s own policy against current Russell Group, QAA, and DfE guidance at least annually, since UK sector positions on generative AI are still being revised.
    6. Record decisions and rationale for auditability — the same accountability logic that underpins principle five.

    Research administration teams drafting these policies may also find it useful to align terminology with the research administration pillar and to cross-check definitions of related governance terms in the CASRAI Dictionary when drafting institutional glossaries for AI policy documents.

    What comes next

    UNESCO’s Recommendation was never designed to be self-executing; its value lies in giving 193 states — and, by extension, their universities and funders — a common ethical baseline to build from. In the UK, that baseline is increasingly visible not as a single “AI ethics law” but as a patchwork of DfE, Russell Group, QAA, JISC, and UKRI guidance that is still being updated as generative AI capabilities evolve through 2025 and beyond. Institutional AI ethics committees that map their own policies against UNESCO’s four values and ten principles now will be better placed to absorb whatever the next round of UK sector guidance requires, rather than rebuilding their governance framework from scratch each time a new instrument is published.

  • AI Growth Lab: What the UK’s Regulatory Sandbox Means for University-Led AI Research

    The AI Growth Lab is the UK government’s proposal for a cross-economy regulatory sandbox that lets firms and, potentially, universities trial AI-enabled products under supervised, time-limited exemptions from rules that would otherwise block deployment. The Department for Science, Innovation and Technology (DSIT) ran a call for evidence on the proposal from 21 October 2025 to 7 January 2026, and an advisory version of the Lab launched on 8 June 2026 with legal services as the first live sector. For research offices, the question is no longer whether the Lab will exist, but how sandbox pilots intersect with university spinouts, clinical AI trials, and research infrastructure such as the AI Research Resource.

    What Is the UK AI Growth Lab?

    DSIT describes the AI Growth Lab as a “pioneering cross-economy sandbox” that would oversee controlled deployment of AI-enabled products and services in live market environments, granting participating firms time-limited regulatory exemptions known as sandbox pilots. The rationale is economic: DSIT’s call-for-evidence document cites OECD modelling suggesting AI could add 0.4 to 1.3 percentage points to UK productivity growth over the next decade — equivalent to £55 billion to £140 billion in additional annual output by 2030 — while only 21% of UK businesses currently use AI, and 60% of respondents to an earlier call for evidence identified regulation as a barrier to adoption.

    The Lab builds on precedent. The UK pioneered the modern regulatory sandbox model with the Financial Conduct Authority’s 2016 fintech sandbox, since echoed by the EU, US, Japan, Estonia and Singapore. DSIT’s proposal also references the FCA’s Innovate Project, the Bank of England/FCA Digital Securities Sandbox, the ICO’s Data Protection Sandbox, and the MHRA’s AI Airlock — the last of which is already piloting oversight of ambient voice technologies (AI tools that transcribe clinician-patient conversations) through its “TORTUS” case study.

    An advisory version of the AI Growth Lab launched on 8 June 2026, bringing together the Legal Services Board, the Solicitors Regulation Authority and other regulators to trial AI products in legal services first, with the Information Commissioner’s Office issuing a supporting statement the same day. Statutory sandbox pilots, which would require primary legislation to grant regulators modification powers, remain subject to further parliamentary process; the House of Lords debated the proposal on 26 March 2026.

    How AI Growth Lab Sandbox Pilots Work

    DSIT’s proposal sets out a consistent operating logic for sandbox pilots, regardless of sector:

    • Issue-specific sandboxes target sectors with clear AI opportunity but where existing regulation impedes adoption — legal services, planning, diagnostic imaging and micromobility/robotics are the named early candidates.
    • Time-limited exemptions are granted to eligible firms and products, allowing them to operate under modified rules while under close supervision, with the Lab able to end a pilot at any time.
    • “Red lines” stay fixed. DSIT proposes that consumer protections, safety provisions, fundamental rights, workers’ protections and intellectual property rights can never be modified or disapplied during a pilot.
    • Successful pilots feed reform. Evidence from a pilot can inform permanent regulatory change — updated guidance, codes of practice, or secondary legislation — subject to parliamentary scrutiny.

    DSIT is weighing two operating models: a centrally operated Lab run by government with an Oversight Committee of sectoral regulators, better suited to cross-sector AI applications; and regulator-operated Labs, where a lead regulator runs the sandbox for its own sector — closer to the MHRA AI Airlock precedent. The table below situates the proposed Lab against sandboxes already operating in the UK.

    Sandbox Lead body Sector focus Modification power
    FCA Innovate Sandbox Financial Conduct Authority Fintech / financial services Advisory + authorisation support
    MHRA AI Airlock Medicines and Healthcare products Regulatory Agency AI as a medical device Advisory, phased case studies
    ICO Data Protection Sandbox Information Commissioner’s Office Cross-sector data protection Advisory
    AI Growth Lab (proposed) DSIT, with sectoral regulators Cross-economy, sector pilots Statutory exemptions (“sandbox pilots”), subject to red lines

    What It Means for University-Led AI Research

    DSIT’s call-for-evidence explicitly invited responses from “a research organisation, university or think tank” as a distinct respondent category, and the proposal’s own framing links the Lab to place-based AI Growth Zones, which are designed to pair university and industry AI capacity — with embodied and infrastructure-heavy AI applications potentially gaining access to the government’s AI Research Resource (AIRR), the shared compute allocation for UK AI research. That link between a regulatory sandbox and a compute-access programme is largely absent from law-firm commentary on the Lab, which has focused on commercial and professional-services angles.

    In practice, the clearest route into a pilot for most universities runs through spinouts and licensed technology transfer, since DSIT’s proposed eligibility criteria favour applicants with a near-market product, a UK nexus, and a demonstrable regulatory barrier — not early-stage research.

    • Opportunities: real-world testing routes for spinouts translating lab research into deployable tools; potential access to data and infrastructure otherwise gated by regulation; earlier sight of which regulatory barriers government is prepared to modify.
    • Risks: eligibility criteria oriented to market-ready products rather than exploratory research; unresolved questions on intellectual property and publication timing inside a supervised pilot; added administrative and ethical-review burden for institutions without dedicated regulatory-affairs capacity.

    Research offices supporting clinical AI should note that DSIT names the Ionising Radiation (Medical Exposure) Regulations as a candidate for pilot modification, given AI’s growing accuracy in interpreting scans — a live example of a pilot touching clinical research governance directly, not just commercial deployment.

    Common Questions About the AI Growth Lab

    What is an AI Growth Lab “sandbox pilot”?

    A sandbox pilot is a time-limited, closely supervised arrangement in which an eligible firm or product receives a targeted exemption from specific regulatory requirements. DSIT can end a pilot at any time, and protections such as consumer rights and safety provisions remain fixed “red lines” throughout.

    Which sector was first to join the AI Growth Lab?

    Legal services became the first sector inside the advisory AI Growth Lab, launched on 8 June 2026 with the Legal Services Board and Solicitors Regulation Authority as founding regulators. DSIT has signalled healthcare, planning and robotics as likely next candidates for issue-specific sandboxes.

    Who can apply to participate in the AI Growth Lab?

    DSIT’s proposal envisages applications from start-ups, established companies, global AI developers and public-sector innovators, with eligibility weighted toward a UK nexus, consumer benefit, and a demonstrable regulatory barrier. Final eligibility criteria were still under consultation as of the call-for-evidence close in January 2026.

    How does the AI Growth Lab differ from AI Growth Zones?

    AI Growth Zones are place-based clusters pairing infrastructure, compute and industry investment in specific UK locations, while the AI Growth Lab is a regulatory mechanism that can operate across the whole economy. DSIT’s proposal treats the two as complementary, with place-based sandbox pilots able to draw on AI Growth Zone infrastructure.

    What Research Offices Should Track Next

    The call for evidence has closed, but several decision points remain open and directly relevant to research administration teams supporting AI-related grants, spinouts and clinical trials:

    • Eligibility criteria finalisation — whether DSIT’s final rules for the Lab explicitly recognise university research organisations or spinouts as a distinct applicant category, beyond commercial firms.
    • Sector rollout order — after legal services, which sector opens next; healthcare/diagnostic imaging and planning are the most research-relevant candidates named in the proposal.
    • Oversight model — whether DSIT adopts a centrally operated Lab or regulator-operated Labs, which will determine which single point of contact a university would need to approach.
    • Primary legislation — statutory modification powers require parliamentary approval; institutions should track Hansard and DSIT announcements for the bill’s progress following the 26 March 2026 Lords debate.
    • AI Research Resource access — whether compute allocation under AIRR becomes formally linked to sandbox participation for embodied or infrastructure-heavy AI pilots.

    None of this displaces existing research governance. Institutional ethics review, data protection obligations, and research integrity processes continue to apply inside a sandbox pilot exactly as DSIT’s “red lines” intend — the Lab modifies sector regulation, not an institution’s own duty of care. Research offices that map their AI-active spinouts and clinical-AI projects against the Lab’s likely next sectors now will be better placed to respond quickly once eligibility criteria and the second wave of issue-specific sandboxes are confirmed.