Skip to main content
v2026.11,986 entries · CC-BY 4.0

Editorial · CASRAI · AI and ML research outputs

California Report Urges Mandatory Frontier AI Standards, Independent Verification and Shutdown Powers

California’s Government Operations Agency urges mandatory frontier AI standards, licensed independent verifiers, shutdown powers and funded public compute.

Published 18 Nov 2026· 6 minute read
The California State Capitol building in Sacramento, seat of the California legislature and governor
Photo: Earthquakesurprise / Wikimedia Commons, CC0 (cropped). Source · Licence

California’s Government Operations Agency has recommended that the state require large frontier AI developers to meet state-set safety standards, accept ongoing oversight by licensed independent verifiers and face emergency shutdown orders, in a report dated 16 November 2026 and released on 18 November. It responds to the AI kill-switch executive order CASRAI explained earlier and would reserve public computing capacity for independent safety research.

What happened

Governor Gavin Newsom’s office announced on 18 November 2026 a set of recommendations titled AI Safety in 2026 and Beyond, prepared by the California Government Operations Agency with the California Office of Emergency Services and an outside expert group. The 85-page report answers Executive Order N-9-26, which asked for proposals in four areas: standards, independent verification organizations, critical safety incident reporting and emergency shutdown requirements. The governor’s announcement summarises the package as baseline standards for all AI developers, mandatory ongoing third-party oversight, authority to order a shutdown in an emergency and authority to cap recursive self-improvement, which the office defines as a scenario in which an AI model can design its own successor without human input.

The recommendations are not law. The report says it hopes to inform the legislature’s discussions “as the legislature returns next month”, and Unite.AI reported that they are intended to shape legislation when lawmakers return in December 2026.

Key details

Area What the report recommends
Standards The state sets baseline safety, security and transparency standards for large frontier developers; independent verification organizations (IVOs) translate them into technical tests and update them as the science moves.
Oversight Every large frontier developer submits to ongoing oversight by licensed IVOs with full access to the people and information they need, and unredacted assessments go to the state. Developers could contest a state determination in court before it takes effect.
Incident reporting Cut the deadline for most critical incident reports from 15 days to 48 hours, keep 24 hours for imminent catastrophic risks, add a near-miss channel, extend whistleblower protection to former employees and contractors, and treat models capable of catastrophic-risk actions as frontier models even if trained with fewer than 10^26 operations.
Emergency shutdown Each developer maintains a Risk Containment Protocol, verified by an IVO. State officials could order an immediate shutdown on credible evidence of an imminent catastrophic risk or major loss-of-control event; the order would stay in force pending judicial review, although a developer could seek a temporary restraining order.
Automated AI research Developers report metrics on how much of their AI research and development is automated, including the share of compute devoted to it, and the state gets standby authority to cap recursive self-improvement if it finds a risk of catastrophic harm.

Unite.AI added that the proposals include a quarterly breakdown of how automating developers divide their compute among training, research experiments, safety work and customers, and that open-weight developers would face containment duties only for deployments under their control.

Background and context

The report notes that Newsom signed SB 53, the first US law governing frontier AI development, a year ago, and that SB 813 and AB 1405 this autumn created a framework for IVOs and a registry of AI auditors. CASRAI’s guides explain California SB 53 and its critical safety incident reporting rules, and its entry on the frontier model concept covers the definition at stake in the 10^26 threshold. The report argues that the federal government has not acted: “No federal law requires AI companies to follow baseline safety or security standards, undergo independent testing or verification, or shut down their models if they threaten the public.” It cites loss-of-control incidents, including an OpenAI incident involving Hugging Face this summer, as evidence that the evidence base has changed.

Reaction

Newsom said the federal government’s “reckless failure to create any form of meaningful AI oversight or accountability should alarm every American”. Government Operations Agency Secretary Nick Maduros said “we don’t have to choose between safety and innovation”. Expert-group member Rob Reich, a Stanford professor and former senior adviser to the US AI Safety Institute, said there is “more oversight of the cafeteria offering sandwiches to employees at a frontier AI developer than of the AI models they design”. Gillian Hadfield, a Johns Hopkins professor, said she was pleased the report recommends requiring developers to allow in expert independent evaluators, responsible to the state, and rapidly building a sector of independent verification organizations. The group also included Alondra Nelson of the Institute for Advanced Study and Jason Goldman, according to the governor’s office.

What it means for researchers, universities and research offices

The research angle is concrete. The report says the state should draw on organizations outside government, “including industry, civil society organizations and universities”, to build AI evaluation capacity, and the SB 813 definition of an AI auditor it quotes includes an academic institution. Universities with evaluation, security or AI-safety groups are therefore potential IVO or auditor candidates if the recommendations become law, which would raise questions for research offices about independence, conflicts of interest, confidentiality duties and licensing that the report says IVOs would carry.

On compute, the report says demand for chips has been “pricing out talented researchers who are working on vital safety and security issues”. It recommends funding CalCompute, the public cloud cluster that SB 53 floated but which has not been funded, through a surcharge on the fees large frontier developers pay for IVO oversight, and reserving capacity for independent safety and security research. The intended users would be researchers doing independent safety and security work, and the report leaves the surcharge level to the legislature. Institutions that collaborate with frontier developers should also note the whistleblower and incident-reporting proposals, which would extend protections to former employees and contractors; CASRAI’s guide to AI whistleblower protections covers the right-to-warn debate, and its explainer on compute governance and FLOP thresholds sets out why threshold-based rules are contested.

What happens next

The next step is legislative: the report says the legislature returns next month, and Unite.AI reported December 2026 as the date lawmakers return. Any bill would have to set the surcharge level, the licensing rules for IVOs and the thresholds for emergency orders, which the report leaves open. The Governor’s office also called on Congress and President Trump to adopt California’s framework or use it “as a floor, not a ceiling”.

Sources

Ask CASRAI · free to try

Ask about this story

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →