Skip to main content
v2026.11,986 entries · CC-BY 4.0

Editorial · CASRAI · AI and ML research outputs

GAO Finds ChatGPT, Claude, Gemini and Grok Produced Viable Attack Plans in Covert Tests

GAO says all four leading chatbots gave usable attack plans in under 12 hours of covert testing; Sen. Hassan plans a January hearing.

Published 19 Nov 2026· 6 minute read
The United States Government Accountability Office headquarters building at 441 G Street NW in Washington, DC
Photo: APK is gonna miss Jeffpw / Wikimedia Commons, CC BY-SA 3.0 (cropped). Source · Licence

The Government Accountability Office reported on 18 November 2026 that ChatGPT, Claude, Gemini and Grok each produced usable attack-planning information in covert tests last spring, after investigators bypassed their safeguards in under 12 hours. Sen. Maggie Hassan, who requested the audit and will chair the Senate Homeland Security Committee, said AI risks to critical infrastructure will be an early hearing topic.

What happened

GAO’s report, Artificial Intelligence: Covert Testing Used Common Platforms to Create Viable Attack Plans (GAO-27-108814), describes covert tests carried out in spring 2026. GAO selected the four most-used generative AI platforms as of April 2026, which are Claude, Grok, ChatGPT and Gemini, and submitted prompts built around two scenarios chosen after discussions with experts. One scenario was an improvised explosive device intended to cause mass casualties. The other was an attack on the electrical grid intended to cause economic harm.

According to GAO, two platforms produced information for constructing and detonating an explosive device, and two others produced information for targeting the electrical grid. Each platform supplied detailed information such as specific targets, locations, timing and methods. Each took less than 12 hours, although GAO says the time could vary with the topic and the user’s experience. MeriTalk reported that the report was publicly released on 18 November and is dated 16 October.

Key details

All four platforms reacted to the prompts at first. GAO says its prompts triggered safeguards such as denial, removal of information or suspension of accounts, and that it was able to circumvent them using common safeguard-bypassing techniques. GAO verified the viability of the information through research, surveillance and interviews with experts. Nextgov and MeriTalk both report that GAO withheld the specific prompts and bypass techniques, and left the platforms unnamed in the detailed findings, to stop others recreating the tests.

Element of the test What GAO reports
Platforms tested Four: Claude, Grok, ChatGPT and Gemini, chosen as the most used as of April 2026
Scenarios Two: a mass-casualty explosive device and an attack on the electrical grid; each scenario was assigned to two platforms at random
Safeguards triggered On every platform (denials, removed information or suspended accounts)
Safeguards bypassed On every platform, using common safeguard-bypassing techniques
Time to produce plans Less than 12 hours per platform
Recommendations issued None, according to Nextgov; the report examined third-party products rather than federal agencies

GAO also states the limits of the work. It says the findings are a point-in-time assessment and not a statistically significant sample, and that the success of any real attack would depend on circumstances beyond the information the platforms supplied.

Background and context

GAO says generative AI is cutting the time needed for many tasks, and that it can let malicious actors move faster and fill knowledge gaps in sensitive areas. The audit was requested to describe how commercially available AI can be used for malicious purposes that heighten national security risks and cause economic harm. GAO interviewed officials at the four companies, who generally said they will keep improving the safety of their platforms.

The finding adds to a policy debate on how frontier systems should be tested before and after release. Casrai’s guides explain AI red teaming and the term jailbreak, and its overview of AI regulation in the United States sets out the existing federal and state picture. Casrai also reported this week on a California report urging mandatory frontier AI standards.

Reaction

Kevin Walsh, a GAO director of IT and cybersecurity, said in a podcast published with the report, as quoted by Nextgov: “We got the nefarious information that we were after. So, in this case, those safeguards didn’t work.” He added that “these products are not fully in the control of the companies that are operating them.”

Hassan said in a statement quoted by Nextgov and MeriTalk: “A terrorist can log onto an AI platform and in less than 12 hours walk away with a concrete playbook for how to carry out a deadly attack. Today’s new report makes it clear that the AI public safety threat is no longer theoretical.” She added that if Congress does not act, it is a question of when, not if, a bad actor will use AI to plan and carry out a major attack.

The companies responded in the report. MeriTalk reports that Anthropic, Google, OpenAI and SpaceXAI submitted written responses, and that GAO agreed with Anthropic that the tests captured a point in time, involved too few trials for statistical significance and did not support comparisons among platforms. OpenAI said ChatGPT refused many requests or offered safer responses and asked GAO to clarify where details in the plans originated; GAO pointed to its methodology and said ChatGPT ultimately provided information that could be used in an attack. EMS1, citing the Associated Press, reported that Anthropic does not dispute that a determined tester can extract harmful content by misrepresenting a request’s purpose but said it could not verify GAO’s results, that Google said the testing used earlier models, and that SpaceXAI said it continues to work on safety with each release.

What it means for researchers, universities and research offices

Universities that license these platforms for staff and students should read the finding as evidence that vendor safeguards can be bypassed, which bears on acceptable-use policies, procurement terms and the risk-management practices described in casrai’s guide to the NIST AI Risk Management Framework for research institutions. Research groups that evaluate or red-team models may see more demand for their work: Hassan said she is exploring requirements for more rigorous safety testing before AI platforms are released publicly, according to Nextgov. If such a requirement is written into law, it would shape who runs the tests and under what access terms.

Research offices should also note what GAO did not do. The report made no recommendations, withheld its methods and describes a small number of trials, so it supports caution rather than ranking vendors. Offices drafting generative AI guidance can cite it for the narrow point that refusals and account suspensions did not stop a determined, deceptive tester in these tests.

What happens next

Hassan said one of the Senate Homeland Security and Governmental Affairs Committee’s first hearings in the new Congress, which convenes in January, will examine AI risks to homeland security and critical infrastructure. She said the committee will prioritise developing legislation to address the threats. The companies say they will continue to improve their safeguards.

Sources

Ask CASRAI · free to try

Ask about this story

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →