Rethink Priorities researchers presented a toolkit for profiling AI risk behavior in Benchmarking AI risk behaviors, a preprint first submitted to arXiv on November 12, 2026. It examines choices under uncertainty across different tasks and contexts.
What the risk toolkit measures
The paper describes evaluation protocols and a profiling pipeline, with prompts and methods in its appendices. Tests compare certain outcomes with 50/50 gambles, varying stakes, beneficiaries and scenario detail. Its nine-model panel includes Claude Opus 5, GPT-5.6, Gemini 3.8 Flash and Llama 4 Maverick; coverage varies by experiment.
For researchers designing a model evaluation suite, the profile separates three questions:
| Profile axis | Question it addresses |
|---|---|
| Level | How cautious is the model on average? |
| Domain sensitivity | Does its risk orientation change across domains? |
| Transition | Does its choice switch sharply or gradually? |
What the authors report
The authors find newer models tend toward greater stability, consistency and risk neutrality, often choosing the higher expected numerical payoff. Opus 5 is more cautious and sensitive to context. These are findings from the tested conditions.
What this means for universities and research offices
The authors explicitly caution that lottery responses need not predict agentic or safety-relevant behavior. The tests therefore do not establish deployment safety.
The institutional implication is a distinction between reproducibility and suitability: a model can repeatedly apply the same calculation while leaving a research office’s underlying responsibilities unresolved. A university pilot could use these profile questions to describe the behavior it is investigating, while keeping separate the evidence needed to entrust a system with consequential decisions. That is an interpretation of the study’s relevance, rather than a demonstrated institutional outcome.
Broader governance remains relevant. NIST describes its AI Risk Management Framework as voluntary and intended to incorporate trustworthiness into AI design, development, use and evaluation. CASRAI’s account of the NIST framework for research institutions provides that wider context. The new paper contributes a specific behavioral lens within this larger evaluation question.
The work is labelled a preprint. arXiv states that its moderation process is not peer review; posting there does not itself establish peer-reviewed validation.
Sources
- arXiv: Benchmarking AI risk behaviors — submission record. Checked 2026-11-14.
- Mikaelson and colleagues: full preprint and appendices. Checked 2026-11-14.
- NIST: AI Risk Management Framework. Checked 2026-11-14.
- arXiv: Content Moderation. Checked 2026-11-14.
