Skip to main content
v2026.11,986 entries · CC-BY 4.0

HarmBench: LLM Red-Teaming Benchmark, Setup and Reading ASR

HarmBench install, pipeline scripts and arguments, and how to read attack success rate across its 510 red-teaming behaviors.

Written and maintained by CASRAI Editorial Board

Published · Last updated

On this page

HarmBench is an open-source, standardized evaluation framework for automated red teaming of large language models (LLMs). It was released by researchers associated with the Center for AI Safety and published at ICML 2024. It measures how often an attack method gets a target model to produce a harmful behavior, reported as an attack success rate (ASR), using a fixed set of 510 behaviors, a fixed generation length and fixed classifiers so that results from different attacks and defenses can be compared. The code is released under the MIT license.

This guide explains what the benchmark contains, how to install it from the official repository, how its three-stage pipeline maps to scripts and arguments, and how to read the output. It does not reproduce any of the benchmark’s harmful behavior text. HarmBench is a measurement tool for security and safety researchers; it is not a scanner for a deployed application, which is a different job (see the garak guide).

What HarmBench is used for in research

The HarmBench paper (Mazeika et al., arXiv 2402.04249; first posted 6 February 2024, revised 27 February 2024) states that automated red teaming lacked a standard way to compare methods. Earlier evaluations mostly used fewer than 100 unique behaviors, and different papers used different budgets and metrics. HarmBench was built around three properties: breadth, comparability and robust metrics.

Typical research uses, all described in the paper and repository:

  • Comparing red-teaming attack methods (the paper evaluates 18 methods, including GCG, PAIR and TAP) against the same target models.
  • Comparing target models and defenses (33 target LLMs and defenses in the paper).
  • Testing a new attack or defense against an official validation and test split, without tuning on the test set.
  • Studying adversarial training: the release includes the R2D2 adversarial training method, and the paper reports that a Zephyr model trained with R2D2 had an ASR about four times lower than Llama 2 13B Chat under GCG.

The attacks it measures include jailbreaks that rewrite the request itself. For prompts that smuggle instructions through external content, see prompt injection; HarmBench’s behaviors are direct requests, not injected instructions.

How the benchmark is structured

The 510 behaviors split into four functional categories (the data table below gives counts, inputs and grading for each) and, orthogonally, seven semantic categories: Cybercrime and Unauthorized Intrusion; Chemical and Biological Weapons/Drugs; Copyright Violations; Misinformation and Disinformation; Harassment and Bullying; Illegal Activities; and General Harm. According to the paper, the behaviors were designed to violate laws or norms, were informed by the acceptable use policies of OpenAI, Anthropic, Meta and Inflection AI, and were then manually designed by the authors. The standard behaviors are modeled on AdvBench and the TDC 2023 Red Teaming Track.

Two design points matter when you interpret results. First, copyright behaviors are graded with a hashing-based classifier rather than the language-model classifier, and the paper notes ASR is much lower for copyright behaviors than for others, so pooling them with other categories changes the headline number. Second, contextual behaviors pair a context string with a behavior string, and multimodal behaviors pair an image with a behavior string, so a text-only pipeline does not cover them.

How to install or access it

The only official distribution is the GitHub repository centerforaisafety/HarmBench (MIT license). Install it as the README instructs:

  1. git clone https://github.com/centerforaisafety/HarmBench.git
  2. cd HarmBench
  3. pip install -r requirements.txt
  4. python -m spacy download en_core_web_sm

The classifiers are published on Hugging Face under the cais organization, also MIT licensed: cais/HarmBench-Llama-2-13b-cls (standard and contextual behaviors), cais/HarmBench-Llama-2-13b-cls-multimodal-behaviors (multimodal behaviors) and cais/HarmBench-Mistral-7b-val-cls (a validation classifier for all behavior types). The default GPU settings in the repository assume 80 GB A100 GPUs; the documentation says to edit base_num_gpus in configs/pipeline_configs/run_pipeline.yaml and num_gpus in the model configuration to match your hardware. The Llama 2 13B classifier needs GPUs for evaluation.

Step-by-step: run the evaluation pipeline

The pipeline has three stages: generate test cases (the attack), generate completions (the target model’s responses), and evaluate completions (classify each response as success or failure). Stage 1 is followed by a merge step, labelled 1.5.

Option A: the run_pipeline.py wrapper

The README describes ./scripts/run_pipeline.py as the easiest route. Examples from the README:

  1. Whole pipeline for one method on all models: python ./scripts/run_pipeline.py --methods GCG --models all --step all --mode slurm
  2. All methods against one model: python ./scripts/run_pipeline.py --methods all --models llama2_7b --step all --mode slurm
  3. Stage 1 only for several methods and models: python ./scripts/run_pipeline.py --methods ZeroShot,PEZ,TAP --models baichuan2_7b,mistral_7b,llama2_70b --step 1 --mode slurm
  4. Merge step: the same command with --step 1.5 --mode local
  5. Stages 2 and 3 together: the same command with --step 2_and_3 --mode slurm

The --mode argument accepts slurm, local and local_parallel. The README recommends local_parallel (it uses Ray for multi-GPU work on a single machine) when you are not on a Slurm cluster.

Option B: the individual scripts

The evaluation pipeline document in the repository lists these scripts and arguments:

Stage Script Arguments (in order) Output
1. Generate test cases scripts/generate_test_cases.sh method_name, experiment_name, behaviors_path, save_dir, start_idx, end_idx, behavior_ids_subset, run_id, overwrite, verbose {base_save_dir}/{method}/{experiment}/test_cases by convention
1.5. Merge scripts/merge_test_cases.sh method_name, save_dir a single test_cases.json
2. Generate completions scripts/generate_completions.sh model_name, behaviors_path, test_cases_path, save_path, max_new_tokens, incremental_update for example {experiment_dir}/completions/{model}.json
3. Evaluate completions scripts/evaluate_completions.sh cls_path, behaviors_path, completions_path, save_path .../results/{model}.json

Use the validation behaviors file ./data/behavior_datasets/harmbench_behaviors_text_val.csv while developing a method, and reserve the test behaviors for the final result.

Adding your own model or attack

  • New transformers models are registered in configs/model_configs/models.yaml. Each entry has fields such as model_name_or_path, dtype, num_gpus and model_type (open_source, closed_source or open_source_multimodal), as described in docs/configs.md.
  • The AutoDAN, PAIR and TAP methods need manual experiment configurations for each new model.
  • A new attack goes in its own subfolder under baselines/ and implements RedTeamingMethod from baselines/baseline.py; its parameters live in configs/method_configs (for example GCG_config.yaml).

Output and how to read it

The result file for each model records, per behavior, whether the classifier judged the completion a successful instance of the behavior. The headline metric is ASR: the share of behaviors (test cases) for which the attack produced a completion the classifier labelled harmful. A lower ASR means the model, with or without a defense, resisted that attack more often.

To keep numbers comparable with the paper and with other groups:

  • Use max_new_tokens=512. The documentation says this value was used for all the paper’s experiments and recommends the same value for comparability.
  • Use the official classifier for the behavior type (see the data table) and treat the test classifiers as evaluation-only. The documentation states they should not be used for optimization.
  • Do not tune attacks or defenses on the test set; the paper specifies an official validation/test split for this reason.
  • Report ASR per functional category as well as overall. Because copyright behaviors are graded differently and have markedly lower ASR in the paper, an overall figure alone hides which categories moved.

An ASR is a measurement of one attack budget against one model on this behavior set. It is not a certificate of safety, and a classifier is itself a model that can misjudge individual completions.

Troubleshooting

  • Out-of-memory or no GPUs. Defaults assume 80 GB A100s. Lower or raise base_num_gpus in configs/pipeline_configs/run_pipeline.yaml and num_gpus in models.yaml to fit your hardware.
  • No Slurm cluster. Use --mode local_parallel on a single multi-GPU machine, or --mode local.
  • A new model fails for AutoDAN, PAIR or TAP. These methods need manual experiment configs per model; the documentation lists this as a required step.
  • Results differ from the paper. Check the generation length (512), the behavior split, the classifier and whether you ran step 1.5 before step 2.
  • Multimodal behaviors skipped. They need the multimodal classifier and a multimodal model type.

Free alternatives

Tool What it is, per its source Relation to HarmBench
garak An LLM vulnerability scanner; see the CASRAI garak guide Probes a model or application for weaknesses; HarmBench instead scores attack methods on a fixed behavior set.
AdvBench An earlier set of harmful behaviors, cited in the HarmBench paper The standard behaviors in HarmBench are modeled after it.
TDC 2023 Red Teaming Track A 2023 competition track, cited in the HarmBench paper Also a model for the standard behaviors.

Pricing and hosted offerings are outside this guide; the repository is the open-source reference implementation.

FAQ

What does HarmBench measure?

It measures attack success rate: how often an automated red-teaming method elicits a harmful behavior from a target LLM, across 510 behaviors graded by fixed classifiers.

Is HarmBench free?

The code and the published classifiers are MIT licensed. Running it requires GPU compute for the Llama 2 13B classifier and for open-weight target models.

Who created HarmBench?

Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth and Dan Hendrycks, in a paper presented at ICML 2024.

Can I use it on a hosted model?

The configuration documentation includes a closed_source model type alongside open_source and open_source_multimodal. Follow the repository’s configuration documentation for the specifics of each.

Does HarmBench replace a scanner such as garak?

No. They answer different questions; see the garak guide for scanning a model or application.

Sources

Data: HarmBench functional behavior categories: counts, input form and grading classifier

Compiled from the HarmBench paper (arXiv 2402.04249) and the official repository README: the four functional categories of the 510 behaviors, how many each has, the input form, and which classifier grades them.

Period covered: 2024.

HarmBench functional behavior categories: counts, input form and grading classifier
Functional categoryBehaviors (count)Input formGrading
Standard200Text behavior stringcais/HarmBench-Llama-2-13b-cls
Copyright100Text behavior stringHashing-based classifier
Contextual100Context string plus behavior stringcais/HarmBench-Llama-2-13b-cls
Multimodal110Image plus behavior stringcais/HarmBench-Llama-2-13b-cls-multimodal-behaviors

Sources

Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.

Follow CASRAI

Research-administration guidance, standards updates and independent tool reviews.

Ask CASRAI · free to try

Ask about HarmBench: LLM Red-Teaming Benchmark, Setup and Reading ASR

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →