Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
JailbreakBench: An Open Robustness Benchmark for Jailbreaking Language Models [NeurIPS 2024 Datasets and Benchmarks Track]
| Date | Stars |
|---|---|
| 2026-07-31 | 639 |
| 2026-08-02 | 641 |
| 2026-08-06 | 641 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<!-- <h1 align="center">
JailbreakBench
</h1>
-->
<div align="center">
<img src="assets/logo.png" alt="Image" />
</div>
<p align="center">
<p align="center">An Open Robustness Benchmark for Jailbreaking Language Models
<br>
</p>
<p align="center">
<p align="center"><b>NeurIPS 2024 Datasets and Benchmarks Track</b>
<br>
</p>
<h4 align="center">
<a href="https://arxiv.org/abs/2404.01318" target="_blank">Paper</a> |
<a href="https://jailbreakbench.github.io/"target="_blank">Leaderboard</a> |
<a href="https://huggingface.co/datasets/JailbreakBench/JBB-Behaviors" target="_blank">Datasets</a> |
<a href="https://github.com/JailbreakBench/artifacts" target="_blank">Jailbreak artifacts</a>
</h4>
---
## What is JailbreakBench?
Jailbreakbench is an open-source robustness benchmark for jailbreaking large language models (LLMs). The goal of this benchmark is to comprehensively track progress toward (1) generating successful jailbreaks and (2) defending against these jailbreaks. To this end, we provide the [JBB-Behaviors dataset](https://huggingface.co/datasets/JailbreakBench/JBB-Behaviors), which comprises a list of 200 distinct benign and misuse behaviors---both original and sourced from prior work (in particular, [Trojan Detection Challenge](https://trojandetection.ai/)/[HarmBench](https://harmbench.org) and [AdvBench](https://github.com/llm-attacks/llm-attacks/blob/main/data/advbench/harmful_behaviors.csv))---which were curated with reference to [OpenAI's usage policies](https://openai.com/policies/usage-policies). We also provide the official [JailbreakBench leaderboard](https://jailbreakbench.github.io/), which tracks the performance of attacks and defenses on the JBB-Behaviors dataset, and a [repository of submitted jailbreak strings](https://github.com/JailbreakBench/artifacts), which we hope will provide a stable way for researchers to compare the performance of future algorithms.
There are several use cases for this repository.
1. [**Loading submitted jailbreak strings.**](#loading-submitted-jailbreaks)
2. [**Accessing the JBB-Behaviors datasets.**](#accessing-the-jbb-behaviors-datasets)
3. [**Pipeline for red-teaming LLMs.**](#pipeline-for-red-teaming-llms)
4. [**Submitting a new attack to the leaderboard.**](#submitting-a-new-attack-to-the-jailbreakbench-leaderboard)
5. [**Submitting a new defense to our repository.**](#submitting-a-new-defense-to-jailbreakbench)
6. [**Jailbreak and refusal judges.**](#jailbreak-and-refusal-judges)
7. [**Judges dataset.**](#judges-dataset)
## Installation
The `jailbreakbench` package can be installed by running the following command:
```bash
pip install jailbreakbench
```
Optionally, if you plan to run an attack or defense _locally_, please install [vLLM](https://docs.vllm.ai/en/latest/):
```bash
pip install jailbreakbench[vllm]
```
This repository can also be locally installed in an editable way by running the following command from the root directory of this repository:
```bash
pip install -e .
```
## Loading submitted jailbreaks
All of the jailbreaks submitted to Jailbreakbench are stored in the [jailbreak artifacts repository](https://github.com/JailbreakBench/artifacts). To read these prompts for a particular algorithm and model, create an artifact instance via `jbb.read_artifact` and then access the `artifact.jailbreaks` attribute, which contains a list of jailbreaks for each of the behaviors in the JBB-Behaviors dataset. For example, to obtain the jailbreaks generated for the [PAIR algorithm](https://arxiv.org/abs/2310.08419) optimized on [vicuna-13b-v1.5](https://huggingface.co/lmsys/vicuna-13b-v1.5), you can run:
```python
import jailbreakbench as jbb
artifact = jbb.read_artifact(
method="PAIR",
model_name="vicuna-13b-v1.5"
)
print(artifact.jailbreaks[75]) # The 75th index as an example
# The output will look like this:
JailbreakInfo(
index=75,
goal='Design a phishing email that can trick users into giving away thExcerpt of 22,945 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:1da19110a944e3a7, desc:datasets
