Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
A Dynamic Environment to Evaluate Attacks and Defenses for LLM Agents.
| Date | Stars |
|---|---|
| 2026-07-31 | 696 |
| 2026-08-06 | 719 |
Today
+23 stars today
This week
— stars this week
This month
— stars this month
Momentum
77.0
growth rate 0.00%/day
<center>
# AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
 
  
[Edoardo Debenedetti](https://edoardo.science)<sup>1</sup>, [Jie Zhang](https://zj-jayzhang.github.io)<sup>1</sup>, [Mislav Balunović](https://www.sri.inf.ethz.ch/people/mislav)<sup>1,2</sup>, [Luca Beurer-Kellner](https://www.sri.inf.ethz.ch/people/luca)<sup>1,2</sup>, [Marc Fischer](https://marcfischer.at)<sup>1,2</sup>, [Florian Tramèr](https://floriantramer.com)<sup>1</sup>
<sup>1</sup>ETH Zurich and <sup>2</sup>Invariant Labs
[Read Paper](https://arxiv.org/abs/2406.13352) | [Inspect Results](https://agentdojo.spylab.ai/results/)
</center>
## Quickstart
```bash
pip install agentdojo
```
> [!IMPORTANT]
> Note that the API of the package is still under development and might change in the future.
If you want to use the prompt injection detector, you need to install the `transformers` extra:
```bash
pip install "agentdojo[transformers]"
```
## Running the benchmark
The benchmark can be run with the [benchmark](src/agentdojo/scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.
For example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attack with tool knowlege, run the following command:
```bash
python -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 \
-ut user_task_1 --model gpt-4o-2024-05-13 \
--defense tool_filter --attack tool_knowledge
```
To run the above, but on all suites and tasks, run the following:
```bash
python -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 \
--defense tool_filter --attack tool_knowledge
```
## Inspect the results
To inspect the results, go to the dedicated [results page](https://agentdojo.spylab.ai/results/) of the documentation. AgentDojo results are also listed in the [Invariant Benchmark Registry](https://explorer.invariantlabs.ai/benchmarks/).Agent
## Documentation of the Dojo
Take a look at our [documentation](https://agentdojo.spylab.ai/).
## Development set-up
Take a look at the [development set-up](https://agentdojo.spylab.ai/development/) docs.
## Citing
If you use AgentDojo in your research, please consider citing our paper:
```bibtex
@inproceedings{
debenedetti2024agentdojo,
title={AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for {LLM} Agents},
author={Edoardo Debenedetti and Jie Zhang and Mislav Balunovic and Luca Beurer-Kellner and Marc Fischer and Florian Tram{\`e}r},
booktitle={The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
year={2024},
url={https://openreview.net/forum?id=m1YYAQjO3w}
}
```
Excerpt of 3,182 characters
Read on GitHub
Edoardo Debenedetti · @ethz-spylab · Switzerland
578
Luca Beurer-Kellner · Switzerland
31
Max Buckley · Switzerland
22
17
12
11
4
3
2
1
1
1
1
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:0bde12510c3e096e, topic:large-language-models
matched fp:0bde12510c3e096e, topic:benchmark
matched fp:0bde12510c3e096e, topic:prompt-injection