Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[NeurIPS D&B '25] The one-stop repository for LLM unlearning
| Date | Stars |
|---|---|
| 2026-07-31 | 574 |
| 2026-08-06 | 576 |
Today
+2 stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<div align="center">
<h3><strong>An easily extensible framework unifying LLM unlearning evaluation benchmarks.</strong></h3>
<div style="display: flex; gap: 10px; justify-content: center; align-items: center;">
<a href="https://arxiv.org/abs/2506.12618"><img src="https://img.shields.io/badge/arXiv-Report-b31b1b?logo=arxiv&logoColor=white" alt="arXiv Paper"/></a>
<a href="https://github.com/locuslab/open-unlearning"><img src="https://img.shields.io/github/stars/locuslab/open-unlearning?style=social" alt="GitHub Repo stars"/></a>
<a href="https://github.com/locuslab/open-unlearning/actions"><img src="https://github.com/locuslab/open-unlearning/actions/workflows/tests.yml/badge.svg" alt="Build Status"/></a>
<a href="https://huggingface.co/open-unlearning"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-blue" alt="HuggingFace 🤗"/></a>
<a href="https://github.com/locuslab/open-unlearning"><img src="https://img.shields.io/github/repo-size/locuslab/open-unlearning" alt="GitHub repo size"/></a>
<a href="https://github.com/locuslab/open-unlearning"><img src="https://img.shields.io/github/languages/top/locuslab/open-unlearning" alt="GitHub top language"/></a>
<a href="https://github.com/locuslab/open-unlearning/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue" alt="License: MIT"/></a>
</div>
</div>
---
## 📖 Overview
We provide efficient and streamlined implementations of the TOFU, MUSE and WMDP unlearning benchmarks while supporting 12+ unlearning methods, 5+ datasets, 10+ evaluation metrics, and 7+ LLM architectures. Each of these can be easily extended to incorporate more variants.
We invite the LLM unlearning community to collaborate by adding new benchmarks, unlearning methods, datasets and evaluation metrics here to expand OpenUnlearning's features, gain feedback from wider usage and drive progress in the field.
---
> 📝 If you've found this repository or the models we've provided in [HuggingFace](https://huggingface.co/open-unlearning) useful, please cite our [technical report](https://arxiv.org/abs/2506.12618) (bibtex at [*Citing this work*](#-citing-this-work)).
---
### 📢 Updates
### [June 20, 2025]
🚨 Our paper `OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics` is now out on [arXiv](https://arxiv.org/abs/2506.12618).
🌟 **Highlights:**
- A detailed technical report on OpenUnlearning covering the design, features, and implementation.
- A meta-evaluation framework for benchmarking unlearning evaluations across 450+ models, open-sourced on HuggingFace 🤗: [TOFU Models w & w/o Knowledge](https://huggingface.co/collections/open-unlearning/tofu-models-w-and-w-o-knowledge-6861e4d935eb99ba162e55cd), [TOFU Unlearned Models](https://huggingface.co/collections/open-unlearning/tofu-unlearned-models-6860f6cf3fe35d0223d92e88).
- Results benchmarking 8 diverse unlearning methods in one place using 10 evaluation metrics on TOFU.
<details>
<summary><b>Older Updates</b></summary>
#### [May 19, 2025]
- **More Methods!** Added support for unlearning methods [UNDIAL](https://aclanthology.org/2025.naacl-long.444/) and [AltPO](https://aclanthology.org/2025.coling-main.252/).
#### [May 12, 2025]
- **Another benchmark!** We now support running the [`WMDP`](https://wmdp.ai/) benchmark with its `Zephyr` task model.
- **More evaluations!** The [`lm-evaluation-harness`](https://github.com/EleutherAI/lm-evaluation-harness) toolkit has been integrated into OpenUnlearning, enabling WMDP evaluations and support for popular general LLM benchmarks, including MMLU, GSM8K, and others.
#### [Apr 6, 2025]
- **More Metrics!** Added 6 Membership Inference Attacks (MIA) (LOSS, ZLib, Reference, GradNorm, MinK, and MinK++), along with Extraction Strength (ES) and Exact Memorization (EM) as additional evaluation metrics.
- **More TOFU Evaluations!** Now includes a holdout set and sExcerpt of 15,199 characters
Read on GitHub
20
18
14
9
9
3
2
2
1
1
1
1
1
Shong Tan
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:1f1c706d37635d79, llm:topics: llm-unlearning, llm-privacy, llm-evaluation-metrics, membership-inference-attacks, privacy-protection, right-to-be-forgotten; description: 'The one-stop repository for LLM unlearning' (NeurIPS D&B '25).
matched fp:1f1c706d37635d79, llm:topics: llm-unlearning, llm-privacy, llm-evaluation-metrics, membership-inference-attacks, privacy-protection, right-to-be-forgotten; description: 'The one-stop repository for LLM unlearning' (NeurIPS D&B '25).
matched fp:1f1c706d37635d79, llm:topics: llm-unlearning, llm-privacy, llm-evaluation-metrics, membership-inference-attacks, privacy-protection, right-to-be-forgotten; description: 'The one-stop repository for LLM unlearning' (NeurIPS D&B '25).