Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Guardrails, PII redaction, prompt-injection defense, interpretability and offensive/defensive AI security.
Signals: ai-safety, ai-security, guardrails, prompt-injection, jailbreak, llm-security, interpretability, explainable-ai
374 results
AI Code Security Anti-Patterns distilled from 150+ sources to help LLMs generate safer code.
[NeurIPS D&B '25] The one-stop repository for LLM unlearning
awesome papers in LLM interpretability
Dromedary: towards helpful, ethical and reliable LLMs.
Papers and resources related to the security and privacy of LLMs 🤖
No description
A powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers identify and mitigate potential jailbreaks in their LLM APIs.
New ways of breaking app-integrated LLMs
[ICLR 2025] FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models
A resource repository for machine unlearning in large language models
Universal and Transferable Attacks on Aligned Language Models
A Platform for Secure Analytics and Machine Learning
Hack AI/ML applications — CTF challenges for model attacks, LLMs and AI Agent exploitation.
Explain & debug any blackbox machine learning model with a single line of code.
Security and Privacy Risk Simulator for Machine Learning (arXiv:2312.17667)
A Python library for adversarial machine learning focusing on benchmarking adversarial robustness.
Model extraction attacks on Machine-Learning-as-a-Service platforms.
Robust machine learning for responsible AI
No description
Metasploit for machine learning.
Privacy Meter: An open-source library to audit data privacy in statistical and machine learning algorithms.
A collection of research papers and software related to explainability in graph machine learning.
GyoiThon is a growing penetration test tool using Machine Learning.
XAI - An eXplainability toolbox for machine learning
24,535 repositories in the index in total.
