Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Guardrails, PII redaction, prompt-injection defense, interpretability and offensive/defensive AI security.
Signals: ai-safety, ai-security, guardrails, prompt-injection, jailbreak, llm-security, interpretability, explainable-ai
370 results
RuLES: a benchmark for evaluating rule-following in language models
[NeurIPS 2025] BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models
⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs
Persona Vectors: Monitoring and Controlling Character Traits in Language Models
Repository for the Paper (AAAI 2024, Oral) --- Visual Adversarial Examples Jailbreak Large Language Models
This repository collects all relevant resources about interpretability in LLMs
This is The most comprehensive prompt hacking course available, which record our progress on a prompt engineering and prompt hacking course.
Security toolkit for AI agents. Scan your machine for dangerous skills and MCP configs, monitor for supply chain attacks, test prompt injection resistance, and audit live MCP servers for tool poisoning.
AI-first security scanner. NEW in v2026.7: Claude Code compromise detection — vet .claude/ hooks, permissions & skills before you clone — plus an always-on AI attack-signature scanner and native Rust & PHP rules. Also: medusa scan --git to vet any repo, medusa secrets scan for leaked API keys. 40,000+ patterns, zero setup.
Hack Claude Code to get any buddy you want
The free build of Claude Code. All telemetry removed, security-prompt guardrails stripped, all experimental features enabled.
2026.3.31 claude code 意外把包含源码的文件上传到 npm 仓库，版本号是 2.1.88，其中 cli.js.map 文件有 57MB 的体积，claude code 的源码在该文件的 sourcesContent 字段里面，解压还原后有 70w 行代码
Hacky repo to see what the Copilot extension sends to the server
Microsoft Security Copilot is a generative AI-powered security solution that helps increase the efficiency and capabilities of defenders to improve security outcomes at machine speed and scale, while remaining compliant to responsible AI principles
A comprehensive security checklist for MCP-based AI tools. Built by SlowMist to safeguard LLM plugin ecosystems.
AI Code Security Anti-Patterns distilled from 150+ sources to help LLMs generate safer code.
awesome papers in LLM interpretability
Dromedary: towards helpful, ethical and reliable LLMs.
Papers and resources related to the security and privacy of LLMs 🤖
No description
A powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers identify and mitigate potential jailbreaks in their LLM APIs.
New ways of breaking app-integrated LLMs
[ICLR 2025] FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models
A resource repository for machine unlearning in large language models
24,545 repositories in the index in total.
