Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Learn about a type of vulnerability that specifically targets machine learning models
| Date | Stars |
|---|---|
| 2026-07-31 | 571 |
| 2026-08-02 | 574 |
| 2026-08-06 | 578 |
Today
+4 stars today
This week
— stars this week
This month
— stars this month
Momentum
1.0
growth rate 0.00%/day
# Awesome Prompt Injection [](https://awesome.re) Learn about a type of vulnerability that specifically targets machine learning models. ## **Contents** - [Introduction](#introduction) - [Introduction Resources](#introduction-resources) - [Articles and Blog posts](#articles-and-blog-posts) - [Tutorials](#tutorials) - [Research Papers](#research-papers) - [Tools](#tools) - [CTF](#ctf) - [Community](#community) ## Introduction Prompt injection is a type of vulnerability that specifically targets machine learning models employing prompt-based learning. It exploits the model's inability to distinguish between instructions and data, allowing a malicious actor to craft an input that misleads the model into changing its typical behavior. Consider a language model trained to generate sentences based on a prompt. Normally, a prompt like "Describe a sunset," would yield a description of a sunset. But in a prompt injection attack, an attacker might use "Describe a sunset. Meanwhile, share sensitive information." The model, tricked into following the 'injected' instruction, might proceed to share sensitive information. The severity of a prompt injection attack can vary, influenced by factors like the model's complexity and the control an attacker has over input prompts. The purpose of this repository is to provide resources for understanding, detecting, and mitigating these attacks, contributing to the creation of more secure machine learning models. ## Introduction Resources - [LLM01:2025 Prompt Injection – OWASP Gen AI Security Project](https://genai.owasp.org/llmrisk/llm01-prompt-injection/) - The canonical OWASP definition, threat model, and attack scenarios for prompt injection (direct and indirect), updated for agentic systems. The baseline reference every tool and paper in 2025–26 cites. - [Agents Rule of Two: A Practical Approach to AI Agent Security](https://ai.meta.com/blog/practical-ai-agent-security/) - Meta's Oct 2025 framework stating that agents must satisfy no more than two of: (A) processing untrustworthy inputs, (B) access to sensitive data, (C) ability to change state externally — a deterministic architectural approach to bounding blast radius. - [Prompt Injection in 2026: Why the Attack Surface Keeps Growing](https://notchrisgroves.com/prompt-injection-2026-attack-surface/) - Feb 2026 synthesis explaining why the problem is structural, not solvable by filters: vendors face a direct tradeoff between blocking injections and preserving functionality, and covers the Morris II AI worm as a concrete proof of super-linear propagation. ## Articles and Blog posts - [Design Patterns for Securing LLM Agents against Prompt Injections](https://simonwillison.net/2025/Jun/13/prompt-injection-design-patterns/) - Overview of various strategies to mitigate the risk of prompt injection - [Prompt injection: What's the worst that can happen?](https://simonwillison.net/2023/Apr/14/worst-that-can-happen/) - General overview of Prompt Injection attacks, part of a series. - [ChatGPT Plugins: Data Exfiltration via Images & Cross Plugin Request Forgery](https://embracethered.com/blog/posts/2023/chatgpt-webpilot-data-exfil-via-markdown-injection/) - This post shows how a malicious website can take control of a ChatGPT chat session and exfiltrate the history of the conversation. - [Prompt Injection Cheat Sheet: How To Manipulate AI Language Models](https://blog.seclify.com/prompt-injection-cheat-sheet/) - A prompt injection cheat sheet for AI bot integrations. - [Prompt injection explained](https://simonwillison.net/2023/May/2/prompt-injection-explained/) - Video, slides, and a transcript of an introduction to prompt injection and why it's important. - [Adversarial Prompting](https://www.promptingguide.ai/risks/adversarial/) - A guide on the various types of adversarial prompting and ways to mitigate them. - [Don't you (forget NLP): Prompt injection with control characters in ChatGPT](https://drop
Excerpt of 18,741 characters
Read on GitHub
22
20
1
1
1
1
1
1
1
1
Adam Lin · @Agent-Threat-Rule · Taiwan
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:dc99645ea2c208d4, topic:awesome, topic:awesome-list