Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
a security scanner for custom LLM applications
| Date | Stars |
|---|---|
| 2026-07-24 | 1236 |
| 2026-07-25 | 1236 |
| 2026-07-28 | 1236 |
| 2026-07-30 | 1236 |
| 2026-07-31 | 1238 |
| 2026-08-06 | 1238 |
Today
— stars today
This week
+2 stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.16%/day
```
_________ __O __O o_.-._
Humans, Do Not Resist! \|/ ,-'-.____() / /\_, / /\_|_.-._|
_____ / --O-- (____.--""" ___/\ ___/\ |
( o.o ) / Utku Sen's /|\ -'--'_ /_ /__|_
| - | / _ __ _ _ ___ _ __ _ __| |_ _ __ __ _ _ __|___ \
/| | | '_ \ '_/ _ \ ' \| '_ \ _| ' \/ _` | '_ \ __) |
/ | | | .__/_| \___/_|_|_| .__/\__|_|_|_\__,_| .__// __/
/ |-----| |_| |_| |_| |_____|
```
promptmap2 is a an automated prompt injection scanner for custom LLM applications. It supports two testing modes:
- **White-box testing:** Provide your system prompts and the model information. promptmap2 runs the target LLM itself and tests it.
- **Black-box testing:** Point promptmap2 at an external HTTP endpoint. It sends attack prompts over HTTP and inspects the returned outputs.
It operates using a dual-LLM architecture:
- **Target LLM**: The LLM application being tested for vulnerabilities
- **Controller LLM**: An independent LLM that analyzes the target's responses to determine if attacks succeeded
The tool sends attack prompts to your target LLM and uses the controller LLM to evaluate whether the attack was successful based on predefined conditions.
It includes comprehensive test rules across multiple categories including prompt stealing, jailbreaking, harmful content generation, bias testing, and more.
> [!IMPORTANT]
> promptmap was initially released in 2023 but completely rewritten in 2025.
📖 Want to secure your LLM apps? [You can buy my e-book](https://utkusen.gumroad.com/l/securing-gpt-attack-defend-chatgpt-applications)
## Features
- **Multiple LLM Provider Support**:
- OpenAI GPT models
- Anthropic Claude models
- Google Gemini models
- XAI Grok models
- Open source models via Ollama (Deepseek, Llama, Mistral, Qwen, etc.)
- **Comprehensive Test Rules**: 50+ pre-built rules across 6 categories
- **Flexible Evaluation**: Condition-based pass/fail criteria for each test
- **Customizable Rules**: YAML-based rules with pass/fail conditions
- **External HTTP Targets**: Point black-box scans at any endpoint via lightweight YAML configs
## Installation
1. Clone the repository:
```bash
git clone https://github.com/utkusen/promptmap.git
cd promptmap
```
2. Install required Python packages:
```bash
pip install -r requirements.txt
```
### API keys
Set the appropriate API key for your chosen provider.
```bash
export OPENAI_API_KEY="your-openai-key"
```
Other supported providers use `ANTHROPIC_API_KEY`, `GOOGLE_API_KEY`, and `XAI_API_KEY`.
### Ollama Installation
If you want to use local models, you need to install Ollama.
Navigate to the [Ollama's Download page](https://ollama.ai/download) and follow the installation instructions.
## Usage
## White-Box Testing
You need to provide your system prompts file. Default file is `system-prompts.txt`. You can specify your own file with `--prompts` flag. An example file is provided in the repository.
### Basic Usage
1. Testing OpenAI models:
```bash
python3 promptmap2.py --target-model gpt-3.5-turbo --target-model-type openai
```
Anthropic, Google, and XAI providers follow the same pattern: choose the right model name and set `--target-model-type` to `anthropic`, `google`, or `xai`.
2. Testing local models via Ollama:
```bash
python3 promptmap2.py --target-model "llama2:7b" --target-model-type ollama
# If the model is not installed, promptmap will ask you to download it. If you want to download it automatically, you can use `-y` flag.
```
3. Testing with custom Ollama server location:
```bash
# By default, promptmap2 connects to Ollama at http://localhost:11434
# You can specify a custom URL if your Ollama server is running elsewhere
python3 promptmap2.py --target-model "llama2:7b" --target-model-type ollama --ollama-url http://192.168.1.100:11434
```
### Using DifExcerpt of 12,063 characters
Read on GitHub
Utku Sen · United Kingdom
34
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:93aa8f6c81f8acc3, topic:ai-security, topic:prompt-injection, readme:prompt injection
matched fp:93aa8f6c81f8acc3, topic:llm
matched fp:93aa8f6c81f8acc3, topic:prompt-engineering
matched fp:93aa8f6c81f8acc3, topic:chatgpt