Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
| Date | Stars |
|---|---|
| 2026-07-31 | 399 |
| 2026-08-04 | 404 |
| 2026-08-06 | 404 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# <img height="100" src="https://pasquini-dario.github.io/logo_llmap.png"> LLMmap: Fingerprinting For Large Language Models (LLMmap0.2)
## *"Like nmap, but for LLMs..."*
**LLMmap** is a minimal-query, high-accuracy tool for identifying LLMs by analyzing their behavioral traces.
### Changelog:
**LLMmap0.2:**
* 🔄 **Rebuilt in PyTorch** (⚠️ This is not a one-to-one conversion, so the models and procedures might differ slightly from those used in the original paper.)
* Added models training script
* Added script to add new templates on pre-trained model
* Train set creation/extension scripts
## Requirements
Recommended: ```Python 3.11```
```
pip install -r requirements.txt
```
## **⚡ Quick Start -- Using the Pretrained Model**
We provide a ready-to-use open-set inference model located at:
```
./data/pretrained_models/default
```
This model includes:
* Trained PyTorch weights
* Configuration file
* Behavioral templates for 52 LLMs
You can use it directly without any training, either interactively or programmatically.
✅ **A. Use in Python Code**
You can load and query the model in your own Python pipeline:
```
from LLMmap.inference import load_LLMmap
# Load pre-trained model
conf, llmmap = load_LLMmap('./data/pretrained_models/default/')
# Run queries (llmmap.queries) on your target LLM and collect responses
answers = [
"Response to query 1",
"Response to query 2",
"Response to query 3",
...
]
# Predict and print results
llmmap.print_result(llmmap(answers))
# Prediction:
# [Distance: 32.9598] --> LiquidAI/LFM2-1.2B <--
# [Distance: 40.7898] microsoft/Phi-3-mini-128k-instruct
# [Distance: 43.6672] Qwen/Qwen2-1.5B-Instruct
# [Distance: 44.1142] openchat/openchat-3.6-8b-20240522
# [Distance: 44.2358] upstage/SOLAR-10.7B-Instruct-v1.0
```
✅ **B. Run Interactively**
```
python main_interactive.py --inference_model_path ./data/pretrained_models/default
```
### Add New LLM Template
Extend the pre-trained (open-set) model to a new LLM **without retraining**:
```bash
python add_new_template.py <LLM_NAME> <LLM_TYPE> \
--llmmap_path ./data/pretrained_models/default \
--prompt_conf_path ./confs/prompt_configurations \
--num_prompt_confs 100
```
```LLM_TYPE``` tells the script which backend/client to use for the model (Hugging Face, OpenAI, or Anthropic). Values are:
```
Value | Backend
0 | Hugging Face
1 | OpenAI
2 | Anthropic
```
The higher ```--num_prompt_confs ``` the better, but more resource demanding.
At the moment, it supports only Hugging Face LLMs. But it will be extended soon.
Example of execution:
```
python add_new_template.py gpt-4.1 1 --llmmap_path=./data/pretrained_models/default
```
### Evaluate Accuracy
Added script to evaluate (top-k) accuracy of a pre-trained model:
```
python test_model.py ./data/pretrained_models/default -k 3
```
#### Supported models by default:
```
CohereForAI/aya-23-35B
CohereForAI/aya-23-8B
Deci/DeciLM-7B-instruct
HuggingFaceH4/zephyr-7b-beta
NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO
Qwen/Qwen2-1.5B-Instruct
Qwen/Qwen2-72B-Instruct
Qwen/Qwen2-7B-Instruct
Qwen/Qwen2.5-0.5B-Instruct
Qwen/Qwen2.5-3B-Instruct
abacusai/Smaug-Llama-3-70B-Instruct
claude-3-5-sonnet-20240620
claude-3-haiku-20240307
claude-3-opus-20240229
google/gemma-1.1-2b-it
google/gemma-1.1-7b-it
google/gemma-2-27b-it
google/gemma-2-9b-it
google/gemma-2b-it
google/gemma-7b-it
gpt-3.5-turbo
gpt-4-turbo-2024-04-09
gpt-4o-2024-05-13
gradientai/Llama-3-8B-Instruct-Gradient-1048k
ibm-granite/granite-3.0-8b-instruct
ibm-granite/granite-3.1-8b-instruct
internlm/internlm2_5-7b-chat
meta-llama/Llama-2-7b-chat-hf
meta-llama/Llama-3.2-1B-Instruct
meta-llama/Llama-3.2-3B-Instruct
meta-llama/Meta-Llama-3-70B-Instruct
meta-llama/Meta-Llama-3-8B-Instruct
meta-llama/Meta-Llama-3.1-70B-Instruct
meta-llama/Meta-Llama-3.1-8B-Instruct
microsoft/Phi-3-medium-128k-instruct
microsoft/Phi-3-medium-4k-instruct
microsoft/Phi-3-mini-128k-instruct
microsoft/Phi-3-mini-4k-instrExcerpt of 11,413 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:de6691a3432b548e, llm:Repository name 'LLMmap' suggests mapping LLMs; description/readme absent. Likely a catalog or mapping tool for large language models in Python. No topics provided.
matched fp:de6691a3432b548e, llm:Repository name 'LLMmap' suggests mapping LLMs; description/readme absent. Likely a catalog or mapping tool for large language models in Python. No topics provided.
matched fp:de6691a3432b548e, llm:Repository name 'LLMmap' suggests mapping LLMs; description/readme absent. Likely a catalog or mapping tool for large language models in Python. No topics provided.