Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
A benchmark and harness for finding and exploiting smart contract bugs
| Date | Stars |
|---|---|
| 2026-07-24 | 435 |
| 2026-07-25 | 436 |
| 2026-07-28 | 436 |
| 2026-07-30 | 436 |
| 2026-08-06 | 436 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<p align="center">
<picture align="center">
<img alt="evmbench cover" src="assets/cover-dark.png">
</picture>
</p>
**evmbench is a benchmark and agent harness for finding and exploiting smart contract bugs.**
<a href="#how-it-works"><b><u>How it works</u></b></a> | <a href="#security"><b><u>Security</u></b></a> | <a href="#key-services"><b><u>Key services</u></b></a> | <a href="#repo-layout"><b><u>Repo layout</u></b></a> | <a href="#quickstart-local-dev"><b><u>Quickstart (local dev)</u></b></a>
This repository contains a companion interface to the `evmbench` detect evaluation ([code](https://github.com/openai/frontier-evals)). For reference, we include the evaluation code as a pinned submodule at `frontier-evals/`.
Upload contract source code, select an agent, and receive a structured vulnerability report rendered in the UI.
## How it works
### Architecture
```
Frontend (Next.js)
│
├─ POST /v1/jobs/start ───► Backend API (FastAPI, port 1337)
│ ├─► PostgreSQL (job state)
├─ GET /v1/jobs/{id} ├─► Secrets Service (port 8081)
│ └─► RabbitMQ (job queue)
└─ GET /v1/jobs/history │
▼
Instancer (consumer)
│
┌─────────┴──────────┐
▼ ▼
Docker backend K8s backend (optional)
│ │
└────────┬───────────┘
▼
Worker container
├─► Secrets Service (fetch bundle)
├─► (optional) OAI Proxy (port 8084) ──► OpenAI API
└─► Results Service (port 8083)
```
### End-to-end flow
1. User uploads a zip of contract files via the frontend. The UI sends the archive, selected model key, and (optionally) an OpenAI API key to `/v1/jobs/start`.
2. The backend creates a job record in Postgres, stores a secret bundle in the Secrets Service, and publishes a message to RabbitMQ.
3. The Instancer consumes the job and starts a worker (Docker locally; Kubernetes backend is optional).
4. The worker fetches its bundle from the Secrets Service, unpacks the uploaded zip to `audit/`, then runs Codex in "detect-only" mode:
- prompt: `backend/worker_runner/detect.md` (copied to `$HOME/AGENTS.md` inside the container)
- model map: `backend/worker_runner/model_map.json` (maps UI model keys to Codex model IDs)
- command wrapper: `backend/worker_runner/run_codex_detect.sh`
5. The agent writes `submission/audit.md`. The worker validates that the output contains parseable JSON with `{"vulnerabilities": [...]}` and then uploads it to the Results Service.
6. The frontend polls job status and renders the report with file navigation and annotations.
## Security
`evmbench` runs an LLM-driven agent against uploaded, untrusted code. Treat the worker runtime (filesystem, logs, outputs) as an untrusted environment.
See `SECURITY.md` for the full trust model and operational guidance.
OpenAI credential handling:
- **Direct BYOK (default)**: worker receives a plaintext OpenAI key (`OPENAI_API_KEY` / `CODEX_API_KEY`).
- **Proxy-token mode (optional)**: worker receives an opaque token and routes requests through `oai_proxy` (plaintext key stays outside the worker).
Enabling proxy-token mode:
```bash
cd backend
cp .env.example .env
# set BACKEND_OAI_KEY_MODE=proxy and OAI_PROXY_AES_KEY=...
docker compose --profile proxy up -d --build
```
Operational note: worker runtime is bounded by default; override the max audit runtime with `EVM_BENCH_CODEX_TIMEOUT_SECONDS` (default: 10800 seconds).
## Key services
| SeExcerpt of 6,622 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:70e79ad04e52ba73, topic:testing
matched fp:70e79ad04e52ba73, topic:agents