Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[NeurIPS 2025 Main] SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications
| Date | Stars |
|---|---|
| 2026-07-31 | 1097 |
| 2026-08-03 | 1097 |
| 2026-08-06 | 1097 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# BIRD-CRITIC 1.0 (SQL) <img src="materials/hku-logo.jpg" alt="HKU Logo" width="50" style="vertical-align:middle;margin-left:10px;"> <img src="materials/google-cloud-logo.png" alt="Google Cloud Logo" width="50" style="vertical-align:middle;margin-left:10px;">
<p align="center">
<img src="materials/red_bird_single.png"
style="width: 30%; min-width: 100px; display: block; margin: auto; border-radius: 15px !important;">
</p>
<div style="display: flex; justify-content: center; align-items: center; gap: 10px;">
<a href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">
<img src="https://img.shields.io/badge/License-CC%20By%20SA%204.0-orange.svg" alt="License">
</a>
<a href="https://bird-critic.github.io/">
<img src="https://img.shields.io/badge/Leaderboard-2025-28a745.svg" alt="Leaderboard">
</a>
<a href="https://huggingface.co/datasets/birdsql/bird-critic-1.0-flash-exp/">
<img src="https://img.shields.io/badge/Dataset-HuggingFace-FFD21E.svg" alt="HuggingFace">
</a>
<a href="https://www.python.org/downloads/release/python-310/">
<img src="https://img.shields.io/badge/Python-3.10+-teal.svg" alt="Python">
</a>
<a href="https://pypi.org/project/openai/">
<img src="https://img.shields.io/badge/OpenAI-1.40+-beige.svg" alt="OpenAI">
</a>
</div>
## News
- 📢 **[2026-03-24]** We release the **BIRD-Critic GT & Test Cases Auto-Reply System**, supporting automatic delivery of full datasets (including sol_sql and test_cases) via email requests to 📧 [email protected] with subject tag [bird-critic-1 GT&Test Cases].
- 📢 **[2026-03-23]** We release [**BIRD-Critic-SQLite**](https://huggingface.co/datasets/birdsql/bird-critic-1.0-sqlite), a dataset containing 500 high-quality user issues focused on real-world SQLite database applications.
- 📢 **[2026-01-08]** We recently updated the evaluation code (specifically the `remove_distinct` function). Please pull the latest code from our GitHub repository.
- 📢 **[2025-07-09]** We release the human performance scores on our [**website**](https://bird-critic.github.io/)! The scores displayed across all three leaderboards reflect human evaluators (database experts) who were allowed to use standard tools (database textbooks, official documentation, or IDEs) but not AI assistants. When another group with the same expertise was permitted to use AI tools (ChatGPT, Claude, or Gemini), the performance increased to **83.33** on Open, **87.90** on PG, and **90.00** on Flash, demonstrating the significant potential of human-AI collaboration in SQL problem-solving.
- 📢 **[2025-06-28]** We release our paper [**SWE-SQL** (a.k.a BIRD-CRITIC)](https://arxiv.org/abs/2506.18951) on arxiv.
- 📢 **[2025-06-09]** We release [**bird-interact-lite**](https://huggingface.co/datasets/birdsql/bird-interact-lite), feature multi-turn conversational and agentic interaction for real-world ambiguous and challenging text-to-SQL tasks.
- 📢 **[2025-06-08]** We release [**bird-critic-1.0-postgresql**](https://huggingface.co/datasets/birdsql/bird-critic-1.0-postgresql), a single-dialect SQL issue debugging set with 530 complex tasks.
- 📢 **[2025-05-30]** We are pleased to release [**LiveSQLBench-Base-Lite**](https://huggingface.co/datasets/birdsql/livesqlbench-base-lite), featuring 18 end-user level databases and 270 tasks (180 SELECT-only, 90 Management tasks). Each task involves unambiguous and straightforward user queries grounded in external knowledge, with medium to hard complexity SQL statements.
## 🧸 Overview
BIRD-Critic 1.0 introduces a novel SQL benchmark designed to evaluate a key capability: **Can large language models (LLMs) diagnose and solve user issues within real-world database environments?**
The benchmark comprises 600 tasks for development and 200 held-out out-of-distribution (OOD) tests. BIRD-CRITIC 1.0 is built on realistic user issues across 4 prominent open-source SQL dialects: MySQL, PostgreSQL, SQL Server, and Oracle. It expands beyond Excerpt of 13,591 characters
Read on GitHub
59
14
8
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:46c41df18c560b3b, llm:Repository description: 'SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications' (NeurIPS 2025 Main). Language: Python.
matched fp:46c41df18c560b3b, llm:Repository description: 'SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications' (NeurIPS 2025 Main). Language: Python.
matched fp:46c41df18c560b3b, llm:Repository description: 'SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications' (NeurIPS 2025 Main). Language: Python.