Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Efficient and general syntactical decoding for Large Language Models
| Date | Stars |
|---|---|
| 2026-07-31 | 338 |
| 2026-08-11 | 339 |
| 2026-08-24 | 338 |
| 2026-09-01 | 339 |
| 2026-09-20 | 339 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<p align="center">
<img width="400" alt="syncode" src="https://github.com/shubhamugare/syncode/assets/14147610/99c30a9d-b5f5-49ab-9295-33738fde1de2" />
</p>
# SynCode: LLM Generation with Grammar Augmentation [![Test Status][test-img]][tests]
<p align="left">
ℹ️ <a href="#-about">About</a>
| 📚 <a href="#-features">Features</a>
| 📖 <a href="#-more-about-syncode">More About SynCode</a>
| 🚀 <a href="#-quick-start">Quick Start</a>
| 👀 <a href="#-example-usage">Example Usage</a>
| 🤔 <a href="#-faq">FAQs</a>
</p>
<p>
<a href="https://arxiv.org/abs/2403.01632"><img src="https://img.shields.io/badge/Paper-arXiv-blue"></a>
</p>
## ℹ️ About
* **SynCode** is a novel framework for the grammar-guided generation of Large Language Models (LLMs) that is scalable to general-purpose programming languages and has soundness and completeness guarantees.
* With **SynCode**, you can ensure that your Language Model is generating output that is syntactically valid with respect to the rules defined by a Context-Free Grammar (CFG).
* For example, SynCode gets 99% accuracy in JSON generation with Gemma-2b (check [here](./notebooks/eval_json.ipynb)) and is 10-20% faster than standard unconstrained generation
## Builtin Grammars
    
Check [Grammars directory](./syncode/parsers/grammars) for supported grammars
Define your own grammar using simple EBNF syntax. Check out our [notebooks directory](./notebooks/) for examples and a quick example at
[<img align="center" src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/drive/1rYm8iehx_qYdtgWmqLkmhIjizhUVTb9E?usp=sharing)
## 📚 Features
| |
|---------------------------------------------------------------------------------------------------------------------|
| 🔥 Fast grammar-guided generation (as little as 10% generation overhead with Python and Go!) |
| 🤖 Seamlessly work with any HuggingFace Language Model, including Code, Chat, and Instruct models |
| 🖍️ Pass in any CFG in the EBNF format (even large grammars for programming languages like Python and Go!) |
| 📝 Built-in CFGs for **Python, Go, Java, SQL, Math, JSON**, and more! |
| 🎲 Sample with any existing decoding strategy (eg. greedy, beam search, nucleus sampling) |
## 🚀 Quick Start
### Python Installation and Usage Instructions
You can install SynCode via PyPI:
```bash
pip install syncode
```
Alternatively, you can install the latest development version directly from GitHub:
```bash
pip install git+https://github.com/structuredllm/syncode.git
```
#### Version Compatibility
SynCode depends on HuggingFace [transformers](https://github.com/huggingface/transformers):
| SynCode version | Required transformers version | Python version |
| -------------- | ----------------------------- | -------------- |
| `v0.4.16` (latest) | `v4.53.2` | 3.6 - 3.12 |
**Note:** Python 3.13 is not currently supported due to dependency constraints.
### Usage option 1:
SynCode can be used as a simple logit processor with HuggingFace [transformers](https://github.com/huggingface/transformers) library interface. Check this [notebook](./notebooks/example_logits_processor.ipynb) for example.
Just import with and initialize it with the appropriate grammar
``` python
from syncode import SyncExcerpt of 20,092 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:ff4f86c9e5b20886, topic:large-language-models, topic:llm
matched fp:ff4f86c9e5b20886, topic:llm-inference