Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
| Date | Stars |
|---|---|
| 2026-07-24 | 8734 |
| 2026-07-25 | 8737 |
| 2026-07-28 | 8737 |
| 2026-07-30 | 8737 |
| 2026-07-31 | 8744 |
| 2026-08-06 | 8744 |
Today
— stars today
This week
+7 stars this week
This month
— stars this month
Momentum
12.0
growth rate 0.08%/day
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://github.com/bentoml/BentoML/assets/489344/d3e6c95d-d224-49a5-9cff-0789f094e127">
<source media="(prefers-color-scheme: light)" srcset="https://github.com/bentoml/BentoML/assets/489344/de4da660-6aeb-4e5a-bf76-b7177435444d">
<img alt="BentoML: Unified Model Serving Framework" src="https://github.com/bentoml/BentoML/assets/489344/de4da660-6aeb-4e5a-bf76-b7177435444d" width="370" style="max-width: 100%;">
</picture>
## Unified Model Serving Framework
🍱 Build model inference APIs and multi-model serving systems with any open-source or custom AI models. 👉 [Join our forum](https://forum.modular.com/c/bento/31)!
[](https://github.com/bentoml/BentoML?tab=Apache-2.0-1-ov-file)
[](https://github.com/bentoml/bentoml/releases)
[](https://github.com/bentoml/BentoML/actions/workflows/ci.yml?query=branch%3Amain)
[](https://twitter.com/bentomlai)
## What is BentoML?
BentoML is a Python library for building online serving systems optimized for AI apps and model inference.
- **🍱 Easily build APIs for Any AI/ML Model.** Turn any model inference script into a REST API server with just a few lines of code and standard Python type hints.
- **🐳 Docker Containers made simple.** No more dependency hell! Manage your environments, dependencies and model versions with a simple config file. BentoML automatically generates Docker images, ensures reproducibility, and simplifies how you deploy to different environments.
- **🧭 Maximize CPU/GPU utilization.** Build high performance inference APIs leveraging built-in serving optimization features like dynamic batching, model parallelism, multi-stage pipeline and multi-model inference-graph orchestration.
- **👩💻 Fully customizable.** Easily implement your own APIs or task queues, with custom business logic, model inference and multi-model composition. Supports any ML framework, modality, and inference runtime.
- **🚀 Ready for Production.** Develop, run and debug locally. Seamlessly deploy to production with Docker containers or [BentoCloud](https://www.bentoml.com/).
## Getting started
Install BentoML:
```
# Requires Python≥3.9
pip install -U bentoml
```
Define APIs in a `service.py` file.
```python
import bentoml
@bentoml.service(
image=bentoml.images.Image(python_version="3.11").python_packages("torch", "transformers"),
)
class Summarization:
def __init__(self) -> None:
import torch
from transformers import pipeline
device = "cuda" if torch.cuda.is_available() else "cpu"
self.pipeline = pipeline('summarization', device=device)
@bentoml.api(batchable=True)
def summarize(self, texts: list[str]) -> list[str]:
results = self.pipeline(texts)
return [item['summary_text'] for item in results]
```
### 💻 Run locally
Install PyTorch and Transformers packages to your Python virtual environment.
```bash
pip install torch transformers # additional dependencies for local run
```
Run the service code locally (serving at http://localhost:3000 by default):
```bash
bentoml serve
```
You should expect to see the following output.
```
[INFO] [cli] Starting production HTTP BentoServer from "service:Summarization" listening on http://localhost:3000 (Press CTRL+C to quit)
[INFO] [entry_service:Summarization:1] Service Summarization initialized
```
Now you can run inference from your browser at http://localhost:3000 or with a Python script:
```python
import bentoml
with bentoml.SyncHTTPClient('http://localhost:3000') as client:
summarized_text: str = client.summarize([bentoml.__doc__])[0]
print(f"Result: {summarized_text}")
```
### 🐳 DepExcerpt of 9,873 characters
Read on GitHub
Chaoyu · United States
567
Aaron Pham · Canada
537
Frost Ming · China
482
@modular, previously @bentoml @douban
356
326
Sherlock Xu · @BentoML
276
Sauyon Lee · @bentoml
200
175
Sean Sheng · BentoML · United States
118
Zhao Shenyang · @bentoml
106
45
Tianxin Dong · @BentoML · United States
37
Jian Shen
36
35
xianxian.zhang · @bentoml
28
yetone · @Isoform · China
17
Jacky Zhao · @replit · United States
17
Steve Guo · Canada
16
Judah Rand · @DigitalInnovation
11
Jithin James · @vibrantlabsai · United States
11
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:019f58458d7fb6c7, topic:llm-inference, topic:model-serving, topic:llm-serving
matched fp:019f58458d7fb6c7, topic:mlops, topic:llmops