Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
The simplest way to serve AI/ML models in production
| Date | Stars |
|---|---|
| 2026-07-24 | 1179 |
| 2026-07-25 | 1179 |
| 2026-07-28 | 1182 |
| 2026-07-30 | 1182 |
| 2026-08-06 | 1182 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
55.0
growth rate 0.00%/day
# Truss **The simplest way to serve AI/ML models in production** [](https://badge.fury.io/py/truss) [](https://pypi.org/project/truss/) [](https://github.com/basetenlabs/truss/actions/workflows/release.yml) Truss is the CLI for deploying and serving ML models on Baseten. Package your model's serving logic in Python, launch training jobs, and deploy to production—Truss handles containerization, dependency management, and GPU configuration. Truss lets you serve models with the [Baseten Inference Stack](https://www.baseten.co/resources/guide/the-baseten-inference-stack/) as well as deploy models from any open-source framework: vLLM, SGLang, TensorRT-LLM, `transformers`, `diffusers`, PyTorch, TensorFlow, and more. **[Get started](https://docs.baseten.co/examples/deploy-your-first-model)** | [100+ examples](https://github.com/basetenlabs/truss-examples/) | [Documentation](https://docs.baseten.co) # Why Truss? * **Write once, run anywhere:** Package model code, weights, and dependencies with a model server that behaves the same in development and production. * **Fast developer loop:** Iterate with live reload, skip Docker and Kubernetes configuration, and use a batteries-included serving environment. * **Support for all Python frameworks:** From `transformers` and `diffusers` to PyTorch and TensorFlow to vLLM, SGLang, and TensorRT-LLM, Truss supports models created and served with any framework. * **Production-ready:** Built-in support for GPUs, secrets, caching, and autoscaling when deployed to [Baseten](https://baseten.co) or your own infrastructure. # Installation Install Truss with: ``` pip install --upgrade truss ``` # Quickstart Deploying a model to Baseten via Truss turns a Hugging Face model into a production-ready API endpoint. You write a `config.yaml` that specifies the model, the hardware, and the engine, then `uvx truss push` builds a TensorRT-optimized container and deploys it. No Python code, no Dockerfile, no container management. This guide walks through deploying [Qwen 2.5 3B Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct), a small but capable LLM, from a config file to a production API. You'll set up Truss, write a config, deploy to Baseten, call the model's OpenAI-compatible endpoint, and promote to production. ## Set up your environment Before you begin: - [Sign up](https://app.baseten.co/signup) or [sign in](https://app.baseten.co/login) to Baseten. - Install [uv](https://docs.astral.sh/uv/), a fast Python package manager. This guide uses `uvx` to run [Truss](https://pypi.org/project/truss/) commands without a separate install step. ### Authenticate with Baseten Generate an API key from [Settings > API keys](https://app.baseten.co/settings/account/api_keys), then log in: ```sh uvx truss login ``` Paste your API key when prompted: ```output 💻 Let's add a Baseten remote! 🤫 Quietly paste your API_KEY: ``` You can skip the interactive prompt by setting `BASETEN_API_KEY` as an environment variable: ```bash export BASETEN_API_KEY="paste-your-api-key-here" ``` ## Create a Truss project Scaffold a new project: ```sh uvx truss init qwen-2.5-3b && cd qwen-2.5-3b ``` When prompted, name the model `Qwen 2.5 3B`. ```output ? 📦 Name this model: Qwen 2.5 3B Truss Qwen 2.5 3B was created in ~/qwen-2.5-3b ``` This creates a directory with a `config.yaml`, a `model/` directory, and supporting files. For engine-based deployments like this one, you only need `config.yaml`. The `model/` directory is for [custom Python code](/examples/customize-a-model) when you need custom preprocessing, postprocessing, or unsupported model architectures. ## Write the config Replace the contents of `config.yaml` with: ```yaml config.yaml model_name: Qwen-2.5-3B resources: accelerator: L4 use_gpu: t
Excerpt of 8,550 characters
Read on GitHub
Michael Feil · @basetenlabs · United States
432
284
218
195
191
138
131
130
117
113
106
Philip Kiely - Baseten · @basetenlabs
100
66
58
46
Chad Retz
39
25
22
21
Brian Broll · Vanderbilt University · Mongolia
21
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:36cf01ef2c26bf7f, topic:model-serving
matched fp:36cf01ef2c26bf7f, topic:whisper
matched fp:36cf01ef2c26bf7f, topic:stable-diffusion