Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
A workload for deploying LLM inference services on Kubernetes
| Date | Stars |
|---|---|
| 2026-07-31 | 267 |
| 2026-08-05 | 269 |
| 2026-08-06 | 269 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
35.0
growth rate 0.00%/day
# RoleBasedGroup (RBG) 🚀
English | [简体中文](./README-zh_CN.md)
[](https://github.com/sgl-project/rbg/blob/main/LICENSE)
[](https://github.com/sgl-project/rbg/releases)
[](https://goreportcard.com/report/github.com/sgl-project/rbg)
> 🎯 A Kubernetes API for orchestrating distributed, stateful AI inference workloads with **multi-role collaboration** and **built-in service discovery**.
**🌐 Official Website**: [rolebasedgroup.github.io](https://rolebasedgroup.github.io)
---
## 🏗️ Architecture
---
## 📰 Latest News
| Date | Release | Highlights |
|:----:|:-------:|:-----------|
| 2026-06-11 | [v0.7.0](https://github.com/sgl-project/rbg/releases/tag/v0.7.0) | `v1alpha2` API stable release, conversion webhooks, CLI multi-node LLM serving, pod port allocator, coordinated policies, gang scheduling |
| 2026-02-18 | [v0.6.0](https://github.com/sgl-project/rbg/releases/tag/v0.6.0) | Coordinated scaling, stateful InstanceSet |
| 2025-12-03 | [v0.5.0](https://github.com/sgl-project/rbg/releases/tag/v0.5.0) | Native InstanceSet, in-place updates, Mooncake integration |
| 2025-09-23 | [v0.4.0](https://github.com/sgl-project/rbg/releases/tag/v0.4.0) | RBGS scaling, Volcano podgroup support |
---
## 🤔 Why RBG?
Traditional Kubernetes primitives (StatefulSets / Deployments) struggle with LLM inference services that:
| Challenge | Description |
|:---------:|:------------|
| Multi-role topologies | gateway → router → prefill → decode |
| Performance-sensitive | GPU/network topology matters |
| Atomic operations | deploy, upgrade, scale, failover across roles |
**RBG** treats an inference service as a **role-based group** — a topologized, stateful, coordinated multi-role organism managed as a single unit.
---
## 🎯 Key Concepts
| Concept | Description |
|:--------|:------------|
| **Role** | Basic scheduling and rollout unit. Each role (prefill, decode) has its own spec, lifecycle and policies. |
| **RoleBasedGroup** | A group of roles forming one logical service (e.g., one LLM inference deployment). |
| **RoleInstance** | A collection of Pods with tightly bound lifecycle. Supports in-place updates and controls upgrades/status for the Pod group. |
| **CoordinatedPolicy** | A separate CRD for coordinating operations across roles. Controls `maxSkew` and `progression` during rolling updates and scaling. |
---
## ✨ Key Features — SCOPE
| Capability | Description |
|:-----------|:------------|
| **Stable** | Topology-aware deterministic operations with unique RoleID injection |
| **Coordination** | Cross-role policy engine: deployment pairing, coordinated upgrades, linked recovery |
| **Orchestration** | Role dependencies, precise startup sequences, topology self-aware service discovery |
| **Performance** | Hardware affinity scheduling: GPU-NVLink → PCIe → RDMA → VPC |
| **Extensible** | Declarative APIs and plugin mechanisms for future architectures |
---
## 🚀 Getting Started
### 📦 Installation
Install from [GitHub Releases](https://github.com/sgl-project/rbg/releases) (latest version):
```shell
VERSION=$(curl -sL https://api.github.com/repos/sgl-project/rbg/releases/latest | grep '"tag_name"' | sed -E 's/.*"v([^"]+)".*/\1/')
helm upgrade --install rbgs https://github.com/sgl-project/rbg/releases/download/v$VERSION/rbgs-$VERSION.tgz \
--namespace rbgs-system --create-namespace --wait
```
For detailed instructions, see [Installation Guide](doc/install.md).
### 🎮 Quick Start
Deploy a basic RoleBasedGroup with two roles and startup dependencies:
```yaml
apiVersion: workloads.x-k8s.io/v1alpha2
kind: RoleBasedGroup
metadata:
name: nginx-cluster
spec:
roles:
- name: frontend
replicas: 1
standalonePattern:
template:
Excerpt of 12,741 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:b8be59a9c89944fd, topic:llm