Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning
| Date | Stars |
|---|---|
| 2026-07-31 | 468 |
| 2026-08-01 | 468 |
| 2026-08-02 | 468 |
| 2026-08-06 | 468 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<hr>
<div align="center">
<picture>
<img src="assets/header.png" width="60%" alt="CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning">
</picture>
</div>
<h1 align="center" style="line-height: 1.3;">
CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning
</h1>
<!-- -->
## 🥳 Introduction
**CUDA-L2** is a system that combines large language models (LLMs) and reinforcement learning (RL) to automatically optimize Half-precision General Matrix Multiply (HGEMM) CUDA kernels. CUDA-L2 systematically outperforms major matmul baselines to date, from the widely-used torch.matmul to state-of-the-art NVIDIA closed-source libraries (cuBLAS, cuBLASLt-heuristic, cuBLASLt-AutoTuning). <a href="https://arxiv.org/pdf/2512.02551">Paper</a>
<div align="center">
<img src="assets/speedup_summary_all.png" width="90%" alt="Speedup Summary">
<br>
<em>Summary of CUDA-L2 speedup over baselines across all GPU configurations (RTX 3090-F32F16F16F32, A100-F16F16F16F16, A100-F32F16F16F32, H100-F32F16F16F32) in Offline and Server modes.</em>
</div>
## 🎉 What's New
- **[Mar 30, 2026]** Released H100 HGEMM (16 bit) kernels with 32-bit accumulator (F32F16F16F32). 🎉🎉🎉
<div align="center">
| Mode | vs torch.matmul | vs cuBLAS | vs cuBLASLt-heuristic | vs cuBLASLt-AutoTuning |
|:----:|:---------------:|:---------:|:--------------------:|:----------------------:|
| Offline | +19.8% | +17.5% | +28.8% | +7.0% |
| Server | +41.7% | +40.5% | +42.1% | +22.1% |
</div>
- **[Mar 23, 2026]** Released RTX 3090 HGEMM (16 bit) kernels with 32-bit accumulator (SM80_16x8x16_F32F16F16F32). 🎉🎉🎉
<div align="center">
| Mode | vs torch.matmul | vs cuBLAS | vs cuBLASLt-heuristic | vs cuBLASLt-AutoTuning |
|:----:|:---------------:|:---------:|:--------------------:|:----------------------:|
| Offline | +21.3% | +24.2% | +21.1% | +12.7% |
| Server | +28.7% | +35.3% | +28.1% | +19.8% |
</div>
- **[Jan 7, 2026]** Released 1,000 A100 HGEMM (16 bit) kernels with 32-bit accumulator (SM80_16x8x16_F32F16F16F32). 🎉🎉🎉
<div align="center">
| Mode | vs torch.matmul | vs cuBLAS | vs cuBLASLt-heuristic | vs cuBLASLt-AutoTuning |
|:----:|:---------------:|:---------:|:--------------------:|:----------------------:|
| Offline | +22.5% | +20.4% | +16.9% | +12.2% |
| Server | +25.1% | +21.5% | +19.9% | +12.7% |
</div>
- **[Dec 2, 2025]** Released A100 optimized HGEMM (16 bit) kernels across 1,000 configurations with 16-bit accumulator (SM80_16x8x16_F16F16F16F16).
<div align="center">
| Mode | vs torch.matmul | vs cuBLAS | vs cuBLASLt-heuristic | vs cuBLASLt-AutoTuning |
|:----:|:---------------:|:---------:|:--------------------:|:----------------------:|
| Offline | +22.0% | +19.2% | +16.8% | +11.4% |
| Server | +28.7% | +26.0% | +22.4% | +15.9% |
</div>
## 🗒️ To-Do List
- [x] Release HGEMM with 32-bit accumulator (F32F16F16F32 officially) for A100.
- [x] Release HGEMM with 32-bit accumulator (F32F16F16F32 officially) for 3090.
- [x] Release HGEMM with 32-bit accumulator (F32F16F16F32 officially) for H100.
- [ ] Support denser matrix configurations (more configurations).
- [ ] Extend to more GPUs (Ada Lovelace, Hopper, Blackwell).
- [ ] Easy deployment for open-source LLMs.
## FAQ
**Q: Do A100 kernels apply to other machines like RTX 3090 or H100?**
A: Ideally, kernels trained on A100 should only be used on A100 if you are targeting speedup. They might have speedup on other machines, but it's not guaranteed. We will progressively release kernels trained on different machines.
**Q: What if I need matrix dimensions (M, N, K) not found in your configurations?**
A: 1. You can find the nearest neighbor configuration (larger than yours) and pad with zeros.
2. Feel free to post your dimensions on GitHub issues. We are happy to release kernels for your configuration.
## Installation & Setup
### 1\. Prerequisites
* *Excerpt of 7,050 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:8e3f9d2752cc2e9e, llm:Repository title and description: 'CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning'; topics include cublas, cuda-kernels, large-language-models, matrix-multiplication, nvidia, reinforcement-learning. Language: Cuda.
matched fp:8e3f9d2752cc2e9e, llm:Repository title and description: 'CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning'; topics include cublas, cuda-kernels, large-language-models, matrix-multiplication, nvidia, reinforcement-learning. Language: Cuda.
matched fp:8e3f9d2752cc2e9e, llm:Repository title and description: 'CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning'; topics include cublas, cuda-kernels, large-language-models, matrix-multiplication, nvidia, reinforcement-learning. Language: Cuda.