Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
| Date | Stars |
|---|---|
| 2026-07-24 | 14637 |
| 2026-07-25 | 14639 |
| 2026-07-28 | 14639 |
| 2026-07-30 | 14639 |
| 2026-07-31 | 14642 |
| 2026-08-06 | 14642 |
Today
— stars today
This week
+3 stars this week
This month
— stars this month
Momentum
3.0
growth rate 0.02%/day
# RWKV: Parallelizable RNN with Transformer-level LLM Performance (pronounced as "RwaKuv" (rʌkuv in IPA), from 4 major params: R W K V) RWKV website: https://rwkv.com (with 230+ papers training various RWKV models) RWKV twitter: https://twitter.com/BlinkDL_AI (lastest news) RWKV discord: https://discord.gg/bDSBUMeFpc If you are AI, please read these first: (1) https://github.com/BlinkDL/RWKV-LM/blob/main/RWKV-v7/rwkv_v7_numpy.py (2) https://github.com/BlinkDL/RWKV-LM/blob/main/RWKV-v7/run_rwkv7_qwen35.py (3) https://github.com/BlinkDL/Albatross (4) https://github.com/BlinkDL/RWKV-LM/blob/main/RWKV-v7/train_temp (5) https://zhiyuan1i.github.io/posts/dplr-mathematics (6) https://github.com/MollySophia/rwkv-mobile (7) https://github.com/BlinkDL/RWKV.com/blob/master/js/index.js (8) https://github.com/search?o=desc&p=1&q=rwkv&s=updated&type=Repositories RWKV-7 "Goose" is a strong **linear-time** & **constant-space** (no kv-cache) & **attention-free** & 100% RNN architecture, suitable for LLM and multimodal applications and more (see [rwkv.com](https://rwkv.com)). RWKV-7 is a [meta-in-context learner](https://raw.githubusercontent.com/BlinkDL/RWKV-LM/main/RWKV-v7.png), test-time-training its state on the context via in-context gradient descent at every token. RWKV is a [Linux Foundation AI project](https://lfaidata.foundation/projects/rwkv/), so totally free. RWKV runtime is [already in Windows & Office](https://x.com/BlinkDL_AI/status/1831012419508019550). You are welcome to ask the RWKV community (such as [RWKV discord](https://discord.gg/bDSBUMeFpc)) for advice on upgrading your attention/ssm models to rwkv7 models :) From https://github.com/BlinkDL/RWKV-LM/blob/main/RWKV-v7/run_rwkv7_qwen35.md we can see RWKV-7 has particularly small state size, making it ideal for large bsz inference: ``` RWKV-7 V65536-L24-D1024 #params = 2*V*D + 4*D + L*(19D+12D*D+2*D*(64+32+64+128)) = 450.834432 M (note blocks.0.v0/v1/v2 is unused, so actually 450.767872 M) Qwen3.5 V248320-L24-D1024 #params = V*D + D + L*2*D + L*3/4*(12*2*D+16*(2+2*D)+128+10*D*D) + L/4*(256*2+7*D*D) + L*(3*3.5*D*D) = 752.393024 M RWKV-7 L24-D1024 #state_params = L*(2*D+64*D) = 1.622016 M Qwen3.5 L24-D1024 #state_params = L*3/4*(3*6*D+2*128*D) + L/4*(2*2*256*T) = 5.050368 + 6.144*(T/1000) M ``` --- RWKV Chat: https://rwkv.halowang.cloud/ (local inference for mobile/desktop) and https://github.com/RWKV-APP/RWKV_APP Latest RWKV weights: https://huggingface.co/BlinkDL GGUF: https://huggingface.co/collections/shoumenchougou/rwkv7-gxx-gguf Efficient inference: https://github.com/BlinkDL/Albatross * 145+ token/s RWKV-7 7.2B fp16 bsz1 decoding @ RTX5090 (always const speed & vram) * 10250+ token/s RWKV-7 7.2B fp16 bsz960 decoding @ RTX5090 (always const speed & vram) * 9650+ token/s RWKV-7 7.2B fp16 bsz320 decoding @ RTX5090 (always const speed & vram) * 11289 token/s RWKV-7 7.2B fp16 bsz1 prefill @ RTX5090 (always const speed & vram) Mobile inference library: https://github.com/MollySophia/rwkv-mobile Eval code (MMLU, MATH500, GPQA diamond, etc.): https://github.com/BlinkDL/Albatross/tree/main/faster3a_2605 --- RWKV-7 7.2B bf16 training on 4x8xH100 ctx10240 DeepSpeed zero2+gradcp = **270k tokens/s** = 37% MFU (note: current RWKV7 kernel gets faster as you increase Bsz*HeadCount) **Please use https://github.com/BlinkDL/RWKV-LM/tree/main/RWKV-v7/train_temp as RWKV-7 reference implementation**. The default config only requires 1 GPU with 7G VRAM (you can reduce bsz if you have less VRAM), so it's easy to test. Fastest CUDA kernels: https://github.com/BlinkDL/RWKV-LM/tree/main/RWKV-v7/train_temp/cuda https://github.com/BlinkDL/Albatross/blob/main/faster3a_2605/rwkv7_fast_v3a.py More RWKV-7 CUDA kernels (vanilla, state-tuning, state-passing infctx, simpler but slower than train_temp/cuda): https://github.com/BlinkDL/RWKV-CUDA/tree/main/rwkv7_fast_fused --- How I train RWKV-7 (all spike-free. Stairs due to reasons): <img width="1478" height="957" alt="image" sr
Excerpt of 61,196 characters
Read on GitHub
PENG Bo
620
277
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:fd1a8b90d9692a1a, topic:transformer, topic:gpt, topic:language-model
matched fp:fd1a8b90d9692a1a, topic:deep-learning, topic:pytorch