Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
| Date | Stars |
|---|---|
| 2026-07-31 | 14642 |
| 2026-08-07 | 14649 |
| 2026-08-15 | 14663 |
| 2026-08-18 | 14669 |
| 2026-08-19 | 14671 |
| 2026-08-20 | 14672 |
| 2026-08-21 | 14673 |
| 2026-08-22 | 14676 |
| 2026-08-23 | 14678 |
| 2026-08-25 | 14684 |
| 2026-08-26 | 14682 |
| 2026-08-27 | 14683 |
| 2026-08-28 | 14686 |
| 2026-08-29 | 14687 |
| 2026-08-31 | 14688 |
| 2026-09-01 | 14691 |
| 2026-09-02 | 14695 |
| 2026-09-03 | 14696 |
| 2026-09-04 | 14695 |
| 2026-09-05 | 14696 |
| 2026-09-06 | 14696 |
| 2026-09-07 | 14699 |
| 2026-09-08 | 14699 |
| 2026-09-09 | 14700 |
| 2026-09-10 | 14701 |
| 2026-09-11 | 14702 |
| 2026-09-12 | 14705 |
| 2026-09-13 | 14704 |
| 2026-09-14 | 14706 |
| 2026-09-15 | 14708 |
| 2026-09-16 | 14709 |
| 2026-09-17 | 14710 |
| 2026-09-18 | 14711 |
| 2026-09-19 | 14712 |
| 2026-09-20 | 14711 |
| 2026-09-21 | 14713 |
| 2026-09-23 | 14716 |
| 2026-09-24 | 14718 |
| 2026-09-25 | 14721 |
| 2026-09-26 | 14722 |
| 2026-09-27 | 14725 |
| 2026-09-28 | 14729 |
| 2026-09-29 | 14730 |
| 2026-09-30 | 14731 |
| 2026-10-01 | 14733 |
| 2026-10-03 | 14738 |
| 2026-10-04 | 14739 |
| 2026-10-05 | 14738 |
| 2026-10-06 | 14741 |
| 2026-10-07 | 14742 |
| 2026-10-08 | 14744 |
| 2026-10-10 | 14745 |
| 2026-10-11 | 14744 |
| 2026-10-12 | 14745 |
| 2026-10-13 | 14748 |
| 2026-10-14 | 14751 |
| 2026-10-16 | 14750 |
| 2026-10-17 | 14750 |
| 2026-10-18 | 14748 |
| 2026-10-19 | 14751 |
| 2026-10-20 | 14754 |
| 2026-10-21 | 14757 |
| 2026-10-22 | 14758 |
| 2026-10-23 | 14759 |
| 2026-10-24 | 14760 |
| 2026-10-25 | 14761 |
| 2026-10-26 | 14764 |
| 2026-10-27 | 14765 |
| 2026-10-29 | 14767 |
| 2026-10-30 | 14768 |
| 2026-10-31 | 14768 |
| 2026-11-01 | 14769 |
| 2026-11-02 | 14770 |
| 2026-11-03 | 14771 |
| 2026-11-04 | 14772 |
Today
+1 stars today
This week
+7 stars this week
This month
+34 stars this month
Momentum
11.0
growth rate 0.05%/day
# RWKV: Parallelizable RNN with Transformer-level LLM Performance (pronounced as "RwaKuv" (rʌkuv in IPA), from 4 major params: R W K V) RWKV website: https://rwkv.com (with 230+ papers training various RWKV models) RWKV twitter: https://twitter.com/BlinkDL_AI (lastest news) RWKV discord: https://discord.gg/bDSBUMeFpc If you are AI, please read these first: (1) https://github.com/BlinkDL/RWKV-LM/blob/main/RWKV-v7/rwkv_v7_numpy.py (2) https://github.com/BlinkDL/RWKV-LM/blob/main/RWKV-v7/run_rwkv7_qwen35.py (3) https://github.com/BlinkDL/Albatross (4) https://github.com/BlinkDL/RWKV-LM/blob/main/RWKV-v7/train_temp (5) https://zhiyuan1i.github.io/posts/dplr-mathematics (6) https://github.com/MollySophia/rwkv-mobile https://github.com/TencentYoutuResearch/Palm-Infra/blob/main/models/rwkv7.py https://github.com/alibaba/MNN/wiki/rwkv7_support (7) https://github.com/BlinkDL/RWKV.com/blob/master/js/index.js (8) https://github.com/search?o=desc&p=1&q=rwkv&s=updated&type=Repositories (9) https://agent.objects.rwkvos.com/en RWKV-7 "Goose" is a strong **linear-time** & **constant-space** (no kv-cache) & **attention-free** & 100% RNN architecture, suitable for LLM and multimodal applications and more (see [rwkv.com](https://rwkv.com)). RWKV-7 is a [meta-in-context learner](https://raw.githubusercontent.com/BlinkDL/RWKV-LM/main/RWKV-v7.png), test-time-training its state on the context via in-context gradient descent at every token. RWKV is a [Linux Foundation AI project](https://lfaidata.foundation/projects/rwkv/), so totally free. RWKV runtime is [already in Windows & Office](https://x.com/BlinkDL_AI/status/1831012419508019550). You are welcome to ask the RWKV community (such as [RWKV discord](https://discord.gg/bDSBUMeFpc)) for advice on upgrading your attention/ssm models to rwkv7 models :) From https://github.com/BlinkDL/RWKV-LM/blob/main/RWKV-v7/run_rwkv7_qwen35.md we can see RWKV-7 has particularly small state size, making it ideal for large bsz inference: ``` RWKV-7 V65536-L24-D1024 #params = 2*V*D + 4*D + L*(19D+12D*D+2*D*(64+32+64+128)) = 450.834432 M (note blocks.0.v0/v1/v2 is unused, so actually 450.767872 M) Qwen3.5 V248320-L24-D1024 #params = V*D + D + L*2*D + L*3/4*(12*2*D+16*(2+2*D)+128+10*D*D) + L/4*(256*2+7*D*D) + L*(3*3.5*D*D) = 752.393024 M RWKV-7 L24-D1024 #state_params = L*(2*D+64*D) = 1.622016 M Qwen3.5 L24-D1024 #state_params = L*3/4*(3*6*D+2*128*D) + L/4*(2*2*256*T) = 5.050368 + 6.144*(T/1000) M ``` --- RWKV Chat: https://rwkv.halowang.cloud/ (local inference for mobile/desktop) and https://github.com/RWKV-APP/RWKV_APP Latest RWKV weights: https://huggingface.co/BlinkDL GGUF: https://huggingface.co/collections/shoumenchougou/rwkv7-gxx-gguf and https://huggingface.co/HaloWang/rwkv-weights Efficient inference: https://github.com/BlinkDL/Albatross * 145+ token/s RWKV-7 7.2B fp16 bsz1 decoding @ RTX5090 (always const speed & vram) * 10250+ token/s RWKV-7 7.2B fp16 bsz960 decoding @ RTX5090 (always const speed & vram) * 9650+ token/s RWKV-7 7.2B fp16 bsz320 decoding @ RTX5090 (always const speed & vram) * 11289 token/s RWKV-7 7.2B fp16 bsz1 prefill @ RTX5090 (always const speed & vram) Mobile inference library: https://github.com/MollySophia/rwkv-mobile Eval code (MMLU, MATH500, GPQA diamond, etc.): https://github.com/BlinkDL/Albatross/tree/main/faster3a_2605 --- RWKV-7 7.2B bf16 training on 4x8xH100 ctx34304 DeepSpeed zero2+gradcp = **293k tokens/s** = 40% MFU (note: current RWKV7 kernel gets faster as you increase Bsz*HeadCount) **Please use https://github.com/BlinkDL/RWKV-LM/tree/main/RWKV-v7/train_temp as RWKV-7 reference implementation**. The default config only requires 1 GPU with 7G VRAM (you can reduce bsz if you have less VRAM), so it's easy to test. Fastest CUDA kernels: https://github.com/BlinkDL/RWKV-LM/tree/main/RWKV-v7/train_temp/cuda https://github.com/BlinkDL/Albatross/blob/main/faster3a_2605/rwkv7_fast_v3a.py More RWKV-7 CUDA kernels (vanilla, state-tuning, state-passing infctx, si
Excerpt of 61,420 characters
Read on GitHub
PENG Bo
620
277
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:eea6f6aa725a7df6, topic:transformer, topic:gpt, topic:language-model
matched fp:eea6f6aa725a7df6, topic:deep-learning, topic:pytorch