Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Speed Always Wins: A Survey on Efficient Architectures for Large Language Models
| Date | Stars |
|---|---|
| 2026-07-24 | 409 |
| 2026-07-25 | 409 |
| 2026-07-28 | 409 |
| 2026-07-30 | 409 |
| 2026-07-31 | 408 |
| 2026-08-06 | 408 |
Today
— stars today
This week
-1 stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<div align="center"> # Awesome-Efficient-Arch [](https://awesome.re) [](https://arxiv.org/pdf/2503.09567) [](https://github.com/weigao266/Awesome-Efficient-Arch) [](https://github.com/weigao266/Awesome-Efficient-Arch) []() </div> ## 🔥 News - **2025.08**: 📢📢📢 The survey paper has been reported by many technical media, including: [Zhihu (知乎)](https://zhuanlan.zhihu.com/p/1944797944537646237), [Synced (机器之心)](https://mp.weixin.qq.com/s/o3U2IM4aLECZ6jVEzIbtcQ), [PaperWeekly](https://mp.weixin.qq.com/s/X0UED7W_Cm5fNvdnM7bcqA), [52CV (我爱计算机视觉)](https://mp.weixin.qq.com/s/1xc_WPhEOCuvnbMzkV-etg), [Deep Learning and NLP (深度学习自然语言处理)](https://mp.weixin.qq.com/s/wQOyd9Csd2hcIEiILSQs_w), [Emergent Clustering Points (涌现聚点)](https://mp.weixin.qq.com/s/fv4qotiMEPqEqySB7tNPSQ), [Global Economic Forum (全球经济论坛)](https://mp.weixin.qq.com/s/x2wH4XEvmI72leNgg6VpeA), and more. People ❤️ "Speed Always Wins"! - **2025.08**: 🎉🎉🎉 We have released a survey paper [Speed Always Wins: A Survey on Efficient Architectures for Large Language Models](https://arxiv.org/pdf/2508.09834), with 449 papers included. Please feel free to open PRs to include your **Awesome-Efficient-Arch** work. ## ✨ Overview <p align="center"> <img src="assets/title.png" width="100%" /> </p> <p align="center"> <img src="assets/overall.png" width="100%" /> </p> <p align="center"> <img src="assets/tree.png" width="100%" /> </p> ## 📄 Paper List ### Table of Contents - 1 [Introduction](#introduction) - 1.1 [Background](#background) - 1.2 [Position and Contributions](#position-and-contributions) - 2 [Linear Sequence Modeling](#linear-sequence-modeling) - 2.1 [Linear Attention](#linear-attention) - 2.2 [Linear RNN](#linear-rnn) - 2.3 [State Space Model](#state-space-model) - 2.4 [Test-Time-Training RNN](#test-time-training) - 2.5 [Unified Linear Sequence Modeling](#unified-linear-sequence-modeling) - 2.6 [Linearization](#linearization) - 2.7 [Hardware-efficient Implementation](#hardware-efficient-implementation) - 3 [Sparse Sequence Modeling](#sparse-sequence-modeling) - 3.1 [Static Sparse Attention](#static-sparse-attention) - 3.2 [Dynamic Sparse Attention](#dynamic-sparse-attention) - 3.3 [Training-free Sparse Attention](#training-free-sparse-attention) - 3.4 [Hardware-efficient Implementation](#hardware-efficient-implementation) - 4 [Efficient Full Attention](#efficient-full-attention) - 4.1 [IO-Aware Attention](#io-aware-attention) - 4.2 [Grouped Attention](#grouped-attention) - 4.3 [Mixture of Attention](#mixture-of-attention) - 4.4 [Quantized Attention](#quantized-attention) - 5 [Sparse Mixture-of-Experts](#sparse-mixture-of-experts) - 5.1 [Routing Mechanisms](#routing-mechanisms) - 5.2 [Expert Architectures](#expert-architectures) - 5.3 [MoE Conversion](#moe-conversion) - 6 [Hybrid Architectures](#hybrid-architectures) - 6.1 [Inter-layer Hybrid](#inter-layer-hybrid) - 6.2 [Intra-layer Hybrid](#intra-layer-hybrid) - 7 [Diffusion Large Language Models](#diffusion-large-language-models) - 7.1 [Non-Autoregressive Diffusion LLM](#non-autoregressive-diffusion-llm) - 7.2 [Bridging Diffusion LLM and Autoregressive](#bridging-diffusion-llm-and-autoregressive) - 7.3 [Extending Diffusion LLM to Multimodality](#extending-diffusion-llm-to-multimodality) - 8 [Applications to Other Modalities](#applications-to-other-modalities) - 8.1 [Vision](#vision) - 8.2 [Audio](#audio) - 8.3 [Multimodality](#multimodality) - 9 [Conclusion and Future Directions](#conclusion-and-future-directions) --- ### Linear Sequence Modeling #### Linear Attention - [Log-Linear Attention](https://arxiv.org/abs/2506.0
Excerpt of 64,227 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:4e2356bd05c0137d, topic:moe, topic:state-space-model, readme:mixture of experts