Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
LLM papers I'm reading, mostly on inference and model compression
| Date | Stars |
|---|---|
| 2026-07-31 | 752 |
| 2026-08-06 | 752 |
| 2026-08-18 | 751 |
| 2026-08-19 | 751 |
| 2026-09-02 | 752 |
| 2026-09-03 | 751 |
| 2026-10-02 | 751 |
| 2026-10-16 | 750 |
| 2026-10-18 | 751 |
| 2026-10-19 | 750 |
| 2026-10-24 | 751 |
| 2026-11-01 | 752 |
| 2026-11-04 | 752 |
Today
— stars today
This week
+1 stars this week
This month
+1 stars this month
Momentum
0.0
growth rate 0.13%/day
Just helping myself keep track of LLM papers that I‘m reading, with an emphasis on inference and model compression. Transformer Architectures * [Attention Is All You Need](https://arxiv.org/abs/1706.03762) * [Fast Transformer Decoding: One Write-Head is All You Need](https://arxiv.org/abs/1911.02150) - Multi-Query Attention * [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://arxiv.org/abs/1901.02860) * [Augmenting Self-attention with Persistent Memory](https://arxiv.org/abs/1907.01470) (Meta 2019) * [MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers](https://arxiv.org/abs/2305.07185) (Meta 2023) * [Hyena Hierarchy: Towards Larger Convolutional Language Models](https://arxiv.org/abs/2302.10866) Foundation Models * [LLaMA: Open and Efficient Foundation Language Models](https://arxiv.org/abs/2302.13971) * [PaLM: Scaling Language Modeling with Pathways](https://arxiv.org/abs/2204.02311) * [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://arxiv.org/abs/2204.06745) * [Language Models are Unsupervised Multitask Learners](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) (OpenAI) - GPT-2 * [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://arxiv.org/abs/1910.13461) * [OpenLLaMA: An Open Reproduction of LLaMA](https://github.com/openlm-research/open_llama) * [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/) * [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://arxiv.org/abs/1910.01108) * [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://github.com/state-spaces/mamba) Position Encoding * [Self-Attention with Relative Position Representations](https://arxiv.org/abs/1803.02155) * [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://arxiv.org/abs/2104.09864) - RoPE * [Transformer Language Models without Positional Encodings Still Learn Positional Information](https://arxiv.org/abs/2203.16634) - NoPE * [Rectified Rotary Position Embeddings](https://github.com/bojone/rerope) - ReRoPE KV Cache * [H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models](https://arxiv.org/abs/2306.14048) (Jun. 2023) * [vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention](https://vllm.ai) * [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://arxiv.org/abs/2205.14135) Activation * [Searching for Activation Functions](https://arxiv.org/abs/1710.05941) * [GLU Variants Improve Transformer](https://arxiv.org/abs/2002.05202) * [PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU](https://github.com/SJTU-IPADS/PowerInfer) Pruning * [Optimal Brain Damage](https://proceedings.neurips.cc/paper/1989/hash/6c9882bbac1c7093bd25041881277658-Abstract.html) (1990) * [Optimal Brain Surgeon](https://proceedings.neurips.cc/paper/1992/file/303ed4c69846ab36c2904d3ba8573050-Paper.pdf) (1993) * [Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning](https://arxiv.org/abs/2208.11580) (Jan. 2023) - Introduces Optimal Brain Quantization based on the Optimal Brain Surgeon * [Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon](https://arxiv.org/abs/1705.07565) * [SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot](https://arxiv.org/abs/2301.00774) * [A Simple and Effective Pruning Approach for Large Language Models](https://arxiv.org/abs/2306.11695) - Introduces Wanda (pruning with Weights and Activations) Quantization * [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://arxiv.org/abs/2208.07339) - Quantization with outlier handling. Might be solving the wrong problem - see "Quantizable Transforme
Excerpt of 8,596 characters
Read on GitHub
13
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:5ed1cfa7e9e67ad7, name:reading list