Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Core deep-learning frameworks and libraries for pretraining and distributed training.
Signals: deep-learning, neural-network, pytorch, tensorflow, jax, distributed-training, training, deepspeed
2,663 results
Build agents which are controlled by LLMs
The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
Muon is Scalable for LLM Training
Information hub for our project training the largest possible historical LLMs.
Official PyTorch implementation of TSDiff models presented in the NeurIPS 2023 paper "Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecasting"
[ICCV 2023] The official implementation of paper "HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generation"
Pytorch implementation of diffusion models on Lie Groups for 6D grasp pose generation https://sites.google.com/view/se3dif/home
The official implementation for Pseudo Numerical Methods for Diffusion Models on Manifolds (PNDM, PLMS | ICLR2022)
Search-R1: An Efficient, Scalable RL Training Framework for Reasoning & Search Engine Calling interleaved LLM based on veRL
Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.
LLM training in simple, raw C/CUDA
IDDM (Industrial, landscape, animate, latent diffusion), support LDM, DDPM, DDIM, PLMS, webui and distributed training. Pytorch实现扩散模型，生成模型，分布式训练
Mistral: A strong, northwesterly wind: Framework for transparent and accessible large-scale language model training, built with Hugging Face 🤗 Transformers.
The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”
From scratch implementation of a sparse mixture of experts language model inspired by Andrej Karpathy's makemore :)
Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways
Implementation of the training framework proposed in Self-Rewarding Language Model, from MetaAI
Pretraining and inference code for a large-scale depth-recurrent language model
An implementation of local windowed attention for language modeling
Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch
Saprot: Protein Language Model with Structural Alphabet (AA+3Di)
PhoBERT: Pre-trained language models for Vietnamese (EMNLP-2020 Findings)
Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI
24,545 repositories in the index in total.
