Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
LLM training code for Databricks foundation models
| Date | Stars |
|---|---|
| 2026-07-24 | 4431 |
| 2026-07-25 | 4431 |
| 2026-07-28 | 4431 |
| 2026-07-30 | 4431 |
| 2026-07-31 | 4432 |
| 2026-08-06 | 4432 |
Today
— stars today
This week
+1 stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.02%/day
<!-- SETUPTOOLS_LONG_DESCRIPTION_HIDE_BEGIN -->
<p align="center">
<a href="https://github.com/mosaicml/llm-foundry">
<picture>
<img alt="LLM Foundry" src="./assets/llm-foundry.png" width="95%">
</picture>
</a>
</p>
<!-- SETUPTOOLS_LONG_DESCRIPTION_HIDE_END -->
<p align="center">
<a href="https://pypi.org/project/llm-foundry/">
<img alt="PyPi Version" src="https://img.shields.io/pypi/pyversions/llm-foundry">
</a>
<a href="https://pypi.org/project/llm-foundry/">
<img alt="PyPi Package Version" src="https://img.shields.io/pypi/v/llm-foundry">
</a>
<a href="https://mosaicml.me/slack">
<img alt="Chat @ Slack" src="https://img.shields.io/badge/slack-chat-2eb67d.svg?logo=slack">
</a>
<a href="https://github.com/mosaicml/llm-foundry/blob/main/LICENSE">
<img alt="License" src="https://img.shields.io/badge/License-Apache%202.0-green.svg">
</a>
</p>
<br />
# LLM Foundry
This repository contains code for training, finetuning, evaluating, and deploying LLMs for inference with [Composer](https://github.com/mosaicml/composer) and the [MosaicML platform](https://forms.mosaicml.com/demo?utm_source=github.com&utm_medium=referral&utm_campaign=llm-foundry). Designed to be easy-to-use, efficient _and_ flexible, this codebase enables rapid experimentation with the latest techniques.
You'll find in this repo:
* `llmfoundry/` - source code for models, datasets, callbacks, utilities, etc.
* `scripts/` - scripts to run LLM workloads
* `data_prep/` - convert text data from original sources to StreamingDataset format
* `train/` - train or finetune HuggingFace and MPT models from 125M - 70B parameters
* `train/benchmarking` - profile training throughput and MFU
* `inference/` - convert models to HuggingFace or ONNX format, and generate responses
* `inference/benchmarking` - profile inference latency and throughput
* `eval/` - evaluate LLMs on academic (or custom) in-context-learning tasks
* `mcli/` - launch any of these workloads using [MCLI](https://docs.mosaicml.com/projects/mcli/en/latest/) and the [MosaicML platform](https://www.mosaicml.com/platform)
* `TUTORIAL.md` - a deeper dive into the repo, example workflows, and FAQs
# DBRX
DBRX is a state-of-the-art open source LLM trained by Databricks Mosaic team. It uses the Mixture-of-Experts (MoE) architecture and was trained with optimized versions of [Composer](https://github.com/mosaicml/composer), LLM Foundry, and [MegaBlocks](https://github.com/databricks/megablocks). The model has 132B total parameters and 36B active parameters. We have released two DBRX models:
| Model | Context Length | Download |
| ------------------ | -------------- | -------------------------------------------------- |
| DBRX Base | 32768 | https://huggingface.co/databricks/dbrx-base |
| DBRX Instruct | 32768 | https://huggingface.co/databricks/dbrx-instruct |
Our model weights and code are licensed for both researchers and commercial entities. The Databricks Open Source License can be found at [LICENSE](https://github.com/databricks/dbrx/blob/main/LICENSE), and our Acceptable Use Policy can be found [here](https://www.databricks.com/legal/acceptable-use-policy-open-model).
For more information about the DBRX models, see https://github.com/databricks/dbrx.
# MPT
Mosaic Pretrained Transformers (MPT) are GPT-style models with some special features -- Flash Attention for efficiency, ALiBi for context length extrapolation, and stability improvements to mitigate loss spikes. As part of MosaicML's Foundation series, we have open-sourced several MPT models:
| Model | Context Length | Download | Commercial use? |
| ------------------ | -------------- | -------------------------------------------------- | --------------- |
| MPT-30B | 8192 | https://huggiExcerpt of 20,949 characters
Read on GitHub
Daniel King
318
Irene Dea
90
87
Abhi Venigalla · MosaicML · United States
82
Mihir Patel
69
59
Vincent Chen
48
37
37
35
Brian
34
Charles Tang
31
27
Jeremy D
23
22
Anna · United States
22
19
MosaicML · United States
19
Hanlin Tang · MosaicML · United States
17
16
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:cce9054f6ed8262d, topic:deep-learning, topic:pytorch
matched fp:cce9054f6ed8262d, topic:llm, readme:model weights, readme:mixture of experts
matched fp:cce9054f6ed8262d, topic:nlp