Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
An open-source framework for training large multimodal models.
| Date | Stars |
|---|---|
| 2026-07-24 | 4115 |
| 2026-07-25 | 4115 |
| 2026-07-28 | 4116 |
| 2026-07-30 | 4116 |
| 2026-08-10 | 4118 |
| 2026-08-18 | 4119 |
| 2026-08-26 | 4118 |
| 2026-08-27 | 4117 |
| 2026-09-02 | 4116 |
| 2026-09-06 | 4115 |
| 2026-09-07 | 4114 |
| 2026-09-09 | 4113 |
| 2026-09-20 | 4113 |
Today
— stars today
This week
— stars this week
This month
-6 stars this month
Momentum
0.0
growth rate 0.00%/day
# 🦩 OpenFlamingo
[](https://badge.fury.io/py/open_flamingo)
[Paper](https://arxiv.org/abs/2308.01390) | Blog posts: [1](https://laion.ai/blog/open-flamingo/), [2](https://laion.ai/blog/open-flamingo-v2/) | [Demo](https://huggingface.co/spaces/openflamingo/OpenFlamingo)
Welcome to our open source implementation of DeepMind's [Flamingo](https://www.deepmind.com/blog/tackling-multiple-tasks-with-a-single-visual-language-model)!
In this repository, we provide a PyTorch implementation for training and evaluating OpenFlamingo models.
If you have any questions, please feel free to open an issue. We also welcome contributions!
# Table of Contents
- [Installation](#installation)
- [Approach](#approach)
* [Model architecture](#model-architecture)
- [Usage](#usage)
* [Initializing an OpenFlamingo model](#initializing-an-openflamingo-model)
* [Generating text](#generating-text)
- [Training](#training)
* [Dataset](#dataset)
- [Evaluation](#evaluation)
- [Future plans](#future-plans)
- [Team](#team)
- [Acknowledgments](#acknowledgments)
- [Citing](#citing)
# Installation
To install the package in an existing environment, run
```
pip install open-flamingo
```
or to create a conda environment for running OpenFlamingo, run
```
conda env create -f environment.yml
```
To install training or eval dependencies, run one of the first two commands. To install everything, run the third command.
```
pip install open-flamingo[training]
pip install open-flamingo[eval]
pip install open-flamingo[all]
```
There are three `requirements.txt` files:
- `requirements.txt`
- `requirements-training.txt`
- `requirements-eval.txt`
Depending on your use case, you can install any of these with `pip install -r <requirements-file.txt>`. The base file contains only the dependencies needed for running the model.
## Development
We use pre-commit hooks to align formatting with the checks in the repository.
1. To install pre-commit, run
```
pip install pre-commit
```
or use brew for MacOS
```
brew install pre-commit
```
2. Check the version installed with
```
pre-commit --version
```
3. Then at the root of this repository, run
```
pre-commit install
```
Then every time we run git commit, the checks are run. If the files are reformatted by the hooks, run `git add` for your changed files and `git commit` again
# Approach
OpenFlamingo is a multimodal language model that can be used for a variety of tasks. It is trained on a large multimodal dataset (e.g. Multimodal C4) and can be used to generate text conditioned on interleaved images/text. For example, OpenFlamingo can be used to generate a caption for an image, or to generate a question given an image and a text passage. The benefit of this approach is that we are able to rapidly adapt to new tasks using in-context learning.
## Model architecture
OpenFlamingo combines a pretrained vision encoder and a language model using cross attention layers. The model architecture is shown below.
Credit: [Flamingo](https://www.deepmind.com/blog/tackling-multiple-tasks-with-a-single-visual-language-model)
# Usage
## Initializing an OpenFlamingo model
We support pretrained vision encoders from the [OpenCLIP](https://github.com/mlfoundations/open_clip) package, which includes OpenAI's pretrained models.
We also support pretrained language models from the `transformers` package, such as [MPT](https://huggingface.co/models?search=mosaicml%20mpt), [RedPajama](https://huggingface.co/models?search=redpajama), [LLaMA](https://huggingface.co/models?search=llama), [OPT](https://huggingface.co/models?search=opt), [GPT-Neo](https://huggingface.co/models?search=gpt-neo), [GPT-J](https://huggingface.co/models?search=gptj), and [Pythia](https://huggingface.co/models?search=pythia) models.
``` python
from open_flamingo import create_model_and_transforms
model, imagExcerpt of 13,805 characters
Read on GitHub
Anas Awadalla
200
174
30
12
Mitchell Wortsman · University of Washington
10
10
Yonatan Bitton
8
4
Isaac Chung · @zendesk · Estonia
3
Yassine
3
Zonglin DI
1
1
1
Chris Alexiuk · NVIDIA · Canada
1
1
1
Sidd Karamcheti · Georgia Tech · United States
1
Gabriel Ilharco · United States
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:67133b581d63e17f, topic:deep-learning, topic:pytorch
matched fp:67133b581d63e17f, topic:language-model
matched fp:67133b581d63e17f, topic:computer-vision