Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Harnessing 1.4M GPT4V-synthesized Data for A Lite Vision-Language Model
| Date | Stars |
|---|---|
| 2026-07-31 | 281 |
| 2026-08-04 | 281 |
| 2026-08-06 | 281 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# ALLaVA: Harnessing GPT4V-synthesized Data for A Lite Vision-Language Model
<!-- <div align=center>
<img src="assets/llavas.png" width = "640" alt="llavas" align=center />
</div> -->
<p align="center">
⚡ALLaVA is a project that provides a large-scale GPT4V-synthesized dataset for training LVLMs.⚡
</p>
<!-- <p align="center">
  
</p> -->
<p align="center">
<img src="https://img.shields.io/badge/Python-3.10-lightblue" alt="Python Version"/>
<img src="https://img.shields.io/badge/PyTorch-2.1.1-lightblue" alt="PyTorch Version"/>
<img src="https://img.shields.io/badge/transformers-4.37.0-lightblue" alt="Transformers Version"/>
</p>
<p align="center">
📃 <a href="https://arxiv.org/abs/2402.11684" target="_blank">Paper</a> • 🌐 <a href="https://allava.freedomai.cn/#/" target="_blank">Demo</a>
</p>
<p align="center">
🤗 <a href="https://huggingface.co/datasets/FreedomIntelligence/ALLaVA-4V" target="_blank">ALLaVA-4V Dataset</a>
</p>
<p align="center">
🤗 <a href="https://huggingface.co/FreedomIntelligence/ALLaVA-Phi3-mini-128k" target="_blank">ALLaVA-Phi3-mini-128k</a>
• 🤗 <a href="https://huggingface.co/FreedomIntelligence/ALLaVA-StableLM2-1_6B" target="_blank">ALLaVA-StableLM2-1_6B</a>
• 🤗 <a href="https://huggingface.co/FreedomIntelligence/ALLaVA-Phi2-2_7B" target="_blank">ALLaVA-Phi2-2_7B</a>
</p>
<!-- <p align="center">
📃 <a href="https://arxiv.org/abs/2402.11684" target="_blank">Paper</a> • 🌐 <a href="https://allava.freedomai.cn/#/" target="_blank">Demo</a> • 🤗 <a href="https://huggingface.co/datasets/FreedomIntelligence/ALLaVA-4V" target="_blank">ALLaVA-4V Dataset</a> • 🤗 <a href="https://huggingface.co/FreedomIntelligence/ALLaVA-3B-Longer" target="_blank">ALLaVA-3B-Longer</a> • 🤗 <a href="https://huggingface.co/FreedomIntelligence/ALLaVA-3B" target="_blank">ALLaVA-3B</a>
<br> <a href="https://github.com/FreedomIntelligence/CMB/blob/main/README_zh.md"> 中文</a> | <a href="https://github.com/FreedomIntelligence/CMB/blob/main/README.md"> English
</p> -->
## ✨ Updates
- [06/25/2024]: We release [ALLaVA-Phi3-mini-128k](https://huggingface.co/FreedomIntelligence/ALLaVA-Phi3-mini-128k), [ALLaVA-StableLM2-1_6B](https://huggingface.co/FreedomIntelligence/ALLaVA-StableLM2-1_6B), [ALLaVA-Phi2-2_7B](https://huggingface.co/FreedomIntelligence/ALLaVA-Phi2-2_7B) which all support loading from 🤗 repo.
- [03/01/2024]: The huggingface repo of **ALLaVA-3B-Longer (recommended)** and ALLaVA-3B are updated, which now supports the `from_pretrained` method to load models.
- [02/29/2024]: The huggingface repo of ALLaVA-4V dataset and [download scripts](#data-preparation) are updated.
- [02/21/2024]: We are thrilled to release 1) **1.4M** data for training LVLMs, 2) two version of our ALLaVA-3B models, 3) inference code and 4) tech report.
## 📚 ALLaVA-4V Data
### Generation Pipeline
<div align=center>
<img src="assets/pipeline.jpg" width = "640" alt="pipeline" align=center />
</div>
* LAION
We leverage the superb GPT-4V to generate captions and complex reasoning QA pairs. Prompt is [here](prompts/prompt_for_laion.txt).
* Vison-FLAN
We leverage the superb GPT-4V to generate captions and detailed answer for the original instructions. Prompt is [here](
prompts/prompt_for_vflan.txt).
* Wizard
We regenerate the answer of Wizard_evol_instruct with GPT-4-Turbo.
### Dataset Cards
All datasets can be found [here](https://huggingface.co/datasets/FreedomIntelligence/ALLaVA-4V).
The structure of naming is shown below:
```bash
ALLaVA-4V
├── ALLaVA-Caption-4V
│ ├── ALLaVA-Caption-LAION-4V
│ └── ALLaVA-Caption-VFLAN-4V
├── ALLaVA-Instruct-4V
│ ├── ALLaVA-Instruct-LAION-4V
│ └── ALLaVA-Instruct-VFLAN-4V
├── Evol-Instruct-GPT4-Turbo-143K
```
The folder structure of the hExcerpt of 21,911 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:a98ee5f587a5bae2, desc:vision-language, desc:vision language model