Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Official repo for "Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models"
| Date | Stars |
|---|---|
| 2026-07-24 | 3329 |
| 2026-07-25 | 3329 |
| 2026-07-28 | 3330 |
| 2026-07-30 | 3330 |
| 2026-08-07 | 3331 |
| 2026-08-15 | 3331 |
| 2026-08-19 | 3330 |
| 2026-08-20 | 3329 |
| 2026-08-21 | 3328 |
| 2026-08-24 | 3327 |
| 2026-09-04 | 3326 |
| 2026-09-08 | 3324 |
| 2026-09-14 | 3323 |
| 2026-09-18 | 3324 |
| 2026-09-20 | 3324 |
Today
— stars today
This week
— stars this week
This month
-4 stars this month
Momentum
0.0
growth rate 0.00%/day
# Official repo for "Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models" <a href='https://mini-gemini.github.io/'><img src='https://img.shields.io/badge/Project-Page-Green'></a> <a href='http://103.170.5.190:7860/'><img src='https://img.shields.io/badge/Project-Demo-violet'></a> <a href='https://huggingface.co/spaces/wcy1122/MGM'><img src='https://img.shields.io/badge/🤗-Open%20In%20Spaces-blue.svg'></a> <a href='https://arxiv.org/pdf/2403.18814.pdf'><img src='https://img.shields.io/badge/Paper-Arxiv-red'></a> <a href='https://huggingface.co/collections/YanweiLi/mgm-6603c50b9b43d044171d0854'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Models-blue'></a> <a href='https://huggingface.co/collections/YanweiLi/mgm-data-660463ea895a01d8f367624e'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Data-green'></a> The framework supports a series of dense and MoE Large Language Models (LLMs) from 2B to 34B with image understanding, reasoning, and generation simultaneously. We build this repo based on LLaVA. ## Release - [05/03] 🔥 We support LLaMA3-based models! Welcome to try them [here](https://huggingface.co/collections/YanweiLi/mgm-6603c50b9b43d044171d0854). - [04/15] 🔥 The [Hugging Face demo](https://huggingface.co/spaces/wcy1122/MGM) is available. It's a 13B-HD version, welcome to watch and try. - [03/28] 🔥 Mini-Gemini is coming! We release the [paper](https://arxiv.org/pdf/2403.18814.pdf), [demo](http://103.170.5.190:7860/), [code](https://github.com/dvlab-research/MGM), [models](https://huggingface.co/collections/YanweiLi/mgm-6603c50b9b43d044171d0854'), and [data](https://huggingface.co/collections/YanweiLi/mgm-data-660463ea895a01d8f367624e)! ## Contents - [Demo](#demo) - [Install](#install) - [Model](#model) - [Preparation](#preparation) - [Train](#train) - [Evaluation](#evaluation) - [Examples](#examples) - [Citation](#citation) - [Acknowledgement](#acknowledgement) - [License](#license) ## Demo We provide some selected examples in this section. More examples can be found in our [project page](https://mini-gemini.github.io/). Feel free to try our online [demo](http://103.170.5.190:7860/)! <div align=center> <img width="100%" src="images/teaser.png"/> </div> ## Install Please follow the instructions below to install the required packages. NOTE: If you want to use the 2B version, please ensure to install the latest version Transformers (>=4.38.0). 1. Clone this repository ```bash git clone https://github.com/dvlab-research/MGM.git ``` 2. Install Package ```bash conda create -n mgm python=3.10 -y conda activate mgm cd MGM pip install --upgrade pip # enable PEP 660 support pip install -e . ``` 3. Install additional packages for training cases ```bash pip install ninja pip install flash-attn --no-build-isolation ``` ## Model The framework is conceptually simple: dual vision encoders are utilized to provide low-resolution visual embedding and high-resolution candidates; patch info mining is proposed to conduct patch-level mining between high-resolution regions and low-resolution visual queries; LLM is utilized to marry text with images for both comprehension and generation at the same time. <div align=center> <img width="98%" src="images/pipeline.png"/> </div> We provide all our fully finetuned models on Stage 1 and 2 data: | Model | LR | HR | Base LLM | Vision Encoder | Finetuning Data | Finetuning schedule | Download | |----------|----------|----------|----------|----------------|---------------|--------------------|------------------| | MGM-2B | 336 | 768 | Gemma-2B | CLIP-L | MGM-Instruct | full_ft-1e | [ckpt](https://huggingface.co/YanweiLi/MGM-2B) | | MGM-7B | 336 | 768 | Vicuna-7B-v1.5 | CLIP-L | MGM-Instruct | full_ft-1e | [ckpt](https://huggingface.co/YanweiLi/MGM-7B) | | MGM-13B | 336 | 768 | Vicuna-13B-v1.5 | CLIP-L | MGM-Instruct | full_ft-1e | [ckpt](https://huggingface.co/YanweiLi/MGM-13B) | | MGM-8B | 336 | 768 | LLaMA-3-8B-Instruct
Excerpt of 21,568 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:0e39ce5c67447acd, topic:vision-language-model, desc:vision-language, readme:vision-language
matched fp:0e39ce5c67447acd, topic:large-language-models
