Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via Iterative Instruction Tuning and Reinforcement Learning
| Date | Stars |
|---|---|
| 2026-07-31 | 271 |
| 2026-08-01 | 271 |
| 2026-08-02 | 271 |
| 2026-08-06 | 271 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<div align="center"> <br> <h3>VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via Iterative Instruction Tuning and Reinforcement Learning</h3> Xianwei Zhuang<sup>1*</sup> Yuxin Xie<sup>1*</sup> Yufan Deng<sup>1*</sup> Dongchao Yang<sup>2</sup> <br> Liming Liang<sup>1</sup> Jinghan Ru<sup>1</sup> Yuguo Yin<sup>1</sup> Yuexian Zou <sup>1</sup> <sup>1</sup> Peking University, <sup>2</sup> The Chinese University of Hong Kong [-b31b1b.svg?logo=arXiv)](https://arxiv.org/pdf/2504.02949) [-b31b1b.svg?logo=arXiv)](https://arxiv.org/pdf/2501.12327) [](https://vargpt1-1.github.io/) [-blue.svg)](https://huggingface.co/VARGPT-family/VARGPT-v1.1) [-blue.svg)](https://huggingface.co/VARGPT-family/VARGPT-v1.1-edit) [](https://huggingface.co/datasets/VARGPT-family/VARGPT_datasets) [](https://github.com/VARGPT-family/VARGPT/blob/main/LICENSE) [](https://mp.weixin.qq.com/s/96yyriyCmwnrGk4_M9_ZUg) </div> https://github.com/user-attachments/assets/0b0a9b25-3637-437c-8396-3cc3a5950879 ## News * **[2025-04-7]** The technical report is released at https://arxiv.org/pdf/2504.02949. * **[2025-04-2]** We release the more powerful unified model of **VARGPT-v1.1** (**7B+2B**) at [VARGPT-v1.1](https://huggingface.co/VARGPT-family/VARGPT-v1.1) and the editing model datasets at [VARGPT-v1.1-edit](https://huggingface.co/VARGPT-family/VARGPT-v1.1-edit). 🔥🔥🔥🔥🔥🔥 * **[2025-04-1]** We release the **training (SFT and RL), inference and evaluation code** of **VARGPT-v1.1 and VARGPT** at [VARGPT-family-training](https://github.com/VARGPT-family/VARGPT-v1.1/tree/main/VARGPT-family-training) for multimodal understanding and generation including image captioning, visual question answering (VQA), text-to-image generation and visual editing. 🔥🔥🔥🔥🔥🔥 ## What is the new about VARGPT-v1.1? <p align="center"> <img src="docs/generation_vis.png" width="888"> </p> <p align="center"> <img src="docs/understanding_vis.png" width="555"> </p> Compared with VARGPT, VARGPT-v1.1 has achieved comprehensive capability improvement. VARGPT-v1.1 integrates: (1) a novel training strategy combining iterative visual instruction tuning with reinforcement learning through Direct Preference Optimization (DPO), (2) an expanded training corpus containing 8.3M visual-generative instruction pairs, (3) an upgraded language backbone using Qwen2, (4) enhanced image generation resolution, and (5) emergent image editing capabilities without architectural modifications. <p align="center"> <img src="docs/vargpt_training_1.png" width="888"> </p> <p align="center"> <img src="docs/vargpt_training_2.png" width="888"> </p> <br/> ## TODO - [X] Release the inference code. - [X] Release the code for evaluation. - [X] Release the model checkpoint. - [X] Supporting stronger visual generation capabilities. - [X] Release the training datasets. - [X] Release the training code. - [X] Release the technical report. ## Hugging Face models and annotations The VARGPT-v1.1 checkpoints can be found on Hugging Face: * [VARGPT-family/VARGPT_v1-1](https://huggingface.co/VARGPT-family/VARGPT-v1.1) The VARGPT-v1.1-edit checkpoints for visual editing can be found on Hugging Face: * [VARGPT-family/VARGPT_v1-1_edit](https://huggingface.co/VARGPT-family/VARGPT-v1.1-edit) The VARGPT checkpoints can be found on Hugging Face: * [VARGPT-family/VARGPT_LLaVA-v1](https://huggingface
Excerpt of 13,938 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:9badc5b8fb79eb77, llm:description: 'VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via Iterative Instruction Tuning and Reinforcement Learning'; topics: 'mllm', 'unified-model'
matched fp:9badc5b8fb79eb77, llm:description: 'VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via Iterative Instruction Tuning and Reinforcement Learning'; topics: 'mllm', 'unified-model'
matched fp:9badc5b8fb79eb77, llm:description: 'VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via Iterative Instruction Tuning and Reinforcement Learning'; topics: 'mllm', 'unified-model'