Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones
| Date | Stars |
|---|---|
| 2026-07-31 | 1316 |
| 2026-08-01 | 1316 |
| 2026-08-06 | 1316 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# TinyGPT-V <font size='5'>**TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones**</font> [Zhengqing Yuan](https://dlyuangod.github.io/zhengqingyuan/)✟, Zhaoxu Li❁, [Weiran Huang](https://www.weiranhuang.com/)❋, [Yanfang Ye](http://yes-lab.org/)✟, [Lichao Sun](https://lichao-sun.github.io/)❁ ✟University of Notre Dame, ❁Lehigh University, ❋Shanghai Jiao Tong University Zhaoxu is a visiting student in the LAIR lab at Lehigh University. </a> <a href='https://arxiv.org/pdf/2602.04816'><img src='https://img.shields.io/badge/Paper-Arxiv-red'></a> <a href='https://huggingface.co/Tyrannosaurus/TinyGPT-V'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-blue'></a> <a href='https://huggingface.co/spaces/llizhx/TinyGPT-V'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue'> English | [简体中文](/README_zh-CN.md) </font> ## News [Apr.08 2024] Update our paper v2. We revised some type errors, provided more details and updated TinyGPT-V lastest results. [Mar.20 2024] Update the Phi-2 weight download link. [Jan.22 2024] Welcome to Hugging Face online demo to try out our models (for Stage-4 v1)! [Jan.19 2024] Major Updates! We are officially releasing v1 of TinyGPT-V! After our evaluation, the performance of TinyGPT-V has reached 98% of InstructBLIP's performance and exceeds the performance of other models of the same period! [Jan.03 2024] Welcome to Hugging Face online demo to try out our models (for Stage-3)! [Dec.28 2023] Breaking! We release the code of our TinyGPT-V. ## TinyGPT-V Model Structure ### Whole Model Structure  ### Language Model Structure  ## TinyGPT-V Traning Process  ## TinyGPT-V Results ### Radar Chart  ### Performance and Efficiency  ## Getting Started ### Installation **1. Prepare the code and the environment** Git clone our repository, creating a python environment and activate it via the following command ```bash git clone https://github.com/DLYuanGod/TinyGPT-V.git cd TinyGPT-V conda env create -f environment.yml conda activate tinygptv ``` **2. Prepare the pretrained LLM weights** **TinyGPT-V** is based on Phi-2. Download the corresponding LLM weights from the following huggingface space via clone the repository using git-lfs. Phi-2 2.7B: [Download](https://huggingface.co/spaces/llizhx/TinyGPT-V/tree/main/phi-2) Then, set the variable *phi_model* in the model config file to the LLM weight path. * Set the LLM path [here](minigpt4/configs/models/minigpt_v2.yaml#L14) at Line 14, [here](minigpt4/configs/models/minigpt4_vicuna0.yaml#L18) at Line 18 and [here](minigpt4/conversation/conversation.py#L16) at Line 16. **3. Prepare the pretrained model checkpoints** Download the pretrained model checkpoints | After stage-1 | After stage-2 | After stage-3| After stage-4 | | ------ | ------ | ------ | -------| | [Download](https://huggingface.co/Tyrannosaurus/TinyGPT-V/blob/main/TinyGPT-V_for_Stage1.pth) |[Download](https://huggingface.co/Tyrannosaurus/TinyGPT-V/blob/main/TinyGPT-V_for_Stage2.pth) | [Download](https://huggingface.co/Tyrannosaurus/TinyGPT-V/blob/main/TinyGPT-V_for_Stage3.pth) |[Download](https://huggingface.co/Tyrannosaurus/TinyGPT-V/blob/main/TinyGPT-V_for_Stage4.pth) | For **TinyGPT-V**, set the path to the pretrained checkpoint in the evaluation config file in [tinygptv_stage1_2_3_eval.yaml](eval_configs/tinygptv_stage1_2_3_eval.yaml#L8) at Line 8 for Stage 1, 2 and 3 version or [tinygptv_stage4_eval.yaml](eval_configs/tinygptv_stage4_eval.yaml#L8) for Stage 4 version. **4. Update the Phi-2 Modeling for transformers lib.** Linux system: ``` cp modeling_phi.py /root/miniconda3/envs/tinygptv/lib/python3.9/site-packages/transformers/models/phi/ ``` Windows system Find your conda yourself: conda_sit/envs/tinygptv/lib/
Excerpt of 8,470 characters
Read on GitHub
33
16
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:8144e7c7aac3c762, desc:multimodal