Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Vision-language models, document understanding and any-to-any architectures.
Signals: multimodal, vision-language-model, vlm, clip, multimodal-llm, visual-question-answering, document-understanding, image-captioning
339 results
Tarsier -- a family of large-scale video-language models, which is designed to generate high-quality video descriptions , together with good capability of general video understanding.
✨✨[NeurIPS 2025] VITA-Audio: Fast Interleaved Cross-Modal Token Generation for Efficient Large Speech-Language Model
Step-Audio 2 is an end-to-end multi-modal large language model designed for industry-strength audio understanding and speech conversation.
R1-onevision, a visual language model capable of deep CoT reasoning.
open-source multimodal large language model that can hear, talk while thinking. Featuring real-time end-to-end speech input and streaming audio output conversational capabilities.
Official code implementation of Vary-toy (Small Language Model Meets with Reinforced Vision Vocabulary)
📖 A curated list of resources dedicated to hallucination of multimodal large language models (MLLM).
LaVIT: Empower the Large Language Model to Understand and Generate Visual Content
LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models (ECCV 2024)
OpenTSLM: Time-Series Language Models for Reasoning over Multivariate Medical Text- and Time-Series Data
VideoLLM-online: Online Video Large Language Model for Streaming Video (CVPR 2024)
Lumina-DiMOO - An Open-Sourced Multi-Modal Large Diffusion Language Model
Code for 3D-LLM: Injecting the 3D World into Large Language Models
[ICCV 2025] LLaVA-CoT, a visual language model capable of spontaneous, systematic reasoning
Official implementation for "Multimodal Chain-of-Thought Reasoning in Language Models" (stay tuned and more will be updated)
PyTorch implementation of Audio Flamingo: Series of Advanced Audio Understanding Language Models
SEED-Story: Multimodal Long Story Generation with Large Language Model
State-of-the-art Image & Video CLIP, Multimodal Large Language Models, and More!
TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones
A flexible and efficient codebase for training visually-conditioned language models (VLMs)
[NeurIPS 2025] SpatialLM: Training Large Language Models for Structured Indoor Modeling
Qwen3-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.
Paper list about multimodal and large language models, only used to record papers I read in the daily arxiv for personal needs.
GPT4Tools is an intelligent system that can automatically decide, control, and utilize different visual foundation models, allowing the user to interact with images during a conversation.
24,545 repositories in the index in total.
