Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Macaw-LLM: Multi-Modal Language Modeling with Image, Video, Audio, and Text Integration
| Date | Stars |
|---|---|
| 2026-07-24 | 1590 |
| 2026-07-25 | 1590 |
| 2026-07-28 | 1590 |
| 2026-07-30 | 1590 |
| 2026-07-31 | 1591 |
| 2026-08-06 | 1591 |
Today
— stars today
This week
+1 stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.06%/day
<div align="center"> <img src="https://raw.githubusercontent.com/lyuchenyang/Macaw-LLM/main/assets/logo-text.png" alt="Logo" width="200"> </div> <div align="center"> <img src="/assets/2-watermark.png" alt="Logo" width="50"> <img src="/assets/1-watermark.png" alt="Logo" width="70"> <img src="/assets/0-watermark.png" alt="Logo" width="50"> <img src="/assets/3-watermark.png" alt="Logo" width="50"> <img src="/assets/4-watermark.png" alt="Logo" width="50"> </div> # Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration <div align="center"> <img src="https://img.shields.io/badge/Version-1.0.0-blue.svg" alt="Version"> <img src="https://img.shields.io/badge/License-CC%20BY%204.0-green.svg" alt="License"> <img src="https://img.shields.io/github/stars/lyuchenyang/Macaw-LLM?color=yellow" alt="Stars"> <img src="https://img.shields.io/github/issues/lyuchenyang/Macaw-LLM?color=red" alt="Issues"> <img src="https://img.shields.io/badge/python-3.8-purple.svg" alt="Python"> <!-- **Authors:** --> **_¹ ² [Chenyang Lyu](https://lyuchenyang.github.io), ³ [Minghao Wu](https://minghao-wu.github.io/), ¹ <sup>*</sup> [Longyue Wang](http://www.longyuewang.com/), ¹ [Xinting Huang](https://timhuang1.github.io/),_** **_¹ [Bingshuai Liu](bingshuailiu.github.io), ¹ [Zefeng Du](https://seeledu.github.io/index-en.html), ¹ [Shuming Shi](https://shumingshi.github.io/), ¹ [Zhaopeng Tu](http://www.zptu.net/)_** <!-- **Affiliations:** --> _¹ Tencent AI Lab, ² Dublin City University, ³ Monash University_ _<sup>*</sup>Longyue Wang is the corresponding author: [[email protected]](mailto:{[email protected])_ </div> Macaw-LLM is an exploratory endeavor that pioneers multi-modal language modeling by seamlessly combining image🖼️, video📹, audio🎵, and text📝 data, built upon the foundations of CLIP, Whisper, and LLaMA. #### 📰 <a href="https://tinyurl.com/4rsexudv" style="color: black; text-decoration: underline;text-decoration-style: dotted;">Paper</a> :building_construction: <a href="https://tinyurl.com/yem9m4nf" style="color: black; text-decoration: underline;text-decoration-style: dotted;">Model (via dropbox)</a> :building_construction: <a href="https://share.weiyun.com/27EFG2pq" style="color: black; text-decoration: underline;text-decoration-style: dotted;">Model (via weiyun)</a> :card_file_box: <a href="/data" style="color: black; text-decoration: underline;text-decoration-style: dotted;">Dataset</a> :bricks: [Code](#usage) :monocle_face: Video :technologist: Demo ## Table of Contents 📚 - [Introduction](#introduction) - [Key Features](#key-features) - [Architecture](#architecture) - [Alignment Strategy](#alignment-strategy) - [Installation](#installation) - [Usage](#usage) - [Future Work and Contributions](#future-work-and-contributions) ## Introduction <img src="/assets/4-watermark.png" alt="Logo" width="40"> <div align="center"> <img src="assets/alignment.png" alt="Figure Description or Alt Text" width="80%"> </div> <!--  --> In recent years, the field of language modeling has witnessed remarkable advancements. However, the integration of multiple modalities, such as images, videos, audios, and text, has remained a challenging task. Macaw-LLM is a model of its kind, bringing together state-of-the-art models for processing visual, auditory, and textual information, namely CLIP, Whisper, and LLaMA. ## Key Features 🔑 Macaw-LLM boasts the following unique features: 1. **Simple & Fast Alignment**: Macaw-LLM enables seamless integration of multi-modal data through simple and fast alignment to LLM embeddings. This efficient process ensures quick adaptation of diverse data types. 2. **One-Stage Instruction Fine-Tuning**: Our model streamlines the adaptation process through one-stage instruction fine-tuning, promoting a more
Excerpt of 11,467 characters
Read on GitHub
98
Longyue Wang · Alibaba International
42
2
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:e3756515e8ba6377, topic:deep-learning
matched fp:e3756515e8ba6377, topic:language-model
matched fp:e3756515e8ba6377, topic:natural-language-processing