Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Reading list for research topics in multimodal machine learning
| Date | Stars |
|---|---|
| 2026-07-24 | 6910 |
| 2026-07-25 | 6911 |
| 2026-07-28 | 6913 |
| 2026-07-30 | 6913 |
| 2026-07-31 | 6915 |
| 2026-08-06 | 6920 |
Today
+5 stars today
This week
+7 stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.10%/day
# Awesome Multimodal Machine Learning By [Paul Liang](http://www.cs.cmu.edu/~pliang/) ([email protected]), [Machine Learning Department](http://www.ml.cmu.edu/) and [Language Technologies Institute](https://www.lti.cs.cmu.edu/), [CMU](https://www.cmu.edu/), with help from members of the [MultiComp Lab](http://multicomp.cs.cmu.edu/) at LTI, CMU. If there are any areas, papers, and datasets I missed, please let me know! ## Course content + workshops Check out our comprehsensive tutorial paper [Foundations and Recent Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions](https://arxiv.org/abs/2209.03430). [Tutorials on Multimodal Machine Learning](https://cmu-multicomp-lab.github.io/mmml-tutorial/cvpr2022/) at CVPR 2022 and NAACL 2022, slides and videos [here](https://cmu-multicomp-lab.github.io/mmml-tutorial/schedule/). New course [11-877 Advanced Topics in Multimodal Machine Learning](https://cmu-multicomp-lab.github.io/adv-mmml-course/spring2022/) Spring 2022 @ CMU. It will primarily be reading and discussion-based. We plan to post discussion probes, relevant papers, and summarized discussion highlights every week on the website. Public course content and lecture videos from [11-777 Multimodal Machine Learning](https://cmu-multicomp-lab.github.io/mmml-course/fall2020/), Fall 2020 @ CMU. ## Table of Contents * [Survey Papers](#survey-papers) * [Core Areas](#core-areas) * [Multimodal Representations](#multimodal-representations) * [Multimodal Fusion](#multimodal-fusion) * [Multimodal Alignment](#multimodal-alignment) * [Multimodal Pretraining](#multimodal-pretraining) * [Multimodal Translation](#multimodal-translation) * [Crossmodal Retrieval](#crossmodal-retrieval) * [Multimodal Co-learning](#multimodal-colearning) * [Missing or Imperfect Modalities](#missing-or-imperfect-modalities) * [Analysis of Multimodal Models](#analysis-of-multimodal-models) * [Knowledge Graphs and Knowledge Bases](#knowledge-graphs-and-knowledge-bases) * [Intepretable Learning](#intepretable-learning) * [Generative Learning](#generative-learning) * [Semi-supervised Learning](#semi-supervised-learning) * [Self-supervised Learning](#self-supervised-learning) * [Language Models](#language-models) * [Adversarial Attacks](#adversarial-attacks) * [Few-Shot Learning](#few-shot-learning) * [Bias and Fairness](#bias-and-fairness) * [Human in the Loop Learning](#human-in-the-loop-learning) * [Architectures](#architectures) * [Multimodal Transformers](#multimodal-transformers) * [Multimodal Memory](#multimodal-memory) * [Applications and Datasets](#applications-and-datasets) * [Language and Visual QA](#language-and-visual-qa) * [Language Grounding in Vision](#language-grounding-in-vision) * [Language Grouding in Navigation](#language-grouding-in-navigation) * [Multimodal Machine Translation](#multimodal-machine-translation) * [Multi-agent Communication](#multi-agent-communication) * [Commonsense Reasoning](#commonsense-reasoning) * [Multimodal Reinforcement Learning](#multimodal-reinforcement-learning) * [Multimodal Dialog](#multimodal-dialog) * [Language and Audio](#language-and-audio) * [Audio and Visual](#audio-and-visual) * [Visual, IMU and Wireless](#visual-imu-and-wireless) * [Media Description](#media-description) * [Video Generation from Text](#video-generation-from-text) * [Affect Recognition and Multimodal Language](#affect-recognition-and-multimodal-language) * [Healthcare](#healthcare) * [Robotics](#robotics) * [Autonomous Driving](#Autonomous-Driving) * [Finance](#Finance) * [Human AI Interaction](#Human-AI-Interaction) * [Workshops](#workshops) * [Tutorials](#tutorials) * [Courses](#courses) # Research Papers ## Survey Papers [Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions](https://arxiv.org/abs/2209.03430), arxiv 2023 [Multimodal Learning with Transformers: A Survey](https://arxiv
Excerpt of 74,712 characters
Read on GitHub
336
Feiyang(Vance) Chen · Creatify AI · United States
45
6
3
3
Mariya Hendriksen · University of Oxford
3
2
2
2
Chaitanya Ahuja · Meta AI, ex-CMU grad
2
1
1
Richard Chen · United States
1
1
Peng Xu （徐鹏） · Tsinghua University · China
1
1
Mark Huang · Singapore
1
Jeevan · India
1
1
Zhmin Zhao · Software Analysis and Intelligence Lab (SAIL) & Lab on Maintenance, Construction and Intelligence of Software (MCIS) · Canada
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:3971d4ff284e4ce7, topic:robotics, readme:robotics, readme:autonomous driving
matched fp:3971d4ff284e4ce7, topic:deep-learning, readme:pretraining
matched fp:3971d4ff284e4ce7, topic:natural-language-processing, readme:machine translation
matched fp:3971d4ff284e4ce7, topic:reinforcement-learning, readme:reinforcement learning