Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
ICCV 2025 | TesserAct: Learning 4D Embodied World Models
| Date | Stars |
|---|---|
| 2026-07-24 | 402 |
| 2026-07-25 | 403 |
| 2026-07-28 | 403 |
| 2026-07-30 | 403 |
| 2026-08-06 | 403 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<p align="center">
<h1 align="center">TesserAct: Learning 4D Embodied World Models</h1>
<p align="center">
ICCV 2025
</p>
<p align="center">
<a href="https://haoyuzhen.com">Haoyu Zhen*</a>,
<a href="https://qiaosun22.github.io/">Qiao Sun*</a>,
<a href="https://icefoxzhx.github.io/">Hongxin Zhang</a>,
<a href="https://senfu.github.io/">Junyan Li</a>,
<a href="https://rainbow979.github.io/">Siyuan Zhou</a>,
<a href="https://yilundu.github.io/">Yilun Du</a>,
<a href="https://people.csail.mit.edu/ganchuang">Chuang Gan</a>
</p>
<p align="center">
<a href="https://arxiv.org/abs/2504.20995">
<img src='https://img.shields.io/badge/Paper-PDF-red?style=flat&logo=arXiv&logoColor=red' alt='Paper PDF'>
</a>
<a href='https://tesseractworld.github.io' style='padding-left: 0.5rem;'>
<img src='https://img.shields.io/badge/Project-Page-blue?style=flat&logo=Google%20chrome&logoColor=blue' alt='Project Page'>
</a>
<a href='https://huggingface.co/anyeZHY/tesseract' style='padding-left: 0.5rem;'>
<img src='https://img.shields.io/badge/Model-Hugging%20Face-yellow?style=flat&logo=Hugging%20face&logoColor=yellow' alt='Model Hugging Face'>
</a>
</p>
</p>
We propose TesserAct, **the first open-source and generalized 4D World Model for robotics**, which takes input images and text instructions to generate RGB, depth, and normal videos, reconstructing a 4D scene and predicting actions.
<p align="center">
<img src="asset/teaser.png" alt="Logo" width="190%">
</p>
<br>
<!-- TABLE OF CONTENTS -->
<details open="open" style='padding: 10px; border-radius:5px 30px 30px 5px; border-style: solid; border-width: 1px;'>
<summary>Tabel of Contents</summary>
<ol>
<li>
<a href="#installation">Installation</a>
</li>
<li>
<a href="#data-preparation">Data Preparation</a>
</li>
<li>
<a href="#training">Training</a>
<ul>
<li>
<a href="#pre-training-or-full-fine-tuning">Pre-training or Full Fine-tuning</a>
</li>
<li>
<a href="#lora-fine-tuning">LoRA Fine-tuning</a>
</li>
</ul>
</li>
<li>
<a href="#inference">Inference</a>
</li>
<li>
<a href="#citation">Citation</a>
</li>
<li>
<a href="#acknowledgement">Acknowledgement</a>
</li>
</ol>
</details>
## News
- [2025-06-25] TesserAct is accepted to ICCV 2025!
- [2025-06-19] We provide an efficient RGB+Depth+Normal LoRA fine-tuning script for custom datasets.
- [2025-06-18] We provide a RGB-only LoRA inference script that achieves the best generalization ability for robotics video generation.
- [2025-06-06] We have released the training code and data generation scripts!
- [2025-05-05] We have updated the gallery and added more results on the [project website](https://tesseractworld.github.io).
- [2025-05-04] We add [USAGE.MD](doc/usage.md) to provide more details about the models and how to use the models on your own data!
- [2025-04-29] We have released the inference code and TesserAct-v0.1 model weights!
## Installation
Create a conda environment and install the required packages:
```bash
conda create -n tesseract python=3.9
conda activate tesseract
pip install -r requirements.txt
git clone https://github.com/UMass-Embodied-AGI/TesserAct.git
cd TesserAct
pip install -e .
```
## Data Preparation
Please refer to [DATA.md](DATA.md) for data generation scripts and dataset preparation.
## Training
### Pre-training or Full Fine-tuning
To pre-train the full TesserAct model from CogVideoX, we provide a training script based on [Finetrainers](https://github.com/a-r-r-o-w/finetrainers). The training code supports distributed training with multiple GPUs or multi-nodes.
To pre-train our TesserAct model, run the following command:
```bash
bash train_i2v_depth_normal_sft.sh
```
To fine-tune our released TesserAct model, modify the model loading code in [tesseract/i2v_depth_normExcerpt of 10,619 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:bbe9fcea355e5a95, topic:robotics, topic:embodied-ai, readme:robotics
matched fp:bbe9fcea355e5a95, topic:video-generation, readme:video generation