Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[ECCV2024] 🐙Octopus, an embodied vision-language model trained with RLEF, emerging superior in embodied visual planning and programming.
| Date | Stars |
|---|---|
| 2026-07-31 | 301 |
| 2026-08-06 | 301 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<p align="center" width="100%">
<img src="https://i.mji.rip/2023/10/13/c4bdc6505f2b3f2304bffb5ea196f5a2.png" width="40%" height="80%">
</p>
<div>
<div align="center">
<font size=5><strong>Octopus: Embodied Vision-Language Programmer from Environmental Feedback</strong></font>
<br>
<a href='https://jingkang50.github.io/' target='_blank'>Jingkang Yang<sup>*,1</sup></a> 
<a href='https://github.com/dongyh20/' target='_blank'>Yuhao Dong<sup>*,2,5</sup></a> 
<a href='https://github.com/choiszt/' target='_blank'>Shuai Liu<sup>*,3,5</sup></a> 
<a href='https://brianboli.com/' target='_blank'>Bo Li<sup>*,1</sup></a> 
</br>
Ziyue Wang<sup>†,1</sup></a> 
Chencheng Jiang<sup>†,4</sup></a> 
Haoran Tan<sup>†,3</sup></a> 
Jiamu Kang<sup>†,2</sup></a> 
</br>
<a href='https://zhangyuanhan-ai.github.io/' target='_blank'>Yuanhan Zhang<sup>1</sup></a> 
<a href='https://kaiyangzhou.github.io/' target='_blank'>Kaiyang Zhou<sup>1</sup></a> 
<a href='https://liuziwei7.github.io/' target='_blank'>Ziwei Liu<sup>1,5,✉</sup></a>
</div>
<div align="center">
<sup>1</sup>S-Lab, Nanyang Technological University 
<sup>2</sup>Tsinghua University 
<br>
<sup>3</sup>Beijing University of Posts and Telecommunications  
<br>
<sup>4</sup>Xi'an Jiaotong University 
<sup>5</sup>Shanghai AI Laboratory 
</br>
<sup>*</sup> Equal Contribution 
<sup>†</sup> Equal Engineering Contribution 
<sup>✉</sup> Corresponding Author
</div>
-----------------
[](https://hits.seeyoufarm.com)
[Project Page](https://choiszt.github.io/Octopus) | [Octopus Paper](https://arxiv.org/abs/2310.08588) | [Demo Video](https://www.youtube.com/watch?v=tmSNw2XonxI)
## 🐙 Introducing Octopus
Octopus is a novel VLM designed to proficiently decipher an agent’s vision and textual task objectives and to formulate intricate action sequences and generate executable code. We provide two models based on the following architectures. Please click
- [LLaVA](./octopus/LLaVA/README.md)
- [Otter](./octopus/Otter/README.md)
## 🪸 Introducing OctoVerse
OctoVerse contains three sub-worlds
| | OS (tested) | Environment Goal |
|---------------|------------------|------------------------------------|
| **OctoGibson** | Ubuntu 20.04 | 500 Tasks on OmniGibson |
| **OctoGTA** | Windows 11 | 20 Tasks to evaluate transfer learning |
| **OctoMC** | Ubuntu/Windows | 20 Tasks to evaluate transfer learning on MineCraft worlds, such as making an axe. |
- Training data collection pipeline in `octogibson` environment
- Evaluation pipeline in `octogibson` environment
- Evaluation pipeline in `octogta` environment
- Training pipeline of the `octopus` model
**Contact: Leave issue or contact `[email protected]` and `[email protected]`. We are on call to respond.**
## 🦾 Updates
**[2023-10]**
1. 🤗 Introducing Project Octopus' homepage: https://choiszt.github.io/Octopus.
2. 🤗 Check our [paper](https://arxiv.org/abs/???) introducing Octopus in details.
## 🏁 Get Started
1. **Training Data Collection:** For data collection from `octogibson` environment, you need to set up two conda environments: `omnigibson` and `gpt4`. The `omnigibson` environment has an agent to act following the instructioExcerpt of 5,178 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:a5243c1b663f22ff, desc:vision-language, desc:vision language model