Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Implementation of π₀, the robotic foundation model architecture proposed by Physical Intelligence
| Date | Stars |
|---|---|
| 2026-07-31 | 581 |
| 2026-08-06 | 582 |
Today
+1 stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<img src="./fig3.png" width="400px"></img>
## pi-zero-pytorch
Implementation of <a href="https://www.physicalintelligence.company/blog/pi0">π₀</a> the robotic foundation model architecture proposed by Physical Intelligence
Summary of this work would be that it is a simplified <a href="https://github.com/lucidrains/transfusion-pytorch">Transfusion</a> (Zhou et al.) with influence from <a href="https://arxiv.org/abs/2403.03206">Stable Diffusion 3</a> (Esser et al.), mainly the adoption of flow matching instead of diffusion for policy generation, as well as the separation of parameters (<a href="https://github.com/lucidrains/mmdit/blob/main/mmdit/mmdit_pytorch.py#L43">Joint Attention</a> from mmDIT). They build on top of a pretrained vision language model, PaliGemma 2B.
Update: The [official repository](https://github.com/Physical-Intelligence/openpi) has been open sourced!
### Appreciation
- [Einops](https://github.com/arogozhnikov/einops) for the amazing [pack and unpack](https://einops.rocks/4-pack-and-unpack/), used extensively here for managing various token sets
- [Flex Attention](https://pytorch.org/blog/flexattention/) for allowing for easy mixture of autoregressive and bidirectional attention
- [@Wonder1905](https://github.com/Wonder1905) for the code review and identifying issues
- [Pranoy](https://github.com/pranoyr) for various bug fixes!
- You? maybe a phd student who wants to contribute to the latest SOTA architecture?
### Install
```bash
$ pip install pi-zero-pytorch
```
### Usage
```python
import torch
from pi_zero_pytorch import π0
model = π0(
dim = 512,
dim_action_input = 6,
dim_joint_state = 12,
num_tokens = 20_000
)
vision = torch.randn(1, 1024, 512)
commands = torch.randint(0, 20_000, (1, 1024))
joint_state = torch.randn(1, 12)
actions = torch.randn(1, 32, 6)
loss, _ = model(vision, commands, joint_state, actions)
loss.backward()
# after much training
sampled_actions = model(vision, commands, joint_state, trajectory_length = 32) # (1, 32, 6)
```
To do online learning, just wrap the model with the `EFPO` class
```python
from pi_zero_pytorch import π0, EFPO
# you'll want to supply your own environment
from pi_zero_pytorch.mock_env import Env
mock_env = Env((256, 256), 2, 32, 1024, 12)
# pass your agent and environment to EFPO for learning to be orchestrated
epo = EFPO(model)
# gather memories from environment
memories = epo.gather_experience_from_env(mock_env, steps = 10)
# learn from memories
epo.learn_agent(memories, batch_size = 2)
```
### Contributing
At the project root, run
```bash
$ pip install '.[test]' # or `uv pip install '.[test]'`
```
Then add your tests to `tests/test_pi_zero.py` and run
```bash
$ pytest tests/
```
That's it
### Citation
```bibtex
@misc{Black2024,
author = {Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, Ury Zhilinsky},
url = {https://www.physicalintelligence.company/download/pi0.pdf}
}
```
```bibtex
@inproceedings{Zhou2024ValueRL,
title = {Value Residual Learning For Alleviating Attention Concentration In Transformers},
author = {Zhanchao Zhou and Tianyi Wu and Zhiyun Jiang and Zhenzhong Lan},
year = {2024},
url = {https://api.semanticscholar.org/CorpusID:273532030}
}
```
```bibtex
@inproceedings{Darcet2023VisionTN,
title = {Vision Transformers Need Registers},
author = {Timoth'ee Darcet and Maxime Oquab and Julien Mairal and Piotr Bojanowski},
year = {2023},
url = {https://api.semanticscholar.org/CorpusID:263134283}
}
```
```bibtex
@inproceedings{Sadat2024EliminatingOA,
title = {Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models},
author = {SExcerpt of 12,753 characters
Read on GitHub
Phil Wang · United States
288
Pranoy · IIT Madras · India
5
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:e62605eef1893dac, topic:deep-learning
matched fp:e62605eef1893dac, topic:robotics