Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
CALVIN - A benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks
| Date | Stars |
|---|---|
| 2026-07-24 | 962 |
| 2026-07-25 | 962 |
| 2026-07-28 | 964 |
| 2026-07-30 | 964 |
| 2026-08-06 | 964 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# CALVIN [](https://github.com/psf/black) [](https://lgtm.com/projects/g/mees/calvin/context:python) [](https://lgtm.com/projects/g/mees/calvin/alerts/) [](https://opensource.org/licenses/MIT) [<b>CALVIN - A benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks</b>](https://arxiv.org/pdf/2112.03227.pdf) [Oier Mees](https://www.oiermees.com/), [Lukas Hermann](https://lukashermann.github.io/), [Erick Rosete](https://www.erickrosete.com/), [Wolfram Burgard](http://www2.informatik.uni-freiburg.de/~burgard) #### CALVIN won the 2022 IEEE Robotics and Automation Letters (RA-L) Best Paper Award! We present **CALVIN** (**C**omposing **A**ctions from **L**anguage and **Vi**sio**n**), an open-source simulated benchmark to learn long-horizon language-conditioned tasks. Our aim is to make it possible to develop agents that can solve many robotic manipulation tasks over a long horizon, from onboard sensors, and specified only via human language. CALVIN tasks are more complex in terms of sequence length, action space, and language than existing vision-and-language task datasets and supports flexible specification of sensor suites.  # :computer: Quick Start To begin, clone this repository locally ```bash git clone --recurse-submodules https://github.com/mees/calvin.git $ export CALVIN_ROOT=$(pwd)/calvin ``` Install requirements: ```bash $ cd $CALVIN_ROOT $ conda create -n calvin_venv python=3.8 # or use virtualenv $ conda activate calvin_venv $ sh install.sh ``` If you encounter problems installing pyhash, you might have to downgrade setuptools to a version below 58. Download dataset (choose which split you want to download with the argument `D`, `ABC` or `ABCD`): \ If you want to get started without downloading the whole dataset, use the argument `debug` to download a small debug dataset (1.3 GB). ```bash $ cd $CALVIN_ROOT/dataset $ sh download_data.sh D | ABC | ABCD | debug ``` ## :weight_lifting_man: Train Baseline Agent Train baseline models: ```bash $ cd $CALVIN_ROOT/calvin_models/calvin_agent $ python training.py datamodule.root_data_dir=/path/to/dataset/ datamodule/datasets=vision_lang_shm ``` The `vision_lang_shm` option loads the CALVIN dataset into shared memory at the beginning of the training, speeding up the data loading during training. The preparation of the shared memory cache will take some time (approx. 20 min at our SLURM cluster). \ If you want to use the original data loader (e.g. for debugging) just override the command with `datamodule/datasets=vision_lang`. \ For an additional speed up, you can disable the evaluation callbacks during training by adding `~callbacks/rollout` and `~callbacks/rollout_lh` You want to scale your training to a multi-gpu setup? Just specify the [number of GPUs](https://pytorch-lightning.readthedocs.io/en/latest/advanced/multi_gpu.html#select-gpu-devices) and DDP will automatically be used for training thanks to [Pytorch Lightning](https://www.pytorchlightning.ai/). To train on all available GPUs: ```bash $ python training.py trainer.gpus=-1 ``` If you have access to a Slurm cluster, follow this [guide](https://github.com/mees/calvin/blob/main/slurm_scripts/README.md). You can use [Hydra's](https://hydra.cc/) flexible overriding system for changing hyperparameters. For example, to train a model with rgb images from both static camera and the gripper camera with relative actions: ```bash $ python training.py datamodule/observation_space=lang_rgb_static_gripper_rel_act model/perceptual_encoder=gripper_cam ``` To train a model with RGB-D from both cameras: ```bash $ python training
Excerpt of 21,070 characters
Read on GitHub
220
41
7
3
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:1600e1ca1b489148, topic:robotics, topic:manipulation, readme:robotics
matched fp:1600e1ca1b489148, topic:deep-learning, topic:pytorch