Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
The repository for the largest and most comprehensive empirical study of visual foundation models for Embodied AI (EAI).
| Date | Stars |
|---|---|
| 2026-07-31 | 508 |
| 2026-08-02 | 508 |
| 2026-08-06 | 508 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Visual Cortex and CortexBench
[Website](https://eai-vc.github.io/) | [Blog post](https://ai.facebook.com/blog/robots-learning-video-simulation-artificial-visual-cortex-vc-1) | [Paper](https://arxiv.org/abs/2303.18240)
<p align="center">
<img src="res/img/vc1_teaser.gif" alt="Visual Cortex and CortexBench" width="600">
<br />
<br />
<a href="https://opensource.fb.com/support-ukraine"><img alt="Support Ukraine" src="https://img.shields.io/badge/Support-Ukraine-FFD500?style=flat&labelColor=005BBB" /></a>
<a href="./MODEL_CARD.md"><img alt="Model Card" src="https://img.shields.io/badge/model--card-VC--1-green.svg" /></a>
<a href="./LICENSE"><img alt="CC-BY-NC License" src="https://img.shields.io/badge/license-CC--BY--NC-blue.svg" /></a>
<a href="Python 3.8"><img alt="Python 3.8" src="https://img.shields.io/badge/python-3.8-blue.svg" /></a>
<a href="https://github.com/psf/black"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg" /></a>
<a href="https://app.circleci.com/pipelines/github/facebookresearch/eai-vc/"><img alt="CicleCI Status" src="https://dl.circleci.com/status-badge/img/gh/facebookresearch/eai-vc/tree/main.svg?style=shield&circle-token=dbbc3a068a155612bcafee8483cac9bf0dda1231" /></a>
</p>
We're releasing CortexBench and our first Visual Cortex model: VC-1. CortexBench is a collection of 17 different EAI tasks spanning locomotion, navigation, dexterous and mobile manipulation. We performed the largest and most comprehensive empirical study of pre-trained visual representations (PVRs) for Embodied AI (EAI), and find that none of the existing PVRs perform well across all tasks. Next, we trained VC-1 on a combination of over 4,000 hours of egocentric videos from 7 different sources and ImageNet, totaling over 5.6 million images. We show that when adapting VC-1 (through task-specific losses or a small amount of in-domain data), VC-1 is competitive with or outperforms state of the art on all benchmark tasks.
## Open-Sourced Models
We're open-sourcing two visual cortex models ([model cards](./MODEL_CARD.md)):
* VC-1 (ViT-L): Our best model, uses a ViT-L backbone, also known simply as `VC-1` | [Download](https://dl.fbaipublicfiles.com/eai-vc/vc1_vitl.pth)
* VC-1-base (VIT-B): pre-trained on the same data as VC-1 but with a smaller backbone (ViT-B) | [Download](https://dl.fbaipublicfiles.com/eai-vc/vc1_vitb.pth)
## Installation
To install our visual cortex models and CortexBench, please follow the instructions in [INSTALLATION.md](INSTALLATION.md).
## Directory structure
- `vc_models`: contains config files for visual cortex models, the model loading code and, as well as some project utilities.
- See [README](./vc_models/README.md) for more details.
- `cortexbench`: embodied AI downstream tasks to evaluate pre-trained representations.
- `third_party`: Third party submodules which aren't expected to change often.
- `data`: Gitignored directory, needs to be created by the user. Is used by some downstream tasks to find (symlinks to) datasets, models, etc.
## Load VC-1
To use the VC-1 model, you can install the `vc_models` module with pip. Then, you can load the model with code such as the following or follow [our tutorial](./tutorial/tutorial_vc.ipynb):
```python
import vc_models
from vc_models.models.vit import model_utils
model,embd_size,model_transforms,model_info = model_utils.load_model(model_utils.VC1_LARGE_NAME)
# To use the smaller VC-1-base model use model_utils.VC1_BASE_NAME.
# The img loaded should be Bx3x250x250
img = your_function_here ...
# Output will be of size Bx3x224x224
transformed_img = model_transforms(img)
# Embedding will be 1x768
embedding = model(transformed_img)
```
## Reproducing Results with VC-1 Model
To reproduce the results with the VC-1 model, please follow the README instructions for each of the benchmarks in [`cortexbench`](./cortexbench/).
## Load Your Own Encoder Model and Run Across All Benchmarks
To load your oExcerpt of 6,624 characters
Read on GitHub
Oleksandr Maksymets · @FacebookResearch · United States
9
2
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:53696bd41710415d, desc:embodied ai