Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[CVPR 2024 - Oral] Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences
| Date | Stars |
|---|---|
| 2026-07-24 | 592 |
| 2026-07-25 | 592 |
| 2026-07-28 | 592 |
| 2026-07-30 | 592 |
| 2026-08-06 | 592 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<div style="text-align: center;">
<h1>Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences</h1>
<p>
<a href="https://scholar.google.com/citations?user=m_SPRGUAAAAJ&hl=en">Axel Barroso-Laguna</a>
·
<a href="https://scholar.google.com/citations?user=l-zRzDEAAAAJ&hl=en">Sowmya Munukutla</a>
·
<a href="https://www.robots.ox.ac.uk/~victor/">Victor Adrian Prisacariu</a>
·
<a href="https://ebrach.github.io/">Eric Brachmann</a>
</p>
<h2 style="font-size:1.7em; margin-top: -0.5rem; margin-bottom: -0.5rem;">CVPR 2024 (Oral)</h2>
<h3><a href="https://nianticlabs.github.io/mickey/">Project Page</a> | <a href="https://storage.googleapis.com/niantic-lon-static/research/mickey/mickey_main_paper.pdf">Paper</a> | <a href="https://arxiv.org/abs/2404.06337">arXiv</a> | <a href="https://storage.googleapis.com/niantic-lon-static/research/mickey/mickey_supp.pdf">Supplemental</a></h3>
</div>
This is the reference implementation of the paper **"Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences"** presented at **CVPR 2024**.
The paper introduces **M**etr**ic Key**points (MicKey), a feature detection pipeline that regresses keypoint positions in camera space.
MicKey presents a differentiable approach to establish metric correspondences via descriptor matching. From the metric correspondences, MicKey recovers metric relative poses.
MicKey is trained in an end-to-end fashion using differentiable pose optimization and requires only image pairs and their ground truth relative poses for supervision.
<p align="center">
<img src="resources/teaser_mickey.png" alt="teaser" width="90%">
</p>
## Setup
Assuming a fresh [Anaconda](https://www.anaconda.com/download/) distribution, you can install dependencies with:
```shell
conda env create -f resources/environment.yml
conda activate mickey
```
We ran our experiments with PyTorch 2.0.1, CUDA 11.6, Python 3.8.17 and Debian GNU/Linux 11.
## Evaluating MicKey
MicKey aims at addressing the problem of instant Augmented Reality (AR) introduced in the [Map-free benchmark](https://research.nianticlabs.com/mapfree-reloc-benchmark).
In the Map-free set up, instead of building 3D maps from hundreds of images and scale calibrations, they propose to use only one photo of a scene as the map.
The Map-free benchmark then evaluates how accurate is the estimated metric relative pose between the reference image (the map)
and the query image (the user).
### Download Map-free dataset
You can find the Map-free dataset in [their project page](https://research.nianticlabs.com/mapfree-reloc-benchmark/dataset).
Extract the test.zip file into `data/mapfree`. Optionally, if you want to train MicKey, also download train and val zip files.
### Pre-trained Models
We provide two [MicKey models](https://storage.googleapis.com/niantic-lon-static/research/mickey/assets/mickey_weights.zip).
* _mickey.ckpt_: These are the default weights for MicKey, without using the overlapping scores provides in Map-free dataset and following the curriculum learning strategy described in the paper.
* _mickey_sc.ckpt_: These are the weights when training MicKey using the min and max overlapping scores defined in Map-free.
Extract mickey_weights.zip into `weights/`. In the zip file, we also provide the default configuration needed to run the evaluation.
### Run the submission script
Similar to Map-free code base, we provide a [submission script](submission.py) to generate submission files:
```shell
python submission.py --config path/to/config --checkpoint path/to/checkpoint --o results/your_method
```
The resulting file `results/your_method/submission.zip` can be uploaded to the Map-free [online benchmark website](https://research.nianticlabs.com/mapfree-reloc-benchmark) and compared against existing methods in the [leaderboard](https://research.nianticlabs.com/mapfree-reloc-benchmark/leaderboard).
### Run the local evaluation
The Map-free benchmark does not pExcerpt of 9,149 characters
Read on GitHub
3
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:26068b51c3832cb6, topic:computer-vision, topic:pose-estimation