Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Unofficial PyTorch implementation of "RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving" (ECCV 2020)
| Date | Stars |
|---|---|
| 2026-07-24 | 302 |
| 2026-07-25 | 302 |
| 2026-07-28 | 302 |
| 2026-07-30 | 302 |
| 2026-08-06 | 302 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# RTM3D-PyTorch
[![python-image]][python-url]
[![pytorch-image]][pytorch-url]
The PyTorch Implementation of the paper:
[RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving](https://arxiv.org/pdf/2001.03343.pdf) (ECCV 2020)
---
## Demonstration
## Features
- [x] Realtime 3D object detection based on a monocular RGB image
- [x] Support [distributed data parallel training](https://github.com/pytorch/examples/tree/master/distributed/ddp)
- [x] Tensorboard
- [x] ResNet-based **K**eypoint **F**eature **P**yramid **N**etwork (KFPN) (Using by setting `--arch fpn_resnet_18`)
- [ ] Use images from both left and right cameras (Control by setting the `use_left_cam_prob` argument)
- [ ] Release pre-trained models
## Some modifications from the paper
- _**Formula (3)**_:
- A negative value can't be an input of the `log` operator, so please **don't normalize dim** as mentioned in
the paper because the normalized dim values maybe less than `0`. Hence I've directly regressed to absolute dimension values in meters.
- Use `L1 loss` for depth estimation (applying the `sigmoid` activation to the depth output first).
- _**Formula (5)**_: I haven't taken the absolute values of the ground-truth,
I have used the **relative values** instead. [The code is here](https://github.com/maudzung/RTM3D/blob/45b9d8af1298a6ad7dacb99a8f538f285696ded4/src/data_process/kitti_dataset.py#L284)
- _**Formula (7)**_: `argmin` instead of `argmax`
- Generate heatmap for the center and vertexes of objects as the CenterNet paper. If you want to use the strategy from RTM3D paper,
you can pass the `dynamic-sigma` argument to the `train.py` script.
## 2. Getting Started
### 2.1. Requirement
```shell script
pip install -U -r requirements.txt
```
### 2.2. Data Preparation
Download the 3D KITTI detection dataset from [here](http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d).
The downloaded data includes:
- Training labels of object data set _**(5 MB)**_
- Camera calibration matrices of object data set _**(16 MB)**_
- **Left color images** of object data set _**(12 GB)**_
- **Right color images** of object data set _**(12 GB)**_
Please make sure that you construct the source code & dataset directories structure as below.
### 2.3. RTM3D architecture
The model takes **only the RGB images** as the input and outputs the `main center heatmap`, `vertexes heatmap`,
and `vertexes coordinate` as the base module to estimate `3D bounding box`.
### 2.4. How to run
#### 2.4.1. Visualize the dataset
```shell script
cd src/data_process
```
- To visualize camera images with 3D boxes, let's execute:
```shell script
python kitti_dataset.py
```
Then _Press **n** to see the next sample >>> Press **Esc** to quit..._
#### 2.4.2. Inference
Download the trained model from [**_here_**](https://drive.google.com/drive/folders/1lKOLHhWZasoC7cKNLcB714LBDS91whCr?usp=sharing) (will be released),
then put it to `${ROOT}/checkpoints/` and execute:
```shell script
python test.py --gpu_idx 0 --arch resnet_18 --pretrained_path ../checkpoints/rtm3d_resnet_18.pth
```
#### 2.4.3. Evaluation
```shell script
python evaluate.py --gpu_idx 0 --arch resnet_18 --pretrained_path <PATH>
```
#### 2.4.4. Training
##### 2.4.4.1. Single machine, single gpu
```shell script
python train.py --gpu_idx 0 --arch <ARCH> --batch_size <N> --num_workers <N>...
```
##### 2.4.4.2. Multi-processing Distributed Data Parallel Training
We should always use the `nccl` backend for multi-processing distributed training since it currently provides the best
distributed training performance.
- **Single machine (node), multiple GPUs**
```shell script
python train.py --dist-url 'tcp://127.0.0.1:29500' --dist-backend 'nccl' --multiprocessing-distributed --world-size 1 --rank 0
```
- **Two machines (two nodes), multiple GPUs**
_**First machine**_
```shell script
python train.pyExcerpt of 10,881 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:00308bf16a3de7cb, topic:autonomous-driving, topic:self-driving-car, desc:autonomous driving
matched fp:00308bf16a3de7cb, topic:pytorch, readme:distributed training, readme:data parallel