Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[NeurIPS 2021] Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation
| Date | Stars |
|---|---|
| 2026-07-24 | 568 |
| 2026-07-25 | 568 |
| 2026-07-28 | 568 |
| 2026-07-30 | 568 |
| 2026-08-06 | 568 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# STCN
## Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation
[Ho Kei Cheng](https://hkchengrex.github.io/), Yu-Wing Tai, Chi-Keung Tang
NeurIPS 2021
[[arXiv]](https://arxiv.org/abs/2106.05210) [[PDF]](https://arxiv.org/pdf/2106.05210) [[Project Page]](https://hkchengrex.github.io/STCN/) [[Papers with Code]](https://paperswithcode.com/task/semi-supervised-video-object-segmentation)
## Check out our new work [Cutie](https://github.com/hkchengrex/Cutie)!
 
*News:* In the [YouTubeVOS 2021 challenge](https://youtube-vos.org/challenge/2021/leaderboard/), STCN achieved 1st place accuracy in novel (unknown) classes and 2nd place in overall accuracy. Our solution is also fast and light.
We present Space-Time Correspondence Networks (STCN) as the new, effective, and efficient framework to model space-time correspondences in the context of video object segmentation.
STCN achieves SOTA results on multiple benchmarks while running fast at 20+ FPS without bells and whistles. Its speed is even higher with mixed precision.
Despite its effectiveness, the network itself is very simple with lots of room for improvement. See the paper for technical details.
**UPDATE (15-July-2021)**
1. CBAM block: We tried without CBAM block and I would say that we don't really need it. For s03 model, we get -1.2 in DAVIS and +0.1 in YouTubeVOS. For s012 model, we get +0.1 in DAVIS and +0.1 in YouTubeVOS. You are welcome to drop this block (see `no_cbam` branch). Overall, the much larger YouTubeVOS seems to be a better evaluation benchmark for consistency.
**UPDATE (22-Aug-2021)**
2. Reproducibility: We have updated the package requirements below. With that environment, we obtained DAVIS J&F in the range of [85.1, 85.5] across multiple runs on two different machines.
**UPDATE (27-Apr-2022)**
Multi-scale testing code (as in the paper) has been added [here](https://github.com/hkchengrex/STCN/tree/ms/ms).
## What do we have here?
1. [A gentle introduction](#a-gentle-introduction)
2. [Quantitative results and precomputed outputs](#results)
1. DAVIS 2016
2. DAVIS 2017 validation/test-dev
3. YouTubeVOS 2018/2019
3. [Try our model on your own data (Interactive GUI available)](#try-your-own-data)
4. Steps to reproduce
1. [Pretrained models](#pretrained-models)
2. [Inference](#inference)
3. [Training](#training)
5. [If you want to look closer](#looking-closer)
6. [Citation](#citation)
## A Gentle Introduction
There are two main contributions: STCN framework (above figure), and L2 similarity. We build affinity between images instead of between (image, mask) pairs -- this leads to a significantly speed up, memory saving (because we compute one, instead of multiple affinity matrices), and robustness. We further use L2 similarity to replace dot product, which improves the memory bank utilization by a great deal.
### Perks
- Simple, runs fast (30+ FPS with mixed precision; 20+ without)
- High performance
- Still lots of room to improve upon (e.g. locality, memory space compression)
- Easy to train: just two 11GB GPUs, no V100s needed
## Requirements
We used these packages/versions in the development of this project.
- PyTorch `1.8.1`
- torchvision `0.9.1`
- OpenCV `4.2.0`
- [Pillow-SIMD](https://github.com/uploadcare/pillow-simd) `7.0.0.post3`
- progressbar2
- [thinspline](https://github.com/cheind/py-thin-plate-spline) for training (`pip install git+https://github.com/cheind/py-thin-plate-spline`)
- gitpython for training
- gdown for downloading pretrained models
- [Other packages in my environment](docs/packages.txt), for reference only.
Refer to the official [PyTorch guide](<https://pytorch.org/>) for installing PyTorch/torchvision, and the [pillow-simd](https://github.com/uploadcare/pillow-simd) guide to install Pillow-SIMD. The rest can be installed by:
Excerpt of 15,512 characters
Read on GitHub
Rex Cheng · Israel
40
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:48898135b13e318e, topic:deep-learning, topic:pytorch
matched fp:48898135b13e318e, topic:computer-vision