Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Optical Flow Prediction with TensorFlow. Implements "PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume," by Deqing Sun et al. (CVPR 2018)
| Date | Stars |
|---|---|
| 2026-07-24 | 530 |
| 2026-07-25 | 530 |
| 2026-07-28 | 530 |
| 2026-07-30 | 530 |
| 2026-08-06 | 530 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Optical Flow Prediction with Tensorflow
This repo provides a TensorFlow-based implementation of the wonderful paper "PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume," by Deqing Sun et al. (CVPR 2018).
There are already a few attempts at implementing PWC-Net using TensorFlow out there. However, they either use outdated architectures of the paper's CNN networks, only provide TF inference (no TF training), only work on Linux platforms, and do not support multi-GPU training.
This implementation provides **both TF-based training and inference**. It is **portable**: because it doesn't use any dynamically loaded CUDA-based TensorFlow user ops, it **works on Linux and Windows**. It also **supports multi-GPU training** (the notebooks and results shown here were collected on a GTX 1080 Ti paired with a Titan X). The code also allows for **mixed-precision training**.
Finally, as shown in the ["Links to pre-trained models"](#links) section, we achieve better results than the ones reported in the official paper on the challenging MPI-Sintel 'final' dataset.
# Table of Contents
- [Background](#background)
- [Environment Setup](#environment-setup)
- [Links to pre-trained models](#links)
- [PWC-Net](#pwc-net)
+ [Basic Idea](#pwc-net-basic-idea)
+ [Network](#pwc-net-network)
+ [Jupyter Notebooks](#pwc-net-jupyter-notebooks)
+ [Training](#pwc-net-training)
* [Multisteps learning rate schedule](#pwc-net-training-multisteps)
* [Cyclic learning rate schedule](#pwc-net-training-cyclic)
* [Mixed-precision training](#pwc-net-training-mixed-precision)
+ [Evaluation](#pwc-net-eval)
+ [Inference](#pwc-net-predict)
* [Running inference on the test split of a dataset](#pwc-net-predict-dataset)
* [Running inference on image pairs](#pwc-net-predict-img-pairs)
- [Datasets](#datasets)
- [References](#references)
- [Acknowledgments](#acknowledgments)
# Background
The purpose of **optical flow estimation** is to generate a dense 2D real-valued (u,v vector) map of the motion occurring from one video frame to the next. This information can be very useful when trying to solve computer vision problems such as **object tracking, action recognition, video object segmentation**, etc.
Figure [[2017a]](#2017a) (a) below shows training pairs (black and white frames 0 and 1) from the [Middlebury](http://vision.middlebury.edu/flow/) Optical Flow dataset as well as the their color-coded optical flow ground truth. Figure (b) indicates the color coding used for easy visualization of the (u,v) flow fields. Usually, vector orientation is represented by color hue while vector length is encoded by color saturation:
The most common measures used to evaluate the quality of optical flow estimation are **angular error (AE)** and **endpoint error (EPE)**. The angular error between two optical flow vectors *(u<sub>0</sub>, v<sub>0</sub>)* and *(u<sub>1</sub>, v<sub>1</sub>)* is defined as *arccos((u<sub>0</sub>, v<sub>0</sub>) . (u<sub>1</sub>, v<sub>1</sub>))*. The endpoint error measures the distance between the endpoints of two optical flow vectors *(u<sub>0</sub>, v<sub>0</sub>)* and *(u<sub>1</sub>, v<sub>1</sub>)* and is defined as *sqrt((u<sub>0</sub> - u<sub>1</sub>)<sup>2</sup> + (v<sub>0</sub> - v<sub>1</sub>)<sup>2</sup>)*.
# Environment Setup <a name="environment-setup"></a>
The code in this repo was developed and tested using Anaconda3 v.5.2.0. To reproduce our conda environment, please refer to the following files:
*On Ubuntu:*
- [`conda list`](tfoptflow/setup/dlubu36.txt) and [`conda env export`](tfoptflow/setup/dlubu36.yml)
*On Windows:*
- [`conda list`](tfoptflow/setup/dlwin36.txt) and [`conda env export`](tfoptflow/setup/dlwin36.yml)
# Links to pre-trained models <a name="links"></a>
Pre-trained models can be found [here](http://bit.ly/tfoptflow). They come in two flavors: "small" (`sm`, with 4,705,064 learned parameters) models don't use dense connections or residual Excerpt of 27,192 characters
Read on GitHub
Phil Ferriere · Freelance · United States
27
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:4fac27daf2df1a75, topic:deep-learning, topic:tensorflow
matched fp:4fac27daf2df1a75, topic:computer-vision, readme:computer vision, readme:object tracking