Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Codebase for ECCV18 "The Sound of Pixels"
| Date | Stars |
|---|---|
| 2026-07-24 | 393 |
| 2026-07-25 | 393 |
| 2026-07-28 | 393 |
| 2026-07-30 | 393 |
| 2026-08-06 | 393 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Sound-of-Pixels
Codebase for ECCV18 "The Sound of Pixels".
*This repository is under construction, but the core parts are already there.
<img src="./teaser.png"/>
## Environment
The code is developed under the following configurations.
- Hardware: 1-4 GPUs (change ```[--num_gpus NUM_GPUS]``` accordingly)
- Software: Ubuntu 16.04.3 LTS, ***CUDA>=8.0, Python>=3.5, PyTorch>=0.4.0***
## Training
1. Prepare video dataset.
a. Download MUSIC dataset from: https://github.com/roudimit/MUSIC_dataset
b. Download videos.
2. Preprocess videos. You can do it in your own way as long as the index files are similar.
a. Extract frames at 8fps and waveforms at 11025Hz from videos. We have following directory structure:
```
data
├── audio
| ├── acoustic_guitar
│ | ├── M3dekVSwNjY.mp3
│ | ├── ...
│ ├── trumpet
│ | ├── STKXyBGSGyE.mp3
│ | ├── ...
│ ├── ...
|
└── frames
| ├── acoustic_guitar
│ | ├── M3dekVSwNjY.mp4
│ | | ├── 000001.jpg
│ | | ├── ...
│ | ├── ...
│ ├── trumpet
│ | ├── STKXyBGSGyE.mp4
│ | | ├── 000001.jpg
│ | | ├── ...
│ | ├── ...
│ ├── ...
```
b. Make training/validation index files by running:
```
python scripts/create_index_files.py
```
It will create index files ```train.csv```/```val.csv``` with the following format:
```
./data/audio/acoustic_guitar/M3dekVSwNjY.mp3,./data/frames/acoustic_guitar/M3dekVSwNjY.mp4,1580
./data/audio/trumpet/STKXyBGSGyE.mp3,./data/frames/trumpet/STKXyBGSGyE.mp4,493
```
For each row, it stores the information: ```AUDIO_PATH,FRAMES_PATH,NUMBER_FRAMES```
3. Train the default model.
```bash
./scripts/train_MUSIC.sh
```
5. During training, visualizations are saved in HTML format under ```ckpt/MODEL_ID/visualization/```.
## Evaluation
0. (Optional) Download our trained model weights for evaluation.
```bash
./scripts/download_trained_model.sh
```
1. Evaluate the trained model performance.
```bash
./scripts/eval_MUSIC.sh
```
## Reference
If you use the code or dataset from the project, please cite:
```bibtex
@InProceedings{Zhao_2018_ECCV,
author = {Zhao, Hang and Gan, Chuang and Rouditchenko, Andrew and Vondrick, Carl and McDermott, Josh and Torralba, Antonio},
title = {The Sound of Pixels},
booktitle = {The European Conference on Computer Vision (ECCV)},
month = {September},
year = {2018}
}
```
Excerpt of 2,535 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:d13ce01a6fc669ed, topic:computer-vision, readme:computer vision