Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
This codebase demonstrates how to synthesize realistic 3D character animations given an arbitrary speech signal and a static character mesh.
| Date | Stars |
|---|---|
| 2026-07-24 | 1258 |
| 2026-07-25 | 1258 |
| 2026-07-28 | 1259 |
| 2026-07-30 | 1259 |
| 2026-08-06 | 1259 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# [VOCA: Voice Operated Character Animation](https://voca.is.tue.mpg.de) This is an official [VOCA](https://voca.is.tue.mpg.de) repository. <p align="center"> <img src="gif/speech_driven_animation.gif"> </p> VOCA is a simple and generic speech-driven facial animation framework that works across a range of identities. This codebase demonstrates how to synthesize realistic character animations given an arbitrary speech signal and a static character mesh. For details please see the scientific publication ``` Capture, Learning, and Synthesis of 3D Speaking Styles. D. Cudeiro*, T. Bolkart*, C. Laidlaw, A. Ranjan, M. J. Black Computer Vision and Pattern Recognition (CVPR), 2019 ``` A pre-print of the publication can be found [here]( https://ps.is.tuebingen.mpg.de/uploads_file/attachment/attachment/510/paper_final.pdf). You can also check out the [VOCA Blender Addon](https://github.com/SasageyoOrg/voca-blender) ## Video See the demo video for more details and results. [](https://youtu.be/XceCxf_GyW4) ## Set-up The code uses Python 3.6.8 and it was tested on Tensorflow 1.14.0. Install pip and virtualenv ``` sudo apt-get install python3-pip python3-venv ``` Install ffmpeg ``` sudo apt install ffmpeg ``` Clone the git project: ``` git clone https://github.com/TimoBolkart/voca.git ``` Set up virtual environment: ``` mkdir <your_home_dir>/.virtualenvs python3.7 -m venv <your_home_dir>/.virtualenvs/voca ``` Activate virtual environment: ``` cd voca source <your_home_dir>/voca/bin/activate ``` Set the right pip version: ``` pip install -U pip==22.0.4 ``` Install mesh processing libraries from [MPI-IS/mesh](https://github.com/MPI-IS/mesh) within the virtual environment. Finally, the requirements (including tensorflow) can be installed using: ``` pip install -r requirements.txt ``` ## Data #### Data to run the demo Download the trained VOCA model, audio sequences, and template meshes from [MPI-IS/VOCA](https://voca.is.tue.mpg.de).<br/> Download FLAME model from [MPI-IS/FLAME](http://flame.is.tue.mpg.de/).<br/> Download the trained DeepSpeech model (v0.1.0) from [Mozilla/DeepSpeech](https://github.com/mozilla/DeepSpeech/releases/tag/v0.1.0) (i.e. deepspeech-0.1.0-models.tar.gz). To download and prepare these data, run: ``` ./fetch_data.sh ``` #### Data used to train VOCA VOCA is trained on VOCASET, a unique 4D face dataset with about 29 minutes of 4D scans captured at 60 fps and synchronized audio from 12 speakers that can be downloaded at [MPI-IS/VOCASET](https://voca.is.tue.mpg.de). Training subjects: ``` FaceTalk_170728_03272_TA, FaceTalk_170904_00128_TA, FaceTalk_170725_00137_TA, FaceTalk_170915_00223_TA, FaceTalk_170811_03274_TA, FaceTalk_170913_03279_TA, FaceTalk_170904_03276_TA, FaceTalk_170912_03278_TA ``` This is also the order of the subjects for the one-hot-encoding (i.e. FaceTalk_170728_03272_TA: 0, FaceTalk_170904_00128_TA: 1, ...) Validation subjects: ``` FaceTalk_170811_03275_TA, FaceTalk_170908_03277_TA ``` Test subjects: ``` FaceTalk_170809_00138_TA, FaceTalk_170731_00024_TA ``` ## Demo We provide demos to 1) synthesize a character animation given an speech signal (VOCA), 2) add eye blinks, alter identity dependent face shape and head pose of an animation sequence using FLAME, and 3) generate templates (e.g. by sampling the [FLAME](http://flame.is.tue.mpg.de/) identity shape space, or by reconstructing a template from an image using [RingNet](https://github.com/soubhiksanyal/RingNet) that can be animated with VOCA.) ##### VOCA output This demo runs VOCA, which outputs the animation meshes given audio sequences, and renders the animation sequence to a video. ``` python run_voca.py --tf_model_fname './model/gstep_52280.model' --ds_fname './ds_graph/output_graph.pb' --audio_fname './audio/test_sentence.wav' --template_fname './template/FLAME_sample.ply' --condition_idx 3 --out_path './animation_output' ``` To run VOCA and visualize the meshes
Excerpt of 10,159 characters
Read on GitHub
Max Planck Institute for Intelligent Systems
21
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:21e4259aeaf10839, topic:computer-vision, readme:computer vision
matched fp:21e4259aeaf10839, topic:tensorflow