- Installation
- Introduction
- Project structure
- Dataset
- Project usage
- Testing
- Continuous integration & delivery
- Contributing
- Todo
- License
- Resources
This project uses:
- Python 3.11+
- PyTorch 2.2+
- uv package manager
git clone https://github.com/filippogiruzzi/voice_activity_detection.git
cd voice_activity_detection/# Install uv if you don't have it
curl -LsSf https://astral.sh/uv/install.sh | sh
# Install all dependencies
uv sync# Install with dev dependencies (ruff, pytest, etc.)
uv sync
# Run linters
make lint
# Run tests
make test
# Auto-format code
make formatBuild and run the CPU Docker image:
make build
make local-nobuildFor GPU support (requires NVIDIA Docker runtime):
make build-gpuThe purpose of this project is to design and implement a real-time Voice Activity Detection algorithm based on Deep Learning.
The designed solution is based on a simple pipeline with MFCC feature extraction and a small 1D-ResNet model (PyTorch) that classifies whether an audio signal is speech or noise.
| Model | Train acc. | Val acc. | Test acc. |
|---|---|---|---|
| 1D-Resnet | 99 % | 98 % | 97 % |
Raw and post-processed inference results on a test audio signal are shown below.
Each audio window of SEQ_LEN = 1024 samples (16 kHz) is converted into a
16 × 65 feature tensor stacking:
- 5 MFCC coefficients,
- 5 MFCC deltas (1st order),
- 5 MFCC delta-deltas (2nd order),
- 1 RMS energy.
These features feed a configurable 1D-ResNet (vad.model.Resnet1D):
stacked residual blocks (3 × Conv1d → BatchNorm1d with a 1×1 shortcut) →
global average pooling → a fully connected head producing a single speech logit.
The architecture is fully described by the ModelConfig dataclass, so the same
configuration must be used at training, export, and inference time.
The core code lives flat inside vad/:
vad/model.py: theResnet1Dmodel architecture and itsModelConfigdataclassvad/data.py: feature extraction, dataset building & the PyTorch DataLoadervad/train.py: training loop & model export (state dict + TorchScript)vad/inference.py: sliding-window inference & visualization
Supporting files:
tests/: pytest integration tests (training → export → inference)scripts/: Docker build / run helpersMakefile: common developer commands (install,lint,format,test,build, ...)pyproject.toml: project metadata, dependencies & tooling configuration
Please download the LibriSpeech ASR corpus dataset from https://openslr.org/12/,
and extract all files to: /path/to/LibriSpeech/.
The dataset contains approximately 1000 hours of 16kHz read English speech from audiobooks, and is well suited for Voice Activity Detection.
I automatically annotated the test-clean set of the dataset with a
pretrained VAD model.
Please feel free to use the labels/ folder and the pre-trained VAD model (only for inference) from this
link .
Important note: As this is only a toy project, it is designed to split the test-clean sub-dataset intro train / val / test for quick iteration, but can be extended to a full large-scale dataset.
uv run vad-data --data-dir /path/to/LibriSpeech/This saves processed .pt files to /path/to/LibriSpeech/dataset/{train,val,test}/.
Use --max-files N to process only a few files per split for a quick run.
uv run vad-train --data-dir /path/to/LibriSpeech/dataset/ --model-dir /path/to/models/Checkpoints are saved to --model-dir, and the final model is exported (state dict +
TorchScript) to <model-dir>/exported/ unless --no-export is passed.
Useful flags: --epochs/-e, --batch-size/-b, --lr, and the architecture flags
--n-filters / --fc-units. Training device (CUDA, Apple MPS, or CPU) is selected
automatically. TensorBoard logs are written to <model-dir>/logs/:
uv run tensorboard --logdir /path/to/models/logs/uv run vad-inference \
--data-dir /path/to/LibriSpeech/ \
--checkpoint /path/to/models/exported/model_state_dict.pt \
--smoothing --max-files 1Note: if you trained with custom
--n-filters/--fc-units, pass the same values tovad-inferenceso the checkpoint loads into a matching architecture.
Run the test suite with coverage:
make testThis runs pytest with coverage over the vad package, printing a
term report with missing lines and writing an HTML report to htmlcov/.
- CI (
.github/workflows/ci.yml): on every push and pull request, runsrufflint,ruff format --check, and the pytest suite. - CD (
.github/workflows/cd.yml): on push tomaster/main, builds the Docker image and pushes it to Docker Hub (tagged with the commit SHA andlatest).
Contributions are welcome! Please:
- Fork the repository and create a feature branch.
- Install the dev environment with
uv sync. - Make sure
make lintandmake testpass before opening a pull request. - Use clear commit messages and keep changes focused.
- Add MLflow experiment tracking (params, metrics, artifacts)
- Reach full unit-test coverage and add CI coverage reporting
- Add online / streaming real-time inference
- Serve the model via a REST/gRPC API (e.g. FastAPI + ONNX Runtime)
- Add data/version control (DVC) and a model registry
- Compare the model against a simple baseline and train on the full dataset
- Add time-series data augmentation and improve class balancing
- Study the ROC curve & tune the classification threshold
- Explore self-supervised speech encoders (e.g. WavLM, wav2vec 2.0)
- Benchmark modern lightweight VAD models (e.g. Silero VAD, Pyannote 3.x)
This project is licensed under the terms of the GNU GPL v3.