Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Audio generation using diffusion models, in PyTorch.
| Date | Stars |
|---|---|
| 2026-07-31 | 2096 |
| 2026-08-06 | 2096 |
| 2026-08-28 | 2096 |
| 2026-09-05 | 2096 |
| 2026-09-06 | 2096 |
| 2026-09-11 | 2095 |
| 2026-09-17 | 2095 |
| 2026-09-20 | 2095 |
Today
— stars today
This week
— stars this week
This month
-1 stars this month
Momentum
0.0
growth rate 0.00%/day
<img src="./LOGO.png"></img>
A fully featured audio diffusion library, for PyTorch. Includes models for unconditional audio generation, text-conditional audio generation, diffusion autoencoding, upsampling, and vocoding. The provided models are waveform-based, however, the U-Net (built using [`a-unet`](https://github.com/archinetai/a-unet)), `DiffusionModel`, diffusion method, and diffusion samplers are both generic to any dimension and highly customizable to work on other formats. **Notes: (1) no pre-trained models are provided here, (2) the configs shown are indicative and untested, see [Moûsai](https://arxiv.org/abs/2301.11757) for the configs used in the paper.**
## Install
```bash
pip install audio-diffusion-pytorch
```
[](https://pypi.org/project/audio-diffusion-pytorch/)
[](https://pepy.tech/project/audio-diffusion-pytorch)
## Usage
### Unconditional Generator
```py
from audio_diffusion_pytorch import DiffusionModel, UNetV0, VDiffusion, VSampler
model = DiffusionModel(
net_t=UNetV0, # The model type used for diffusion (U-Net V0 in this case)
in_channels=2, # U-Net: number of input/output (audio) channels
channels=[8, 32, 64, 128, 256, 512, 512, 1024, 1024], # U-Net: channels at each layer
factors=[1, 4, 4, 4, 2, 2, 2, 2, 2], # U-Net: downsampling and upsampling factors at each layer
items=[1, 2, 2, 2, 2, 2, 2, 4, 4], # U-Net: number of repeating items at each layer
attentions=[0, 0, 0, 0, 0, 1, 1, 1, 1], # U-Net: attention enabled/disabled at each layer
attention_heads=8, # U-Net: number of attention heads per attention item
attention_features=64, # U-Net: number of attention features per attention item
diffusion_t=VDiffusion, # The diffusion method used
sampler_t=VSampler, # The diffusion sampler used
)
# Train model with audio waveforms
audio = torch.randn(1, 2, 2**18) # [batch_size, in_channels, length]
loss = model(audio)
loss.backward()
# Turn noise into new audio sample with diffusion
noise = torch.randn(1, 2, 2**18) # [batch_size, in_channels, length]
sample = model.sample(noise, num_steps=10) # Suggested num_steps 10-100
```
### Text-Conditional Generator
A text-to-audio diffusion model that conditions the generation with `t5-base` text embeddings, requires `pip install transformers`.
```py
from audio_diffusion_pytorch import DiffusionModel, UNetV0, VDiffusion, VSampler
model = DiffusionModel(
# ... same as unconditional model
use_text_conditioning=True, # U-Net: enables text conditioning (default T5-base)
use_embedding_cfg=True, # U-Net: enables classifier free guidance
embedding_max_length=64, # U-Net: text embedding maximum length (default for T5-base)
embedding_features=768, # U-Net: text mbedding features (default for T5-base)
cross_attentions=[0, 0, 0, 1, 1, 1, 1, 1, 1], # U-Net: cross-attention enabled/disabled at each layer
)
# Train model with audio waveforms
audio_wave = torch.randn(1, 2, 2**18) # [batch, in_channels, length]
loss = model(
audio_wave,
text=['The audio description'], # Text conditioning, one element per batch
embedding_mask_proba=0.1 # Probability of masking text with learned embedding (Classifier-Free Guidance Mask)
)
loss.backward()
# Turn noise into new audio sample with diffusion
noise = torch.randn(1, 2, 2**18)
sample = model.sample(
noise,
text=['The audio description'],
embedding_scale=5.0, # Higher for more text importance, suggested range: 1-15 (Classifier-Free Guidance Scale)
num_steps=2 # Higher for better quality, suggested num_steps: 10-100
)
```
### Diffusion Upsampler
Upsample audio from a lower sample rate to higher sample rate using diffusion, e.g. 3kHz to 48kHz.
```py
from auExcerpt of 11,246 characters
Read on GitHub
184
1
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:a4baaef93b652a87, topic:deep-learning