Description
WaveGlow is a flow-based generative network for speech synthesis, developed by NVIDIA researchers Ryan Prenger, Rafael Valle and Bryan Catanzaro. It combines ideas from Glow and WaveNet to generate high-quality speech audio from mel-spectrograms, without the autoregressive decoding step that earlier vocoders relied on.
Read More
The network is implemented as a single model trained with a single cost function: maximizing the likelihood of the training data. According to the accompanying paper, this makes training simpler and more stable than multi-stage approaches, and the PyTorch implementation reportedly generates audio at a rate of 1200 kHz on an NVIDIA V100 GPU, with mean opinion scores comparable to the best publicly available WaveNet implementation at the time.
The repository provides everything needed to reproduce or build on this work. A pretrained WaveGlow checkpoint and matching mel-spectrogram files are available for running inference immediately, using the inference.py script to synthesize audio from a list of mel-spectrogram files. For training a new model, the repo includes scripts built around the LJ Speech dataset format: file lists are split into training and test sets, mel-spectrograms are generated with mel2samp.py, and training runs through train.py, with distributed.py available for multi-GPU setups. Mixed-precision (fp16) training can be turned on through the model's configuration file. A separate conversion script, convert_model.py, updates older checkpoints saved before the current architecture to work with it, since the current version fuses the residual and skip connections into the network.
Setup starts with cloning the repository and initializing its git submodule, installing the Python requirements, and installing NVIDIA's Apex library, which the training and inference code depend on. Because the project is a research release rather than a packaged application, using it means working directly with these command-line scripts rather than a graphical interface.
WaveGlow is aimed at speech-synthesis researchers and engineers who want a reference vocoder implementation to run inference, retrain on new voice data, or compare against other generative approaches such as WaveNet. Audio samples generated by the model are available on the project's linked website, and the accompanying paper documents the mean opinion score evaluation used to judge audio quality.
- CategoryText to Speech
- Pricing ModelOpen source
- DeploymentCloud, Self-hosted
- PlatformsNVIDIA Blackwell, DRIVE, Jetson, RTX
WaveGlow's Core Features
Generates speech audio directly from mel-spectrograms without an autoregressive decoding step
Trained with a single maximum-likelihood cost function
Ships a pretrained WaveGlow checkpoint and matching mel-spectrogram data for inference
Includes scripts to train a new model on the LJ Speech dataset format
Supports multi-GPU distributed training via NCCL
Supports mixed-precision (fp16) training and inference
Includes a conversion script for updating older checkpoints to the current architecture
Reports audio generation at 1200 kHz on an NVIDIA V100 GPU per the accompanying paper
Getting Started with WaveGlow
Clone the repository: Clone NVIDIA/waveglow and initialize its submodule.
Install dependencies: Install the Python requirements and the Apex library.
Get data: Download the pretrained WaveGlow model and sample mel-spectrograms, or the LJ Speech dataset to train your own.
Run inference: Generate audio from mel-spectrograms with inference.py.
Train (optional): Run train.py, or distributed.py for multi-GPU, on your own dataset.
Evaluate output: Compare generated audio quality and adjust settings such as fp16 or the sigma parameter.
WaveGlow's Use Cases
- Text-to-speech vocoding
- Vocoder research
- Custom voice training
