WaveNet, introduced by van den Oord and colleagues at DeepMind in 2016, demonstrated that a neural network could generate raw audio waveforms at the individual sample level with quality far surpassing all previous synthesis methods. The model uses a stack of dilated causal convolutions to efficiently capture long-range temporal dependencies while maintaining the autoregressive property: each audio sample is generated conditioned on all previous samples. WaveNet's success fundamentally changed the trajectory of speech synthesis research and inspired a generation of neural audio models.
Dilated Causal Convolutions
Dilated convolution: (F *_d x)_t = ∑_{k=0}^{K-1} f_k · x_{t-d·k}
Dilation pattern: d = 1, 2, 4, 8, ..., 512 (repeated)
Receptive field: grows exponentially with depth
Conditioning: local (mel spectrogram) or global (speaker ID)
The key architectural innovation of WaveNet is the use of dilated causal convolutions. Causal convolutions ensure that the prediction for sample t depends only on previous samples, maintaining the autoregressive property. Dilation exponentially increases the receptive field without increasing the number of parameters or computation per layer: with dilation factors of 1, 2, 4, ..., 512, a stack of 10 layers achieves a receptive field of 1024 samples, and repeating this stack multiple times extends coverage to thousands of samples (hundreds of milliseconds at 16 kHz).
Gated Activations and Conditioning
Each WaveNet layer uses gated activation units inspired by the LSTM gating mechanism: z = tanh(W_f * x) ⊙ σ(W_g * x), where the filter and gate convolutions modulate information flow. Residual and skip connections allow information to bypass layers, facilitating gradient flow and enabling the training of very deep networks (typically 30-40 layers). Conditioning on auxiliary information is implemented by adding bias terms derived from the conditioning signal to the gate and filter convolutions.
The original WaveNet was impractically slow for synthesis, requiring minutes to generate one second of audio because each of the 16,000 samples per second had to be generated sequentially. This motivated a series of faster alternatives: Parallel WaveNet (2018) used probability density distillation to train a parallel feed-forward model from the autoregressive teacher. WaveRNN (2018) achieved real-time synthesis on a single CPU core through a compact recurrent architecture. WaveGlow (2019) used normalizing flows for parallel generation. These efficiency improvements were essential for deploying WaveNet-quality synthesis in production systems.
WaveNet conditions on local features (mel spectrograms, linguistic features) to generate speech corresponding to specific content, and on global features (speaker embeddings) to produce speech in different voices. When conditioned on mel spectrograms predicted by Tacotron 2, WaveNet achieves a mean opinion score of 4.53, approaching the 4.58 of natural speech and far exceeding the previous state of the art. This combination established the dominant two-stage neural TTS paradigm.
Beyond speech synthesis, WaveNet's architecture has influenced generative modeling across audio domains: music generation (NSynth), speech enhancement, audio source separation, and speech coding. The broader impact of WaveNet lies in demonstrating that autoregressive neural networks can model complex, high-dimensional sequential data at the raw signal level, a principle that has since been extended to images, video, and other modalities.