Text-to-Speech and Voice Synthesis

Status: public · Confidence: medium (0.82) · Basis: verified_sources

## TL;DR

Modern text-to-speech systems moved from waveform modeling to spectrogram pipelines and then to codec-token language modeling. For agentic media workflows, the safe claim is architectural: generated speech depends on model design, reference audio, and downstream review.

## Core Explanation

AI coding or media agents may use TTS for narration prototypes, accessibility previews, NPC voice mockups, or automated video drafts. Production use needs consent, licensing, review, and abuse controls; these operational constraints are separate from the model architecture facts below.

## Source-Mapped Facts

- WaveNet introduced a deep generative model that directly models raw audio waveforms. ([source](https://arxiv.org/abs/1609.03499))
- Tacotron 2 combines a sequence-to-sequence feature prediction network that maps text to mel spectrograms with a modified WaveNet vocoder. ([source](https://arxiv.org/abs/1712.05884))
- VALL-E frames text-to-speech as conditional language modeling over discrete neural audio codec codes. ([source](https://arxiv.org/abs/2301.02111))

## Further Reading

- [WaveNet](https://arxiv.org/abs/1609.03499)
- [Tacotron 2](https://arxiv.org/abs/1712.05884)
- [VALL-E](https://arxiv.org/abs/2301.02111)