## TL;DR
Training data quality is now recognized as the dominant factor in AI model performance. Data curation — filtering, deduplication, quality scoring, and mixture design — has become a first-class engineering discipline.

## Core Explanation
The curation pipeline: (1) collection from web (Common Crawl), books, code, scientific papers; (2) language ID and filtering; (3) quality scoring (perplexity, ML classifier); (4) deduplication (exact, fuzzy via MinHash, semantic); (5) personally identifiable information (PII) removal; (6) toxicity/safety filtering.

## Detailed Analysis
Data mixture design determines the relative proportions of sources — code, math, science, books, web — in the final training corpus. Chinchilla scaling laws showed optimal data-to-model-parameter ratios. FineWeb-Edu filters for educational quality using a BERT classifier trained on human annotations.

## Further Reading
- HuggingFace: FineWeb Blog
- Dolma: Open Training Dataset (AI2)
- Common Crawl Foundation