Central Limit Theorem

The Central Limit Theorem says that sums and averages of many independent random variables become approximately normal under broad conditions.

For iid variables with mean $\mu$ and variance $\sigma^2$:

as $n \to \infty$.

Equivalently:

for large $n$.

Why it matters: it lets us approximate uncertainty for sample means even when the original data is not normally distributed.

This is the foundation for:

• Standard errors
• Confidence intervals
• Many hypothesis tests
• Normal approximations to evaluation metrics

Limitations:

• Heavy-tailed data may converge slowly
• Strong dependence can break the usual result
• Small samples may not be close to normal
• The CLT applies to averages/sums, not arbitrary statistics automatically

In ML: when evaluating average loss over a test set, the CLT often justifies approximate confidence intervals for the mean test loss.

See also: Sampling Distributions, Confidence Intervals, Key Probability Distributions