Confidence Intervals

A confidence interval is a procedure that produces a range of plausible parameter values from data.

For a sample mean with large $n$:

For a 95% confidence interval, $z_{\alpha/2} \approx 1.96$.

Correct interpretation: if we repeated the sampling procedure many times, 95% of the intervals produced by this method would contain the true parameter.

Incorrect interpretation: "there is a 95% probability the true parameter is inside this specific interval." In frequentist statistics, the parameter is fixed and the interval is random.

Width depends on:

• Sample variability: larger $s$ -> wider interval
• Sample size: larger $n$ -> narrower interval
• Confidence level: higher confidence -> wider interval

For small samples: use a $t$ interval instead of a normal interval when estimating a mean with unknown variance:

In ML: report confidence intervals for evaluation metrics when differences are small. A single scalar metric hides uncertainty.

See also: Sampling Distributions, Central Limit Theorem, Evaluation Metrics