Evaluation Metrics

1 min read

Classification metrics:

Metric	Formula	When to use
Accuracy	$\frac{TP+TN}{TP+TN+FP+FN}$	Balanced classes only
Precision	$\frac{TP}{TP+FP}$	When false positives are costly (spam filter)
Recall (Sensitivity)	$\frac{TP}{TP+FN}$	When false negatives are costly (disease detection)
F1 Score	$\frac{2 \cdot P \cdot R}{P + R}$	Harmonic mean of precision and recall
ROC-AUC	Area under ROC curve	Overall ranking quality, threshold-independent

When accuracy is misleading: imbalanced classes. 99% accuracy on a dataset with 99% negatives means the model predicts all negatives — 0% recall on the positive class.

ROC curve: plots True Positive Rate (recall) vs. False Positive Rate at all thresholds. AUC = 1.0 is perfect, 0.5 is random.

Precision-Recall curve: more informative than ROC when the positive class is rare.

Confusion matrix: the $K \times K$ table of predicted vs. actual classes — read this first before choosing a scalar metric.

Regression metrics: MSE, RMSE, MAE, $R^2$ (explained variance ratio).

Evaluation Metrics

Linked from