# topic/ai-ml/classical

Bias-Variance Tradeoff

The expected test error of a model decomposes into three terms:

Classical Machine Learning

Classical machine learning studies how models generalize from data without relying on large neural networks. The useful mental model is: choose a hypothesis class, define a loss…

Cross-Validation

Cross-validation provides honest estimates of model performance on unseen data.

Evaluation Metrics

Classification metrics:

K-Nearest Neighbors

KNN is a non-parametric algorithm: it stores all training data and classifies new points by majority vote among the nearest neighbors.

Linear Regression

Linear regression models the relationship (or with bias absorbed).

Logistic Regression

Logistic regression is a linear classifier that models the probability of class membership:

Principal Component Analysis (PCA)

PCA finds the directions of maximum variance in data and projects onto them for dimensionality reduction.

Random Forest

Random forest is an ensemble of decision trees that reduces variance through bagging and feature randomization.

Regularization

Regularization adds a penalty to the loss function to prevent overfitting by constraining model complexity.