Bayes' Theorem

Components:

• $P(A|B)$ — posterior: updated belief about $A$ after observing $B$
• $P(B|A)$ — likelihood: probability of evidence $B$ given $A$
• $P(A)$ — prior: belief about $A$ before seeing evidence
• $P(B)$ — marginal likelihood (normalizer): $P(B) = \sum_i P(B|A_i)P(A_i)$

Intuition: Bayes' theorem is a principled way to update beliefs with evidence. Start with a prior → observe data → compute posterior.

Where it appears:

• MAP = argmax of posterior = MLE + prior
• Naive Bayes classifier — assumes feature independence given the class
• Bayesian inference — treat model parameters as random variables with priors
• Bayesian RL — maintain posterior over environment dynamics or reward functions

Concrete example: a medical test is 99% accurate. Disease prevalence is 1%. If you test positive:

A positive test only means 50% chance of disease — the prior matters enormously.

See also: KL Divergence, Key Probability Distributions