Expectation and Variance

1 min read

# topic/math/probability

Expectation (mean): the average value of a random variable.

  • Discrete: E[X]=∑xx⋅p(x)\mathbb{E}[X] = \sum_x x \cdot p(x)
  • Continuous: E[X]=∫x⋅f(x)dx\mathbb{E}[X] = \int x \cdot f(x) dx
  • Linearity: E[aX+bY]=aE[X]+bE[Y]\mathbb{E}[aX + bY] = a\mathbb{E}[X] + b\mathbb{E}[Y] (always, even if dependent)

Variance: measures spread around the mean.

Var(X)=E[(X−μ)2]=E[X2]−(E[X])2\text{Var}(X) = \mathbb{E}[(X - \mu)^2] = \mathbb{E}[X^2] - (\mathbb{E}[X])^2
  • Var(aX)=a2Var(X)\text{Var}(aX) = a^2\text{Var}(X)
  • For independent X,YX, Y: Var(X+Y)=Var(X)+Var(Y)\text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y)

Standard deviation: σ=Var(X)\sigma = \sqrt{\text{Var}(X)} — same units as XX.

Covariance: measures linear co-movement.

Cov(X,Y)=E[(X−μX)(Y−μY)]=E[XY]−E[X]E[Y]\text{Cov}(X, Y) = \mathbb{E}[(X-\mu_X)(Y-\mu_Y)] = \mathbb{E}[XY] - \mathbb{E}[X]\mathbb{E}[Y]
  • Cov(X,X)=Var(X)\text{Cov}(X, X) = \text{Var}(X)
  • Independent ⇒\Rightarrow Cov=0\text{Cov} = 0 (but not the converse!)

Covariance matrix for vector X\mathbf{X}: Σij=Cov(Xi,Xj)\Sigma_{ij} = \text{Cov}(X_i, X_j) — always positive semi-definite. This is what Principal Component Analysis (PCA) eigendecomposes.

Correlation: ρ(X,Y)=Cov(X,Y)σXσY∈[−1,1]\rho(X,Y) = \frac{\text{Cov}(X,Y)}{\sigma_X \sigma_Y} \in [-1, 1] — normalized covariance.

See also: Random Variables, Key Probability Distributions

Linked from