Linear Transformations

A linear transformation $T : \mathbb{R}^n \to \mathbb{R}^m$ satisfies $T(c\mathbf{u} + \mathbf{v}) = cT(\mathbf{u}) + T(\mathbf{v})$ . Every linear transformation can be represented as multiplication by a matrix $A$ , and every matrix defines one.

What matrices do geometrically:

Matrix type	Effect
Diagonal	Scale each axis independently
Orthogonal ( $Q^\top Q = I$ )	Rotate / reflect (preserves lengths)
Symmetric ( $A = A^\top$ )	Scale along eigenvector directions
Projection ( $A^2 = A$ )	Collapse onto a subspace
Shear	Slant one axis relative to another

Composition: applying $B$ then $A$ = multiplying $AB$ . This is why Matrix Multiplication is not commutative — order of transformations matters.

The fundamental theorem: for $A : \mathbb{R}^n \to \mathbb{R}^m$ :

Image (column space) = all possible outputs → dimension is the rank
Kernel (null space) = inputs mapped to $\mathbf{0}$ → see Rank and Null Space
rank + nullity = $n$ (dimension of input space)

Why this is the lens for ML:

A dense layer computes $\mathbf{y} = A\mathbf{x} + \mathbf{b}$ — a linear transformation plus translation (affine)
Without Activation Functions, stacking layers collapses to a single linear transformation
Eigendecomposition reveals the principal axes of a transformation
Singular Value Decomposition (SVD) generalizes this to non-square matrices

Linear Transformations

Linked from