Determinant and Inverse

The determinant $\det(A)$ of a square matrix is a scalar that captures how the transformation scales volume:

$|\det(A)|$ = factor by which $A$ scales $n$ -dimensional volume
$\det(A) < 0$ means the transformation reverses orientation (mirror)
$\det(A) = 0$ means the transformation collapses a dimension → $A$ is singular (not invertible)

Key properties:

$\det(AB) = \det(A)\det(B)$
$\det(A^\top) = \det(A)$
$\det(A^{-1}) = 1/\det(A)$
For triangular matrices: product of diagonal entries

The inverse $A^{-1}$ exists iff $\det(A) \neq 0$ , and satisfies $AA^{-1} = A^{-1}A = I$ .

Solving $A\mathbf{x} = \mathbf{b}$ : if $A$ is invertible, $\mathbf{x} = A^{-1}\mathbf{b}$ . In practice, use Gaussian Elimination (LU decomposition) — never actually compute $A^{-1}$ .

Where this matters in ML:

Multivariate Gaussians: $p(\mathbf{x}) \propto \exp(-\frac{1}{2}(\mathbf{x}-\mu)^\top \Sigma^{-1}(\mathbf{x}-\mu))$ requires $\Sigma$ to be invertible → must be positive definite
Maximum Likelihood Estimation for Gaussians involves $\det(\Sigma)$ in the normalizing constant
Change of variables in normalizing flows: $p(\mathbf{x}) = p(\mathbf{z})|\det(J)|^{-1}$ where $J$ is the Jacobian Matrix
A near-singular matrix (small determinant) means ill-conditioning → numerically unstable gradients

Determinant and Inverse

Linked from