Spectral Theorem

PCA, SVD, Gaussian processes, quantum mechanics-all of these are the spectral theorem in action. Understanding how self-adjoint operators 'diagonalize' into an orthonormal basis reveals the unified mathematics behind these seemingly different methods.

• **PCA and SVD**: the covariance matrix is a self-adjoint operator; PCA is its spectral decomposition; low-rank approximation = truncating to the largest lambda values
• **Gaussian processes**: the covariance kernel k(x,y) generates a self-adjoint integral operator; Mercer's theorem = its spectral decomposition; GP prediction = functional calculus
• **Quantum mechanics**: the Hamiltonian H is a self-adjoint operator; energies = eigenvalues; time evolution U(t) = exp(-iHt/hbar) = functional calculus

Предварительные знания

• Introduction to Spectral Theory

Self-Adjoint Operators

On a Hilbert space H, the **adjoint** T* of T satisfies <Tx, y> = <x, T*y> for all x, y in H. T is **self-adjoint** (Hermitian) if T = T*. Key properties: all eigenvalues are real; eigenvectors for distinct lambda are orthogonal; ||T|| = r(T) = sup{|lambda| : lambda in sigma(T)}.

**Examples**: a symmetric matrix A = A^T is a self-adjoint operator on R^n. The multiplication operator (Tf)(x) = phi(x) f(x) on L^2(mu) with real phi is self-adjoint. The Laplacian d^2/dx^2 with zero boundary conditions on [0,1] is an unbounded self-adjoint operator.

Spectral Decomposition

**Spectral theorem** (compact self-adjoint): if T is a compact self-adjoint operator on a Hilbert space H, there exists an orthonormal basis {phi_n} of eigenvectors T phi_n = lambda_n phi_n, and T = sum_n lambda_n <*, phi_n> phi_n. Every x in H decomposes as x = sum_n <x, phi_n> phi_n.

**Spectral measure** (general case): for any self-adjoint operator T there exists a projection-valued measure E: B(R) -> L(H) such that T = integral lambda dE(lambda). For compact T the measure is discrete: E(Lambda) = sum_{lambda_n in Lambda} P_n, where P_n = <*, phi_n>phi_n is the rank-1 projector onto the n-th eigenvector.

Functional Calculus

**Functional calculus**: for a self-adjoint operator T = integral lambda dE(lambda), define f(T) = integral f(lambda) dE(lambda) for any Borel function f. For compact T: f(T) = sum_n f(lambda_n) <*, phi_n> phi_n. This gives meaning to T^{1/2}, exp(T), log(T), sin(T) for operators.

**Applications**: matrix exponential exp(A) = sum A^n/n! = V diag(exp(lambda_i)) V^{-1} (for diagonalizable A). Hamiltonian exponential in quantum mechanics: U(t) = exp(-iHt/hbar)-the unitary evolution operator. In numerical methods: exp(-tL) with graph Laplacian L-diffusion on a graph.

PCA as functional calculus: the covariance matrix C = X^T X / n is a self-adjoint operator. PCA = spectral decomposition C = V Lambda V^T. Low-rank approximation = keeping k largest eigenvalues. Whitening = multiplying by C^{-1/2} via functional calculus.

Key Ideas

• **Self-adjointness**: T = T* implies all eigenvalues are real, eigenvectors for distinct eigenvalues are orthogonal, and ||T|| = r(T)
• **Spectral decomposition**: T = sum lambda_n P_n (compact case) = integral lambda dE(lambda) (general); every x decomposes in an eigenvector orthonormal basis
• **Functional calculus**: f(T) = integral f(lambda) dE(lambda); defines T^{1/2}, exp(T), log(T); the mathematical foundation of PCA, GP, and quantum mechanics

Related Topics

The spectral theorem connects functional analysis to applications:

• Introduction to Spectral Theory — Spectrum, resolvent, compact operators-the foundation on which the spectral theorem is built
• Sobolev Spaces — The Laplacian -nabla^2 is a self-adjoint operator on L^2; its eigenfunctions (sines, cosines) define the Sobolev space basis

Вопросы для размышления

• PCA finds directions of maximum variance. How does this connect to finding the largest eigenvalues of the covariance matrix viewed as a self-adjoint operator?
• Functional calculus defines exp(-tL) for the graph Laplacian L. What does applying this operator to a vector f in R^n mean geometrically?
• The spectral theorem applies to self-adjoint operators. What happens for non-self-adjoint operators-does an analogue exist?

Связанные уроки

• la-14-diagonalization