Probability Theory

Random Matrices

Random matrices are not just theoretical curiosities: the same Wigner semicircle law governs nuclear energy-level statistics, noise in high-dimensional PCA, and the spectral gap of wireless MIMO channels. One mathematical object unifies all of them.

**Denoising covariance matrices:** a 500x500 S&P stock-price covariance matrix is cleaned by comparing its spectrum to the theoretical Marchenko-Pastur distribution, used in production at quantitative hedge funds
**PCA diagnostics:** in the Cancer Genome Atlas (d=20000 genes, n=200 patients) the leading principal components are biased below the BBP threshold, requiring shrinkage or regularization before any downstream analysis
**Tail risk:** reinsurance models for portfolios of 10000 policies use the Tracy-Widom law to estimate extreme-event probabilities at the spectral edge, where classical CLT approximations break down
**MIMO wireless:** the capacity of an n-antenna channel is determined by the eigenvalue distribution of a random channel matrix, a direct application of the Marchenko-Pastur and GOE/GUE theory

Предварительные знания

Concentration of measure: Hanson-Wright inequality for quadratic forms of Gaussian vectors
Spectral theorem: eigenvalues and eigenvectors of symmetric matrices
Convergence in distribution and the method of moments (Catalan numbers)
Basics of the multivariate normal and sample covariance

The Wigner Ensemble and Semicircle Law

Eugene Wigner proposed in 1955 to model heavy-nuclei Hamiltonians by random symmetric matrices and discovered the universal semicircle law for eigenvalues. The same principle drives portfolio analysis at JPMorgan: a $500 \times 500$ S&P stock-price covariance matrix is denoised by comparing its spectrum to the theoretical Marchenko-Pastur distribution.

A Wigner matrix $W_{1000}$ (normalized by $1/\sqrt n$). Where are eigenvalues concentrated?

By the semicircle law with $1/\sqrt n$ normalization eigenvalues live in $[-2, 2]$, with $\lambda_{\max} \to 2$.

Tracy-Widom Law: Spectral Edge

Craig Tracy and Harold Widom found in 1994 the exact limiting distribution of the largest eigenvalue of a Wigner matrix. Surprisingly, the same law appears in problems as different as the longest increasing subsequence of a random permutation and the growth of random interfaces. Tail-risk models for insurance portfolios with 10000 policies, used by reinsurers, rely on the Tracy-Widom law.

Fluctuations of $\lambda_{\max}(W_n)$ around the edge $2$ have order:

Edge fluctuations scale as $n^{-2/3}$, which is the Tracy-Widom rate, slower than the CLT rate $n^{-1/2}$.

PCA Phase Transition: the Spike Model

Jinho Baik, Gerard Ben Arous, and Sandrine Peche found a sharp PCA phase transition in 2005: at $d/n \to \gamma$ a weak signal below the threshold $\sigma\sqrt\gamma$ is fully drowned by noise. This explains why genetic analyses with $d = 20000$ genes and $n = 200$ patients in The Cancer Genome Atlas yield biased principal components and require regularization.

PCA with $d = 500$ features and $n = 2000$ samples. Minimum SNR for signal detection?

BBP threshold: $\mathrm{SNR}_* = \sqrt{d/n} = \sqrt{0.25} = 0.5$. Below it, the top principal component is uncorrelated with the signal.

Where random matrix theory lives

Random matrices sit at the intersection of probability, linear algebra, and mathematical physics. Spectral theory connects them to PCA and machine learning, while the GOE/GUE link to quantum mechanics and free probability.

Concentration of measure — The Hanson-Wright inequality bounds quadratic forms of random matrices and underpins edge estimates
Optimal transport — The Wasserstein distance between empirical spectral measures is the natural metric for comparing random matrix ensembles
Free probability — Voiculescu's free convolution describes the sum and product of independent large random matrices beyond classical probability
Diagonalization — The GOE ensemble is a probability measure on symmetric matrices with an explicit eigenvalue joint density

Итоги

**Semicircle law:** eigenvalues of a normalized Wigner matrix converge to the semicircle on [-2, 2] universally, regardless of entry distribution (zero mean, unit variance suffice)
**Tracy-Widom law:** fluctuations of the largest eigenvalue scale as n^{-2/3} (not n^{-1/2} as in the CLT) and are described by the Fredholm determinant of the Airy kernel
**BBP phase transition:** PCA detects a spike signal only when SNR > sqrt(d/n); below the threshold the leading eigenvector is uncorrelated with the true signal direction
**Marchenko-Pastur:** the limiting spectral distribution of the sample covariance at aspect ratio d/n → γ; the standard tool for cleaning financial covariance matrices
**In practice:** compare the empirical spectrum from numpy/scipy to the Wigner or Marchenko-Pastur density as a standard noise-vs-signal diagnostic

Связанные уроки

la-15-svd

Предварительные знания

Concentration of measure: Hanson-Wright inequality for quadratic forms of Gaussian vectors

Spectral theorem: eigenvalues and eigenvectors of symmetric matrices

Convergence in distribution and the method of moments (Catalan numbers)

Basics of the multivariate normal and sample covariance

The Wigner Ensemble and Semicircle Law

A Wigner matrix $W_{1000}$ (normalized by $1/\sqrt n$). Where are eigenvalues concentrated?

By the semicircle law with $1/\sqrt n$ normalization eigenvalues live in $[-2, 2]$, with $\lambda_{\max} \to 2$.

Tracy-Widom Law: Spectral Edge

Fluctuations of $\lambda_{\max}(W_n)$ around the edge $2$ have order:

Edge fluctuations scale as $n^{-2/3}$, which is the Tracy-Widom rate, slower than the CLT rate $n^{-1/2}$.

PCA Phase Transition: the Spike Model

PCA with $d = 500$ features and $n = 2000$ samples. Minimum SNR for signal detection?

BBP threshold: $\mathrm{SNR}_* = \sqrt{d/n} = \sqrt{0.25} = 0.5$. Below it, the top principal component is uncorrelated with the signal.

Итоги

**Semicircle law:** eigenvalues of a normalized Wigner matrix converge to the semicircle on [-2, 2] universally, regardless of entry distribution (zero mean, unit variance suffice)

**Tracy-Widom law:** fluctuations of the largest eigenvalue scale as n^{-2/3} (not n^{-1/2} as in the CLT) and are described by the Fredholm determinant of the Airy kernel

**BBP phase transition:** PCA detects a spike signal only when SNR > sqrt(d/n); below the threshold the leading eigenvector is uncorrelated with the true signal direction

**Marchenko-Pastur:** the limiting spectral distribution of the sample covariance at aspect ratio d/n → γ; the standard tool for cleaning financial covariance matrices

**In practice:** compare the empirical spectrum from numpy/scipy to the Wigner or Marchenko-Pastur density as a standard noise-vs-signal diagnostic