Measure Theory
Radon-Nikodym Theorem
Bayesian updating with millions of observations amounts to computing the Radon-Nikodym derivative dP/dQ. This 1930 construction unifies probability theory with measure theory, and without it there is no way to rigorously define conditional distributions when conditioning on zero-probability events.
- Bayesian inference: posterior measure update computed as the Radon-Nikodym derivative dP/dQ
- Probability theory: probability density is a special case of the Radon-Nikodym derivative
- Mathematical finance: risk-neutral measure change via Girsanov theorem uses Radon-Nikodym
- Statistics: sufficient statistics characterized by factoring the Radon-Nikodym derivative
- Machine learning: KL divergence equals the integral of log(dP/dQ) under measure P
- Stochastic control: Girsanov theorem changes measure for diffusions via Radon-Nikodym
In 1930, Johann Radon proved that if measure μ is absolutely continuous with respect to measure ν on ℝⁿ, then there exists a nonnegative measurable function f such that μ(A) = ∫_A f dν for every measurable set A. This function f, the Radon-Nikodym derivative, underpins Bayesian inference: with 10⁶ observations, updating a posterior measure is computed exactly via dP/dQ.
**About this lesson:** Density of one measure with respect to another. The Radon-Nikodym derivative and its applications in probability theory and statistics.
The Radon-Nikodym Theorem
In 1930, Johann Radon proved that if measure μ is absolutely continuous with respect to measure ν on ℝⁿ, then there exists a nonnegative measurable function f such that μ(A) = ∫_A f dν for every measurable set A. This function f, the Radon-Nikodym derivative, underpins Bayesian inference: with 10⁶ observations, updating a posterior measure is computed exactly via dP/dQ.
Applications: Probability and Statistics
In statistics, the likelihood ratio is the Radon-Nikodym derivative: dP_θ/dP_0(x). Netflix uses Bayesian models with Radon-Nikodym derivatives to update recommendation distributions across billions of interactions daily.
The Radon-Nikodym Theorem
In 1930, Johann Radon proved that if measure μ is absolutely continuous with respect to measure ν on ℝⁿ, then there exists a nonnegative measurable function f such that μ(A) = ∫_A f dν for every measurable set A. This function f, the Radon-Nikodym derivative, underpins Bayesian inference: with 10⁶ observations, updating a posterior measure is computed exactly via dP/dQ.
What does the condition μ ≪ ν (absolute continuity of μ with respect to ν) mean?
Absolute continuity μ ≪ ν means: every set of ν-measure zero also has μ-measure zero. This is the necessary and sufficient condition for the Radon-Nikodym derivative dμ/dν to exist.
Applications: Probability and Statistics
In statistics, the likelihood ratio is the Radon-Nikodym derivative: dP_θ/dP_0(x). Netflix uses Bayesian models with Radon-Nikodym derivatives to update recommendation distributions across billions of interactions daily.
How does the Radon-Nikodym derivative relate to importance sampling?
The change-of-measure formula ∫g dP_θ = ∫g·(dP_θ/dP_0) dP_0 directly defines importance sampling weights as values of the Radon-Nikodym derivative.
Key Ideas
- Absolute continuity: Measure μ is absolutely continuous with respect to ν (μ ≪ ν) when every ν-null s
- Radon-Nikodym derivative: If μ ≪ ν, there exists a unique (ν-a.e.) nonnegative measurable function f = dμ/
- Integral representation: The measure μ of any measurable set A is computed by integrating the density f o
- Change of measure formula: Integration with respect to μ reduces to integration with respect to ν with the
- Likelihood ratio: The likelihood ratio is the Radon-Nikodym derivative of measure P_θ (at paramete
- For absolutely continuous measures: If both measures are absolutely continuous with respect to the Lebesgue measure,