Measure Theory
Conditional Expectation
Stochastic optimization algorithms at Google Brain process 10^9 parameters through adapted sequences - martingales. Conditional expectation E[X|G] as defined measure-theoretically by Kolmogorov in 1933 is the foundation of all martingale theory and the Kalman filter used in every navigation system.
- Stochastic optimization: SGD is a martingale relative to the sigma-algebra of history
- Kalman filter: optimal state estimate = conditional expectation E[X | observations]
- Quantitative finance: derivative pricing via conditional expectations under martingale measure
- Bayesian neural networks: prediction = E[y | x, weights] under the posterior
- Markov chains: transition distributions as conditional probabilities given current state
- Information theory: mutual information I(X;Y) = E[log P(X|Y) / P(X)]
Kolmogorov in 1933 redefined conditional expectation measure-theoretically: E[X|G] is a G-measurable random variable whose integral over any G-set equals the integral of X. This construction underpins all of modern martingale theory; stochastic optimization algorithms at Google Brain process 10⁹ parameters through adapted sequences built on exactly this definition.
**About this lesson:** Measure-theoretic definition of conditional expectation via sub-sigma-algebras. Projection in L² and the connection to martingales.
Definition via Sub-sigma-algebra
Kolmogorov in 1933 redefined conditional expectation measure-theoretically: E[X|G] is a G-measurable random variable whose integral over any G-set equals the integral of X. This construction underpins all of modern martingale theory; stochastic optimization algorithms at Google Brain process 10⁹ parameters through adapted sequences built on exactly this definition.
Properties and Martingales
The properties of conditional expectation form an algebraic structure. The tower property E[E[X|G₁]|G₂] = E[X|G₂] for G₂ ⊆ G₁ means coarser information absorbs finer information. DeepMind applies this in reinforcement learning: value functions at time step t contain 10³ parameters and are built as conditional expectations of future rewards.
Definition via Sub-sigma-algebra
Kolmogorov in 1933 redefined conditional expectation measure-theoretically: E[X|G] is a G-measurable random variable whose integral over any G-set equals the integral of X. This construction underpins all of modern martingale theory; stochastic optimization algorithms at Google Brain process 10⁹ parameters through adapted sequences built on exactly this definition.
What guarantees the existence and uniqueness of conditional expectation E[X|G]?
Conditional expectation exists as the Radon-Nikodym derivative of a bounded measure. Uniqueness holds P-almost surely: two G-measurable functions with equal integrals on all G-sets coincide P-a.s.
Properties and Martingales
The properties of conditional expectation form an algebraic structure. The tower property E[E[X|G₁]|G₂] = E[X|G₂] for G₂ ⊆ G₁ means coarser information absorbs finer information. DeepMind applies this in reinforcement learning: value functions at time step t contain 10³ parameters and are built as conditional expectations of future rewards.
What does the tower property E[E[X|G₁]|G₂] = E[X|G₂] for G₂ ⊆ G₁ state?
With G₂ ⊆ G₁, taking conditional expectation on G₂ from E[X|G₁] (already averaged over the finer G₁) yields E[X|G₂], the average over the coarser G₂. Less information means more averaging.
Key Ideas
- G-measurability: Conditional expectation must be measurable with respect to the sub-sigma-algebra
- Characteristic property: For every G-measurable set G, the integrals of E[X|G] and X over that set coinci
- Connection to Radon-Nikodym: Conditional expectation is the Radon-Nikodym derivative of the restriction of me
- Projection in L²: In the Hilbert space L²(Ω, F, P), conditional expectation is the orthogonal proj
- Tower property: If G₂ ⊆ G₁ (G₂ contains less information), then conditioning on G₂ from a quanti
- Pulling out measurable factors: A G-measurable factor Z can be extracted from the conditional expectation. Intui