Stochastic Processes

Stochastic Partial Differential Equations

Цели урока

  • Build the mild solution of the stochastic heat SPDE via the Walsh integral
  • Explain the reduced regularity of the solution (1/4 in t, 1/2 in x)
  • Derive the Zakai equation for the nonlinear filtering problem
  • Connect SPDEs to neural solvers (FNO, PINN)

Предварительные знания

  • Stochastic control
  • Ito integral
  • Heat equation
  • Stochastic control
  • Brownian motion and Ito integral

Tesla FSD fuses data from 8 cameras, radar, and ultrasound every 10 ms. The Kalman filter is the Zakai equation for the linear case. All of sensor fusion is an SPDE solved in real time.

  • Tesla FSD: Zakai equation as the basis for nonlinear sensor fusion
  • NVIDIA Modulus: neural solvers for turbulent SPDEs
  • FNO (Fourier Neural Operator): operator learning for parametric SPDEs
  • Finance: Heath-Jarrow-Morton interest rate model as an SPDE

Walsh, Zakai, and the birth of SPDE theory

John Walsh defined the stochastic integral with respect to a space-time Brownian sheet in 1984, opening the door to a rigorous SPDE theory. Moshe Zakai derived his filtering equation in 1969, generalizing the Kalman-Bucy filter. Nualart, Da Prato, and others systematically studied solution regularity in the 1990s-2000s. Modern applications: Anima Anandkumar and Zongyi Li developed FNO in 2020 - neural networks for solving SPDEs on large grids in seconds.

SPDEs and the Stochastic Heat Equation

NVIDIA Modulus, 2024. A neural simulator for turbulence accelerates CFD computations 100x. Under the hood: SPDEs for space-time random fields. The stochastic heat equation describes diffusion driven by a random source and has become the standard model for spatial noise.

In dimension d >= 2 space-time white noise is too rough: u has no pointwise L^2 regularity. For d >= 2, noise must be smoothed (colored noise) or generalized solutions must be used.

Why does the mild solution of the stochastic heat SPDE have 1/4 regularity in time rather than 1/2 like Brownian motion?

White noise dot{W}(t,x) is a generalized function in both variables. The heat kernel smooths in x, giving 1/2-Holder there. In t: without spatial integration it would be 1/2, but double roughness gives 1/4.

The Zakai Equation and Nonlinear Filtering

GPS gives position with 2-3 meter error. A camera gives a different error. Lidar gives a third. How to combine all sensors into a single state estimate? This is nonlinear filtering. The Zakai equation is its rigorous mathematical solution.

Kalman filter as a special case

Linear Zakai equation

With f(x) = Ax (linear dynamics) and h(x) = Cx (linear observation), initial distribution X_0 ~ N(m_0, P_0), the filter pi_t = N(m_t, P_t) remains Gaussian for all t. The Zakai equation reduces to the Kalman-Bucy equations: dm_t = A m_t dt + P_t C^T (dY_t - C m_t dt), dP_t/dt = AP_t + P_t A^T + Q - P_t C^T R^{-1} C P_t. The Riccati equation again.

The particle filter (sequential Monte Carlo) is a numerical solution to the Zakai equation through a weighted particle set. Used in Tesla FSD for sensor fusion: GPS + IMU + cameras + lidar.

How does the Zakai equation differ from the Fokker-Planck equation?

Fokker-Planck: dt p(t,x) = L* p(t,x) - deterministic PDE for unconditional density. Zakai: d sigma_t(phi) = sigma_t(L phi) dt + sigma_t(h phi) dY_t - SPDE with dY_t because it incorporates observations.

Solution Regularity and Neural PDE Solvers

Neural PDE solvers (Physics-Informed Neural Networks, FNO) handle SPDEs through random realizations. The Fourier Neural Operator by Anima Anandkumar solves parametric SPDEs in milliseconds instead of hours with FEM. The key question: what regularity needs to be approximated?

FNO (Fourier Neural Operator) is trained on pairs (initial condition, noise realization) -> SPDE solution. Generalization to new noise realizations is automatic. This is operator learning: FNO approximates the map from function spaces to function spaces, not a single fixed solution.

For space-time SPDEs, spectral methods are effective: expanding in eigenfunctions of the operator L gives explicit formulas for the variance of each mode. Efficient for linear SPDEs.

FNO learns to approximate an SPDE solution. Does it approximate a function or an operator?

FNO approximates the operator G: (u_0, W) -> u(T,*). Generalization is achieved by training on many (noise realization, solution) pairs. For new realizations FNO gives solutions in milliseconds.

Connections to other topics

SPDEs connect functional analysis, stochastic calculus, and numerical methods

  • Nonlinear filtering — Related topic
  • Neural PDE solvers — Related topic
  • Rough paths theory — Related topic
  • Financial models — Related topic

Итоги

  • Stochastic heat SPDE: mild solution via Walsh integral, regularity 1/4 in t and 1/2 in x
  • Variance grows like sqrt(t) - slower than an SDE because thermal diffusion dampens noise
  • Zakai equation: SPDE for the unnormalized conditional distribution given observations
  • Linear Zakai = Kalman-Bucy; nonlinear = particle filter

Вопросы для размышления

  • Why does space-time white noise in dimension d >= 2 prevent pointwise solution values for the SPDE?
  • How does the Zakai equation extend when the observations are themselves nonlinear in noise?
  • What fundamentally distinguishes FNO from PINN when solving parametric SPDEs?

Связанные уроки

  • sp-25-stochastic-control — The Zakai equation is an SPDE arising from nonlinear filtering
  • sp-20 — Walsh integral generalizes the Ito integral to space-time
  • sp-27 — Rough paths theory extends SPDEs to irregular noise
  • sp-24-levy-processes
Stochastic Partial Differential Equations

0

1

Sign In