Differential Equations
Classification of PDEs
Choosing the wrong numerical method for a given PDE type is like using a hammer to drive a screw. It may work in principle but is catastrophically slow or unstable. PDE classification is the map that points to the right tool.
- **Game physics simulation:** wave equation (sound, explosions) - upwind; heat equation (fire) - implicit scheme; statics (deformations) - elliptic solver. All three types in one engine.
- **Numerical weather prediction:** Navier-Stokes equations mix hyperbolic (advection) and elliptic (pressure) parts. Operator splitting separates them and solves each with the appropriate method.
- **Inverse problems in ML:** recovering a heat source from observations is an ill-posed inverse problem. Tikhonov regularization = adding a penalty term for stabilization.
Предварительные знания
Classification of Second-Order PDEs
Linear second-order PDE: **A·u_xx + 2B·u_xy + C·u_yy + lower = G**. The type is determined by the discriminant **Δ = B² - AC**: Δ < 0 - **elliptic** (Laplace, Poisson); Δ = 0 - **parabolic** (heat equation); Δ > 0 - **hyperbolic** (wave equation).
| Type | Δ = B²-AC | Prototype | Physics |
|---|---|---|---|
| Elliptic | < 0 | ∇²u = 0 (Laplace) | Steady-state fields |
| Parabolic | = 0 | ∂u/∂t = α∇²u | Diffusion, heat |
| Hyperbolic | > 0 | ∂²u/∂t² = c²∇²u | Waves, propagation |
**Variable type:** the Tricomi equation (u_xx + y·u_yy = 0) is elliptic for y > 0 and hyperbolic for y < 0. It arises in transonic aerodynamics at the transition through the speed of sound. Simulation requires different methods in different regions.
Jacques Hadamard and well-posed problems (1902-1923)
In 1902 Hadamard, lecturing at Yale, stated three axioms for a 'sensible' physical problem: existence, uniqueness, and continuous dependence on data. In 1923 in 'Lectures on Cauchy's Problem' he proved the Cauchy problem for Laplace's equation violates the third: a perturbation of order e^(-n) in the data produces a solution of order e^n. This killed many 'naive' inverse-problem ideas and gave birth to regularization of ill-posed problems (Tikhonov, 1963).
Modern ML regularization (L2, dropout, weight decay) descends from Tikhonov regularization, which grew out of Hadamard's analysis.
Classify u_xx + 4u_xy + 4u_yy = 0.
Characteristic Curves
**Characteristics** are special curves in the (x,y) plane along which information propagates. A hyperbolic equation has two families of characteristics. A parabolic equation has one. An elliptic equation has none - information arrives 'from all directions'.
For the wave equation u_tt = c²u_xx, characteristics are: **x - ct = const** and **x + ct = const** - these are the 'influence cones'. The point (x,t) receives information only from the interval [x-ct, x+ct] of initial data. This is the finite speed of propagation principle.
**Heat equation paradox:** ∂u/∂t = α∂²u/∂x² predicts instantaneous heat propagation - a small disturbance at one point immediately (at t=0+) affects the entire domain. Physically unrealistic but mathematically convenient. Hyperbolic heat equations fix this.
Wave equation Cauchy problem with initial data on [-2, 2]. At t=1, c=1: which points influence x=3?
Well-Posedness (Hadamard)
A PDE problem is **well-posed in the sense of Hadamard** if: 1. a solution exists 2. it is unique 3. it depends continuously on the data (stability). Violating any condition makes the problem ill-posed - physically meaningless for modeling.
| PDE type | Well-posed formulation | Ill-posed |
|---|---|---|
| Elliptic | Boundary conditions everywhere | Cauchy problem (unstable) |
| Parabolic | IC at t=0, BCs in x, march forward in t | Backward problem (t→-∞) |
| Hyperbolic | IC at t=0 (u and ∂u/∂t), march forward | BCs on two characteristic surfaces |
Cauchy problem for Laplace's equation (initial data on a line, not a closed boundary). Is it well-posed?
Choosing Numerical Methods by PDE Type
The PDE type determines which numerical method is efficient. **Elliptic** - no time direction, global solution - use iterative or direct solvers (Gauss-Seidel, multigrid, FFT). **Parabolic** - march in time, explicit or implicit schemes. **Hyperbolic** - march with characteristics, upwind schemes.
| Type | Method | Key requirement |
|---|---|---|
| Elliptic | Iterative (CG, MG), direct (LU) | No explicit marching direction |
| Parabolic | Explicit (FTCS), implicit (Crank-Nicolson) | Stability: r≤0.5 for explicit |
| Hyperbolic | Upwind, Lax-Wendroff, WENO | CFL condition: c·Δt/Δx ≤ 1 |
| Mixed | Different methods in different regions | Transonic aerodynamics |
Steady-state electrostatics (∂u/∂t = 0) in 3D. Which method is preferred?
Key Ideas
- **Discriminant Δ = B² - AC:** < 0 elliptic, = 0 parabolic, > 0 hyperbolic.
- **Characteristics:** hyperbolic - 2 families (influence cone); parabolic - 1; elliptic - none.
- **Hadamard well-posedness:** existence + uniqueness + stability. Inverse problems are often ill-posed.
- **Numerical method selection:** elliptic→iterative solvers; parabolic→time marching; hyperbolic→upwind + CFL.
Related Topics
PDE classification unifies all PDE lessons into one system:
- The Heat Equation — Parabolic type: one family of characteristics, time marching
- The Wave Equation — Hyperbolic type: two families of characteristics, finite speed of propagation
- Laplace's Equation — Elliptic type: no characteristics, global iterative solvers
Вопросы для размышления
- The Tricomi equation changes type at y=0. How can its numerical solution be approached in a domain containing both regions?
- The backward heat equation is ill-posed: small noise → huge errors. How do ML methods (neural networks) handle such problems?
- The CFL condition c·Δt/Δx ≤ 1 for hyperbolic equations: explain via characteristics why violating it causes instability?