Automata and Cognition
Causal Reasoning and Counterfactuals
Цели урока
- Understand the three levels of the Ladder of Causation (L1: Seeing, L2: Doing, L3: Imagining)
- Apply do-calculus: the difference between P(Y|X) and P(Y|do(X)), graphical arrow removal
- Use Backdoor and Frontdoor criteria to estimate causal effects from observational data
- Perform counterfactual inference using the three-step procedure: Abduction - Action - Prediction
- Explain why LLMs operate at L1 and what this means for their understanding of causality
Предварительные знания
- POMDP and probabilistic models (lesson 05)
- Predictive Processing (lesson 12)
- Basic probability theory: conditional probability, marginalization
Medicine spent 200 years developing RCTs precisely because P(Y|X) does not equal P(Y|do(X)). Pearl in 2000 gave a formal language for causes - that is what separates "correlation" from "this treatment works".
- **Medical RCTs** - the gold standard precisely because randomization implements do(Treatment), eliminating confounders
- **Libratus (2017)** - the first AI to beat top poker professionals uses Counterfactual Regret Minimization (L3)
- **AlphaStar (2019)** - DeepMind uses counterfactual planning in StarCraft II
- **LLMs and causality** - GPT-4 systematically confuses correlation with causation on Simpson's Paradox tasks
- **Economic policy** - backdoor criterion allows estimating minimum wage effects without randomization
The Causal Revolution of Judea Pearl
Judea Pearl started with Bayesian networks (1988) - a compact way to represent joint distributions. But in the 1990s he realized Bayesian networks cannot answer questions like "what will happen if we intervene". The problem: they encode correlations, not causes. From 1993 to 2000 he developed do-calculus - a complete axiomatization for causal inference. In 2000 the book "Causality" was published, in 2011 he received the Turing Award. In 2018 "The Book of Why" brought these ideas to a wider audience.
Pearl's Ladder of Causation
**Year 2000. Judea Pearl demonstrates: all of statistics up to that point operated only at L1 - observation.** Randomized controlled trials (RCT) became the gold standard in medicine precisely because they operate at L2. Without understanding this ladder, any ML model confuses "people carrying umbrellas are more often wet" with "umbrellas make people wet".
**Ladder of Causation (Pearl, 2000):** three levels of causal understanding. L1: Seeing - observing correlations, P(Y|X). L2: Doing - intervention, P(Y|do(X)). L3: Imagining - counterfactual, P(Y_x|X', Y'). Each next level requires qualitatively different information - data alone is never enough.
| Level | Question | Requirements | Example |
|---|---|---|---|
| L1: Association | P(Y|X) - what if I SEE X? | Data | Umbrella carriers are more often wet |
| L2: Intervention | P(Y|do(X)) - what if I DO X? | Causal model | If given an umbrella - will they get wetter? |
| L3: Counterfactual | P(Y_x|X',Y') - what WOULD HAVE happened if X? | Structural model + specific case | Would the patient have survived if treated? |
**Classic confounder example:** studies show people who eat breakfast are on average slimmer. Does breakfast cause weight loss? No - the confounder "healthy lifestyle" affects both the habit of eating breakfast and weight. Observation (L1) gives no answer. Intervention (L2) is needed - randomize groups by force.
Big data and strong correlations replace causal analysis
No amount of observations (L1) answers an interventional question (L2) without a causal model
Simpson's Paradox: a correlation can fully reverse when stratifying by a confounder. The famous kidney stones example - treatment A looks better overall, but treatment B is better for both small and large stones. Aggregation hides true causality.
Research shows people who eat breakfast are on average slimmer. If people are forced to eat breakfast, will they lose weight?
do-calculus and Causal Graphs
**P(Y|X) does not equal P(Y|do(X)) - this is the key inequality of all causal statistics.** The observation "patients taking medication recover more often" may be a selection artifact: healthier patients self-select into taking medication. do(X) is a forced intervention that breaks this link.
**Pearl's do-operator:** do(X=x) means "forcibly set X=x". Graphically - remove all incoming arrows into X. After this, X no longer depends on confounders. The difference P(Y|do(X)) - P(Y) is the true causal effect of X on Y.
Backdoor and Frontdoor Criteria
Pearl proved: if a set of variables Z can be identified that blocks all "backdoor" paths (through confounder X <- ... -> Y), then P(Y|do(X)) can be computed from observational data without a real experiment.
| Criterion | When to use | Formula |
|---|---|---|
| Backdoor | Z blocks all backdoor paths X<-..->Y, Z is not a descendant of X | P(Y|do(X)) = sum_z P(Y|X,Z) P(Z) |
| Frontdoor | Backdoor unavailable, mediator M exists: X->M->Y | P(Y|do(X)) = sum_m P(M|X) sum_x' P(Y|M,x') P(x') |
| RCT | Randomization eliminates all confounders experimentally | P(Y|do(X)) = P(Y|X) directly |
Backdoor: treatment effect with confounder Age
Graph: Age -> Treatment, Age -> Recovery, Treatment -> Recovery. Backdoor path: Treatment <- Age -> Recovery. Z = {Age} blocks it. Formula: P(Recovery | do(Treatment)) = sum_age P(Recovery | Treatment, Age) P(Age). Computable from data without randomization.
Frontdoor criterion is rarely needed - backdoor is usually sufficient
Frontdoor is critical when the confounder is unobservable (hidden U)
The classic example - smoking (X) -> tar in lungs (M) -> cancer (Y), with a possible hidden genetic confounder U. Backdoor fails (U unobservable). Frontdoor through mediator M allows estimating P(Cancer|do(Smoking)) from purely observational data.
What is the key difference between P(Y|X) and P(Y|do(X))?
Counterfactual Reasoning
**L3 - the most powerful level: "The patient did not take the medication and died. Would they have survived if they had?"** This is not an intervention into the future (L2) - it is a retrospective question about a specific case. Pearl proved: answering requires a Structural Causal Model (SCM) plus three steps.
**Three steps of counterfactual inference:** 1. Abduction - infer exogenous variables U from observed facts. 2. Action - apply the hypothetical intervention do(X=x'). 3. Prediction - run the model forward with the same U and new X. Result: what would have happened in that same world, but with a different choice.
**Counterfactuals are not identifiable from data in general.** Even knowing P(Y|do(X)) for all X, computing P(Y_x=y|X=x', Y=y') is impossible without a structural model. This is a fundamental limitation - a specific case requires individual abduction of U, not just population statistics.
A counterfactual is just an interventional query P(Y|do(X)) with a different X
A counterfactual is a question about a specific world with specific U, not about a distribution under intervention
P(Y|do(X)) is an intervention into the future for a population. P(Y_x|X=x', Y=y') is a question about a specific past case. These are different mathematical objects. A patient who died under one treatment might not have survived under another even if P(survive|do(treatment=1)) is high.
Why does a counterfactual require specific facts of the case, not just a causal model?
Planning via Counterfactual Imagination
**Planning is simulating counterfactuals: "what will happen if I do X?"** That is exactly L2-L3 reasoning. LLMs operate at L1 - they reproduce textual patterns of causal explanations without having a causal model of the world. That is why they can imitate planning but cannot do it reliably.
| System | Level | Capability | Limitation |
|---|---|---|---|
| Classical ML (SVM, RF) | L1 | Correlations in data | No understanding of causes |
| LLM (GPT, Claude) | L1 + imitation L2-L3 | Reproduces causal reasoning patterns | No causal world model |
| Causal ML (DoWhy, EconML) | L2 | Estimating intervention effects | Requires correct model |
| Structural Causal Model | L2 + L3 | Interventions + counterfactuals | Hard to verify the model |
| CFR (Libratus, AlphaStar) | L2-L3 | Planning under incomplete information | Only for structured environments |
**Key insight:** imagination uses the same causal model as perception, but with sensory input "switched off" (top-down only). This is why Embodied Cognition and Predictive Processing (lessons 15 and 12) are fundamentally linked to causality - the brain's generative model is a causal model of the world.
Connections to other topics
Causality is the foundation for planning and genuine agent understanding.
- Embodied Cognition — Affordances - causal knowledge: action -> consequence in the environment
- Predictive Processing — The brain's generative model is a causal model of the world used for prediction
- Cognitive Architectures — SOAR/ACT-R style architectures incorporate causal reasoning for action planning
Chain-of-Thought prompting turns an LLM into a causal planner
CoT improves the quality of L1 imitation but does not create a causal model: L2 and L3 require variables U, the do-operation and abduction, not a longer textual chain
CoT helps on tasks where the correct answer is a function of observations in the training distribution. Counterfactuals and interventions require a structural object (an SCM) that the LLM does not build. On out-of-distribution causal tasks (modified Simpson's Paradox, rephrased RCTs) CoT agents fail in predictable ways - this is empirically confirmed by Schölkopf et al. and the CLadder benchmark.
Connection to previous lessons
Predictive Processing sits at Pearl's L1 (correlation); Embodied Cognition supplies the sensorimotor substrate for interventions. Causal inference closes the picture with a formal language for L2/L3.
- Predictive Processing — an L1 machine in Pearl's terms: models P(Y|X) but not P(Y|do(X))
- Embodied Cognition — sensorimotor substrate for interventions that turn correlation into causal evidence
- Pearl ladder of causation — formal language that explains the L1 ceiling of purely statistical LLM systems
Summary
- Pearl's Ladder: L1 association (P(Y|X)), L2 intervention (P(Y|do(X))), L3 counterfactual (P(Y_x|X'=x', Y'=y')) - each rung is strictly more expressive than the previous and cannot be reduced to it
- The do-operator differs from conditioning: do(X=x) removes all incoming arrows into X in the SCM graph, severing confounder influence on X
- Backdoor criterion identifies a valid adjustment set Z: block every backdoor path between X and Y through Z without opening new paths via colliders
- Counterfactuals require three steps: abduction (recover P(U|evidence)), action (apply do), prediction (compute Y in the modified model) - L1 and L2 data alone are insufficient
- LLMs trained on correlational data are fundamentally stuck at L1: without explicit SCM structure they fail out-of-distribution causal tests (CLadder benchmark, rephrased Simpson's Paradox)
Вопросы для размышления
- Think of a claim from news or research that sounded causal. What level was it actually at - L1, L2, or L3? What would need to be done to verify it as L2?
Связанные уроки
- aut-15-embodied — Embodied agent creates causal chains in environment
- aut-13-iit — IIT phi is a measure of causal integration
- prob-04-bayes — Bayesian inference is the basis of causal updating
- cc-01-dags — Causal DAG is the formal language of Pearl causality
- cc-09-counterfactuals — Counterfactuals are the next level of causal inference
- prob-03-conditional