Logic

Bayesian Thinking

The doctor says: 'The cancer test is positive.' How worried should you be? Intuition screams '99%!', but math says '10%'. The difference can cost you years of stress or an unnecessary operation. Bayes' theorem is not an abstraction - it is a tool for making decisions under uncertainty.

  • **Medical diagnostics:** why doctors order repeat tests after a positive result - one test doesn't provide enough certainty for rare diseases
  • **Spam filters:** Bayesian filters consider how likely the word 'viagra' is in spam vs. normal emails, and update their estimate with each word
  • **Forensic science:** understanding base rates helps correctly interpret DNA evidence and avoid convicting the innocent

Bayes' Theorem

Imagine: a test for a rare disease comes back positive. How likely is it that you're actually sick? Most people would answer 'very likely' - and they'd be wrong. **Bayes' theorem** is a mathematical tool that helps you correctly update probabilities when you receive new data.

**Bayes' theorem:** P(H|E) = P(E|H) × P(H) / P(E) Where: • P(H|E) - the probability of hypothesis H given evidence E • P(E|H) - the probability of seeing E if H is true • P(H) - the prior probability of H (before seeing E) • P(E) - the overall probability of seeing E

Why so? Because there are far more healthy people than sick ones. Even with a low error rate, the number of false positives exceeds the number of true positives. This is called the **false positive paradox** - and understanding Bayes' theorem protects against this trap.

A drug test has 95% accuracy. 2% of people use drugs. If the test is positive, what is the approximate probability that the person actually uses drugs?

0

1

Sign In

Prior and Posterior Probability

In Bayesian thinking, the key concepts are the **prior** and the **posterior**. The prior is what you believe *before* receiving new data. The posterior is what you *should* believe after.

**Key terms:** • **Prior P(H)** - the initial probability of a hypothesis before receiving evidence • **Posterior P(H|E)** - the updated probability after receiving evidence E • **Likelihood P(E|H)** - the probability of observing E if H is true • **Evidence P(E)** - the overall probability of observing E (normalizing factor)

Notice: **the prior matters**. If there were not 1,000 suspects but 10 million (the whole country), the posterior would be very different. This explains why rare events need stronger evidence - a low prior 'resists' updating.

**The subjectivity of the prior** is a common argument against Bayesianism. But this is not a weakness - it is honesty: we *acknowledge* that we have initial beliefs and *explicitly* record them. With enough data, different priors converge to the same posterior - data 'overcomes' biases.

Why might two people rationally arrive at different posteriors given the same evidence (a positive test)?

Updating Beliefs

Bayesian thinking is not a one-time calculation but a **continuous process**. Each new piece of evidence becomes the input for the next update. Yesterday's posterior becomes today's prior.

**Rules for updating:** 1. **Each piece of evidence is counted once** - you can't use the same fact twice 2. **Order doesn't matter** - with the same evidence, the final posterior is the same 3. **Evidence is independent** - if dependent, a correction is needed 4. **Updating must be proportional** - strong evidence changes more

**Confirmation bias in Bayesian terms:** people seek evidence with high P(E|H), ignoring P(E|¬H). If evidence is equally probable under any hypothesis, it **does not change** the posterior. 'The astrologer predicted I'd meet someone today' is not evidence, because P(meeting|astrology is true) = P(meeting|astrology is false).

**Important:** updating works both ways. Evidence can both *increase* and *decrease* your confidence in a hypothesis. If an experiment found no effect, that is also information - decreasing the posterior of the hypothesis.

You estimate the probability of rain at 30%. A friend says the radio forecast predicted rain. How will your estimate change?

The Strength of Evidence

Not all evidence is equally valuable. The **strength of evidence** is determined by how much it *distinguishes* between hypotheses. The key measure is the **likelihood ratio**.

**Likelihood Ratio (LR):** LR = P(E|H) / P(E|¬H) • LR > 1 → evidence supports H • LR < 1 → evidence is against H • LR = 1 → evidence is uninformative **Rule:** Posterior odds = Prior odds × LR Very strong evidence: LR > 10 or LR < 0.1

**The rule of 'miraculous' evidence:** if hypothesis H predicts event E, which would be a miracle without H, then E is strong evidence for H. A DNA match is a 'miracle' for an innocent person (1 in a million), but not for the guilty (100%). That's why the LR is enormous.

**The danger of weak evidence:** it seems significant but barely changes the posterior. 'The suspect appeared nervous' - P(nervous|guilty) is high, but P(nervous|innocent being questioned by police) is also high! The LR is close to 1. This is pseudo-evidence.

Any confirming piece of evidence is equally valuable

The value of evidence is determined by how much it distinguishes between hypotheses (likelihood ratio)

Evidence E that is equally probable under H and ¬H does not change your confidence. 'The suspect is breathing' is true, but it is not evidence of guilt because innocent people also breathe. The strength of evidence = how 'surprising' it is if the hypothesis is false.

Which piece of evidence most strongly supports the hypothesis 'this person is a doctor'?

Key Takeaways

  • **Bayes' theorem** connects the prior (initial probability) with the posterior (updated one) via the likelihood (plausibility of evidence)
  • **The prior matters:** for rare events, even a very accurate test produces many false positives
  • **Updating is gradual:** each new piece of evidence adjusts beliefs; yesterday's posterior is today's prior
  • **Evidence strength (LR):** the more evidence distinguishes between hypotheses, the more it changes the posterior

Related Topics

Bayesian thinking is the mathematical foundation for the rational updating of beliefs:

  • Abduction — Bayes provides a formal way to select the 'best explanation' via the likelihood ratio
  • Base Rate — The prior (base rate) is critically important - ignoring it leads to probability estimation errors

Вопросы для размышления

  • Recall a situation where you overreacted to a 'positive test' (medical, interview, review). What was the actual base rate?
  • Which evidence do you overvalue (weak LR, but seems important)? For example, 'he looked honest' when evaluating people.
  • How would your relationship with the news change if you explicitly estimated the prior and likelihood for every sensational headline?

Связанные уроки

  • ml-18
Bayesian Thinking