Probability Theory
Continuous Distributions
Цели урока
- Understand the difference between discrete and continuous distributions
- Master the uniform distribution as the "ideal case"
- Understand the exponential distribution and its properties
- See the connection between the Poisson process and the exponential distribution
- Apply these distributions to real-world problems
Предварительные знания
- Random variables and probability density
- Discrete distributions (especially Poisson)
- Integration
**The bus-waiting paradox.** Buses arrive every 10 minutes on average. Arrival at the stop happens at a random moment. How long is the wait? Intuition says: 5 minutes (half the interval). But in practice the wait is **longer**! Why? The answer lies in how the distribution of intervals is shaped.
- Radioactive decay: when will a nucleus decay?
- Hardware reliability: when will a server fail?
- Queues: how long to wait at the supermarket checkout?
- Telecommunications: intervals between data packets
- Insurance: when will an insured event occur?
The randomness of radioactive decay
Physicists noticed that the clicks of a Geiger counter followed a strange pattern: the intervals between clicks are **not uniform**, but obey a simple formula. This is the exponential distribution - and it shows up everywhere that events "have no memory".
Continuous Distributions
Continuous distributions describe variables that take any value in an interval: waiting time, height, temperature. They have no 'weight' at a single point - instead they have a **probability density** $f(x)$, and probabilities over intervals come from integration.
Main continuous distributions: **uniform** (all outcomes in an interval equally likely), **exponential** (time between rare events), **normal** (the classic - next lesson). Each has its own story and its own formulas for $\mathbb{E}$ and $\text{Var}$.
The 'memoryless' property of the exponential distribution explains the bus paradox: the longer you wait, the expected remaining time does not decrease. Mathematically exact, even if counterintuitive.
What is the key difference between a continuous and a discrete random variable?
For a continuous $X$, only $P(a<X<b)=\int_a^b f(x)dx$ is meaningful, where $f$ is the density. Point probabilities are always zero. This is what distinguishes the integral definition from the discrete sum.
1. Uniform distribution - democracy of probabilities
1. Uniform distribution - democracy of probabilities
Start with the simplest case: **all values are equally likely**.
Consider a random point on the segment [0, 10]. What is the probability of landing closer to 3 than to 7? Closer to the start than to the end?
The density is **constant** over the entire interval:
Why $1/(b-a)$? Because the area under the density curve must equal 1:
**Characteristics:** $E[X] = \frac{a + b}{2}$ - midpoint of the interval $Var[X] = \frac{(b-a)^2}{12}$
The probability of landing in a sub-interval is simply the **ratio of lengths**:
The ideal bus
If intervals were uniform
Buses arrive exactly every 10 minutes. Arrival at the stop happens at a random moment. The waiting time $X \sim U(0, 10)$. $E[X] = \frac{0 + 10}{2} = 5$ minutes - as expected. $P(X > 8) = \frac{10 - 8}{10 - 0} = 20\%$ But in reality intervals are **not** uniform...
A random point is chosen uniformly on [0, 100]. What is the probability it falls in [40, 60]?
$P(40 \leq X \leq 60) = \frac{60 - 40}{100 - 0} = \frac{20}{100} = 20\%$. Length of the sub-interval divided by the length of the full interval.
2. Exponential distribution - waiting time
2. Exponential distribution - waiting time
Now for reality. In a Poisson process, events occur **randomly and independently**. The number of events in time t is $Poisson(\lambda t)$.
Question: how much **time** will pass until the next event?
The density decays exponentially:
The distribution function (probability that the event occurs before time t):
And a very useful formula for the "tail":
**Characteristics:** $E[T] = \frac{1}{\lambda}$ - mean waiting time $Var[T] = \frac{1}{\lambda^2}$ $\sigma = \frac{1}{\lambda} = E[T]$ - standard deviation equals the mean.
Server reliability
Server crashes on average once a month
$\lambda = 1$ crash/month, so $E[T] = 1$ month. **What is the probability the server runs for 2 months without a crash?** $P(T > 2) = e^{-1 \cdot 2} = e^{-2} \approx 0.135 = 13.5\%$ **What is the probability of a crash in the first week?** 1 week ≈ 0.25 months. $P(T < 0.25) = 1 - e^{-0.25} \approx 0.22 = 22\%$
Customers arrive at a store at an average rate of 6 per hour (Poisson). What is the average time between customers?
If λ = 6 customers/hour, the average time between customers is E[T] = 1/λ = 1/6 hour = 10 minutes. Exponential distribution!
3. The memoryless property
3. The memoryless property
The exponential distribution has a key property - **memorylessness**:
Translation: after s minutes of waiting with the event not yet happening, the probability of waiting another t minutes is the same as if the wait had just started!
Proof:
If I've been waiting a long time, something must happen soon
In the exponential distribution, the past has no effect on the future
Intuition misleads us! If the server has been running for 2 months without crashing, that does **not** mean it's "about to fail". The probability of crashing next month is exactly the same. The past is "erased".
**Is this realistic?** For radioactive decay - yes. For light bulbs and servers - not quite. Real devices "age", and the probability of failure increases over time. For such cases, the Weibull distribution is used.
A light bulb with an average lifetime of 1,000 hours has been running for 999 hours. What is the probability it will run for another 1,000 hours?
Because of memorylessness, the 999 hours already run do NOT affect the future! P(another 1,000 hours) = P(T > 1,000) = e^(-1) ≈ 37%. The same probability as for a brand-new bulb.
4. Resolving the bus paradox
4. Resolving the bus paradox
Now we can explain the paradox! If buses arrive according to a **Poisson process**, then the intervals between them follow an **exponential distribution**.
Let buses arrive on average every 10 minutes (λ = 0.1 buses/min). The interval between buses:
But here's the trick: arriving at a **random** moment makes landing in a **long** interval more likely!
**Inspection paradox:** picking a random point on the time axis, the probability of landing in an interval is proportional to its length. Long intervals "catch" more random moments.
Because of the exponential distribution of intervals, the average waiting time for a **random** passenger is:
This is because the exponential distribution is memoryless: regardless of arrival moment, the average time until the next bus is still 10 minutes.
Buses follow a Poisson process with an average interval of 15 minutes. Arrival at the stop happens at a random moment. What is the average wait?
In a Poisson process, waiting time is exponential. Due to memorylessness, the average wait = average interval = 15 minutes. This is the "bus paradox"!
5. Other continuous distributions
5. Other continuous distributions
Uniform and exponential are the basic building blocks. Here are a few more important ones:
| Distribution | Application | Key feature |
|---|---|---|
| Gamma(α, β) | Time until the α-th event | Sum of α exponentials |
| Beta(α, β) | Probabilities, proportions | Defined on [0, 1] |
| Weibull(k, λ) | Reliability with "aging" | Generalizes exponential |
| Normal(μ, σ) | Everywhere! | Next lesson! |
**Poisson ↔ Exponential ↔ Gamma relationship:** - Number of events in time t → Poisson(λt) - Time until the 1st event → Exp(λ) - Time until the k-th event → Gamma(k, λ)
Which distribution describes the time until the 3rd call arrives at a call center, if calls follow a Poisson process?
Time until the 1st event - exponential. Time until the k-th event - the sum of k independent exponentials, which gives a gamma distribution with parameter α = k.
Practice
Practice
Page load time is uniformly distributed between 2 and 8 seconds. Find the probability that the page loads in under 3 seconds.
$P(X < 3) = \frac{3 - 2}{8 - 2} = \frac{1}{6} \approx 16.7\%$
A server crashes on average once every 30 days. What is the probability it survives the next 60 days without crashing?
$\lambda = \frac{1}{30}$ $P(T > 60) = e^{-\frac{60}{30}} = e^{-2} \approx 0.135 = 13.5\%$ Only a 13.5% chance of surviving two average periods!
A radioactive isotope has a half-life of 10 years (half the nuclei decay in 10 years). Find λ and the probability that a specific nucleus decays within 1 year.
$P(T < 10) = 1 - e^{-10\lambda} = 0.5$ $e^{-10\lambda} = 0.5$ $-10\lambda = \ln(0.5) = -\ln(2)$ $\lambda = \frac{\ln(2)}{10} \approx 0.0693$ year$^{-1}$ $P(T < 1) = 1 - e^{-0.0693} \approx 0.067 = 6.7\%$ About 6.7% of nuclei will decay in the first year.
A server crashes on average once every 30 days (exponential time-to-failure). What is the probability it survives the next 60 days?
For $T \sim Exp(\lambda)$ with $E[T] = 30$ we have $\lambda = 1/30$. Then $P(T > 60) = e^{-\lambda\cdot 60} = e^{-2} \approx 0.135$. The exponential is memoryless, so its tail decays fast.
A bridge between discrete and continuous
These distributions link counting events to measuring time.
- Poisson Process — Number of events (discrete) + time between them (continuous)
- Normal Distribution — Limit of many distributions at large n
- Queueing Theory — Modeling service systems
- Reliability Analysis — Time to failure and reliability metrics
Итоги
- **Uniform U(a,b):** all values equally likely, $E = (a+b)/2$
- **Exponential Exp(λ):** waiting time, $f(t) = \lambda e^{-\lambda t}$, $E = 1/\lambda$
- **Memorylessness:** $P(T > s+t | T > s) = P(T > t)$ - the past has no effect on the future
- **Bus paradox:** in a Poisson process, wait = mean interval, not half of it
- **Relationship:** Poisson(count) ↔ Exponential(time to 1st) ↔ Gamma(time to k-th)
Вопросы для размышления
- Back to the bus paradox: how would the average wait change if intervals were uniform (not exponential)?
- Why is memorylessness realistic for radioactive decay but not for mechanical wear?
- If a server's average uptime is 1 year, what fraction of servers will survive 3 years?
- The exponential distribution is the only continuous distribution with the memoryless property. Which discrete distribution has this property too?