λ

Bayesian Thinking

The most powerful framework for reasoning under uncertainty

1

Start with a real problem

2

Build intuition through visualization

3

Discover the formula yourself

4

Apply it to machine learning

~10 minutes • Interactive

THE CHALLENGE

The Doctor's Dilemma

A disease affects 10% of the population.

A test has 90% accuracy (for both positive and negative).

You test positive. What's the probability you're actually sick?

Drag to make your guess:

50%

0% 100%

Most people guess around 80-90%. Let's see why they're wrong...

KEY INSIGHT

Think in Areas

Imagine 1000 people. Let's sort them:

Healthy (900)

810 test negative ✓

90 false positives ✗

Sick (100)

90 true positives ✓

10 false negatives ✗

False positive True positive

Who tests positive?

90 sick + 90 healthy = 180 total

Of these, only 90 are actually sick

P(sick | positive) = 90/180 = 50%

CHECK UNDERSTANDING

Why did most people guess wrong?

What crucial information did they ignore?

The test accuracy (90%)

The base rate of disease (10%)

The number of people tested

The age of patients

THE FORMULA

Bayes' Theorem

P(A|B) = P(B|A) × P(A) / P(B)

P(B|A)

Likelihood

P(A)

Prior

P(A|B)

Posterior

In our medical example:

P(sick | test+) = P(test+ | sick) × P(sick) / P(test+)

= 0.9 × 0.1 / 0.18

= 0.5

Core insight: Your prior belief matters just as much as the evidence.

INTERACTIVE

MLE vs MAP: The Coin Flip

This coin is biased (70% heads). Watch how estimators behave.

Flips: 0 Heads: 0 | Tails: 0

True θ = 0.70 ●

MLE = 0.50 ●

MAP (α=2, β=2) = 0.50 ●

Adjust prior (Beta distribution):

α: 2

β: 2

MLE can jump to 0 or 1 with few flips. MAP stays closer to the prior.

CHECK UNDERSTANDING

What's the key difference?

Between MLE and MAP?

MLE incorporates prior knowledge, MAP doesn't

MAP adds prior knowledge to prevent overfitting

They always give the same answer

MLE is always better with more data

KEY REVELATION

Regularization = Prior

L2 Regularization

loss + λ||w||²

↕

Gaussian prior

L1 Regularization

loss + λ|w|

↕

Laplace prior

Practical: When you add regularization, you're saying "I believe weights should be small." That's a Bayesian prior.

INTERACTIVE

Regularization Strength

Regularization (λ) 0.10

No prior (overfit) Strong prior (underfit)

Low λ → model fits noise (overfitting)

SYNTHESIS

At its core, Bayesian thinking is:

A way to compute exact probabilities

A framework for updating beliefs with evidence

A replacement for neural networks

Only useful for medical diagnosis

SUMMARY

What You've Learned

1. Base Rate Matters

Prior probability dramatically affects posteriors.

2. MLE Overfits, MAP Regularizes

MAP adds prior knowledge as a regularizer.

3. Regularization = Prior

L2 = Gaussian prior. L1 = Laplace prior.

4. Update Beliefs Correctly

Bayes' theorem is the correct way to incorporate evidence.

Ready for the skill check?

3 questions to test your understanding

SKILL CHECK

Test Your Understanding

1. Spam filter: 95% accurate, 2% spam. If flagged, P(actually spam)?

2. 3 flips, 3 heads. MLE estimate for P(heads)?

3. Weight decay (L2) is equivalent to:

COMPLETE

λ

3 / 3 Correct

Key Formulas

Bayes P(A|B) = P(B|A) × P(A) / P(B)

MAP θ = argmax P(D|θ) × P(θ)

L2 loss + λ||w||² ↔ Gaussian

Next: Apply these to your ML models. Every regularization is a prior.