A positive result from an accurate test can still leave a disease unlikely. Bayes' theorem says how to weigh new evidence against what you believed before, one observation at a time.
You take a medical test and it comes back positive. If the test is 95% accurate, it is tempting to conclude that there is a 95% chance you have the disease. The right answer can be far lower. This part is about how a belief should change as each observation arrives.
Suppose a disease affects 1% of the population, and a test correctly flags 95% of sick people (its sensitivity) and correctly clears 95% of healthy people (its specificity). A positive result then means about a 16% chance of having the disease.
Figure 1 divides the population, drawn as a unit square, into the four outcomes: true and false positives, true and false negatives.
Figure 1. The area model of Bayes' theorem. Each rectangle's area is the joint probability of that outcome; dashed outlines mark everyone who tested positive. At the default 1% base rate the true positives are a thin blue column at the left and the false positives an amber strip along the top.
Most of the outlined area belongs to healthy people: the healthy group is so much larger that its 5% false-positive rate adds up to more cases than the sick group's true positives. At 10% prevalence the same test gives a posterior above 67%.
The formula behind the figure is Bayes' theorem:
In words: the posterior (the probability of the hypothesis given the evidence) equals the likelihood (the probability of the evidence under the hypothesis), times the prior (the base rate of the hypothesis), divided by the evidence (the total probability of seeing what we saw).
The denominator counts every way to see the evidence, false positives included, which is why a low base rate pulls the posterior down.
Bayes' theorem can be applied again each time evidence arrives. Start with a coin of unknown bias and a uniform prior, every bias from 0 to 1 equally plausible; after each flip the posterior becomes the prior for the next.
Figure 2. Sequential updating. Top: the belief distribution over the coin's bias (a beta distribution). Bottom: the posterior mean, the estimated probability of heads, as observations accumulate.
A common objection to Bayesian reasoning is that the prior is subjective. In Figure 3, three observers watch the same flips from different priors: uniform (no opinion), skeptical (the coin is probably fair) and credulous (it is probably biased toward heads).
Figure 3. Three priors, same evidence. Auto-flips are simulated from a coin with bias 0.6; the Heads and Tails buttons add flips of your choosing.
With few observations the three estimates differ; with enough they agree, but slowly, because the skeptical prior counts as 40 imaginary flips. A Beta(a, b) prior and h heads in n flips give a posterior mean of (a + h)/(a + b + n), which differs from the fraction h/n by a term that shrinks like 1/n. Each observer's own uncertainty, the posterior standard deviation, shrinks only like 1/√n. Figure 4 measures both.
Figure 4. The three priors of Figure 3 fed the same simulated flips, for 20 seeds, on log scales. Purple: the spread from the lowest to the highest posterior mean. Blue: the uniform observer's posterior standard deviation. Lines are medians over seeds, bands the 10th to 90th percentiles.
So the observers disagree by less than their own uncertainty long before they agree closely: at bias 0.6 after about 100 flips, against nearly 600 to agree within 0.01. At 0.9 both take over a thousand.
Figure 5 reads the same flips two ways: the frequentist way, as the fraction of heads with a 95% confidence interval (the Wilson score interval), and the Bayesian way, as a posterior from a uniform prior with its 95% credible interval. Only the posterior supports a direct probability statement: after 69 heads in 115 flips it puts 73% probability on the bias lying between 0.55 and 0.65.
Figure 5. The same simulated flips analyzed both ways: frequentist panel first (left, or above on a phone), Bayesian second. The slider sets the simulated coin's bias; moving it starts a new coin and clears the flips.
The two intervals answer different questions. The frequentist asks, "What procedure gives intervals that, over many repeats of this experiment, contain the true value 95% of the time?" Any one interval either contains the fixed true bias or it does not. The Bayesian asks, "Given what I have seen, what do I believe about the true value now?" Figure 6 runs both procedures on many simulated experiments.
Figure 6. Thirty simulated experiments of 20 flips at bias 0.85, each with its Wilson interval and its skeptical-prior credible interval (red: misses the true bias), and the exact share of experiments each kind of interval covers at every bias. Click the curve to move the bias.
At 20 flips the Wilson interval covers between 92% and 98% of experiments at every bias from 0.05 to 0.95. The skeptical interval covers 90% or more only between 0.37 and 0.63, and it can never reach 0.85. Its 95% is an average over coins drawn from its own prior, and over those it is exact. The uniform prior's interval stays within 5 points of Wilson's from 0.1 to 0.9.
Here the evidence arrives the same way whatever the observer does. Part 6 turns to an agent whose actions change what it observes, and which learns from the rewards those actions earn.