2.2 Building a model

A Bayesian model is built from three ingredients: a story about how the data arise, the parameters that appear in that story, and prior distributions that encode what we know before seeing the data. Using the globe-tossing example from Section 2.1, this section shows how to combine these ingredients into a posterior distribution.

2.2.1 The three ingredients of a Bayesian model

Every Bayesian model has three components:

1. The likelihood: A function that specifies the plausibility of the data, given the parameters. It tells us how the data could arise from the model.
2. The parameters: The unknown quantities we want to estimate. They index the family of possible data-generating processes.
3. The prior: A distribution over the parameters that encodes what we know (or believe) before seeing the data.

The posterior distribution combines all three:

$p(\theta \mid D) = \frac{p(D \mid \theta) \cdot p(\theta)}{p(D)} \propto p(D \mid \theta) \cdot p(\theta)$

The proportionality is sufficient for most computations, because we can always normalize at the end.

2.2.2 The globe-tossing model

For the globe-tossing example, the model is:

• Likelihood: $D \sim \text{Binomial}(n, p)$ — the number of waters in $n$ tosses follows a binomial distribution with probability $p$.
• Parameter: $p$ — the proportion of the globe covered by water.
• Prior: $p \sim \text{Uniform}(0, 1)$ — all values of $p$ are equally plausible before seeing the data.

Combining these, the posterior is:

$p(p \mid D) \propto \text{Binomial}(D \mid n, p) \cdot \text{Uniform}(p)$

# Build the globe-tossing model from its three ingredients
# 1. Parameter grid
p_grid <- seq(from = 0, to = 1, length.out = 100)

# 2. Prior: uniform over [0, 1]

prior <- rep(1, length(p_grid))

# 3. Likelihood: binomial probability of 6 waters in 9 tosses

likelihood <- dbinom(6, size = 9, prob = p_grid)

# 4. Posterior: multiply and normalize

unstd_posterior <- likelihood * prior
posterior <- unstd_posterior / sum(unstd_posterior)

# Summarize

cat("Posterior mean:", round(sum(p_grid * posterior), 4), "\n")
cat("Posterior mode:", round(p_grid[which.max(posterior)], 4), "\n")
cat("95% credible interval: [",
round(p_grid[which(cumsum(posterior) >= 0.025)[1]], 4), ",",
round(p_grid[which(cumsum(posterior) >= 0.975)[1]], 4), "]\n")

The posterior mean is approximately $0.67$, which matches the observed proportion $6/9$. The 95% credible interval gives us a range of plausible values for $p$.

2.2.3 A more refined grid

With 100 grid points instead of 20, the posterior is much smoother. Let us compare:

# Compare 20-point and 100-point grid approximations
par(mfrow = c(1, 2))

# 20-point grid

p20 <- seq(0, 1, length.out = 20)
post20 <- dbinom(6, 9, p20) / sum(dbinom(6, 9, p20))
plot(p20, post20, type = "b", pch = 16, col = "steelblue",
xlab = "p", ylab = "posterior", main = "20 grid points", lwd = 2)

# 100-point grid

p100 <- seq(0, 1, length.out = 100)
post100 <- dbinom(6, 9, p100) / sum(dbinom(6, 9, p100))
plot(p100, post100, type = "l", col = "firebrick",
xlab = "p", ylab = "posterior", main = "100 grid points", lwd = 2)

par(mfrow = c(1, 1))

The 20-point grid produces a jagged posterior, while the 100-point grid is smooth. In practice, we use enough grid points to capture the shape of the posterior without excessive computation. For models with many parameters, grid approximation becomes impractical, and we turn to quadratic approximation or MCMC (Chapter 8).

2.2.4 From the story to the model

The most important step in building a Bayesian model is writing down the data story. The story describes how the data arise, step by step. For the globe-tossing example:

1. The true proportion of water on the globe is $p$.
2. Each toss is independent, and the probability of landing on water is $p$.
3. The number of waters in $n$ tosses follows a binomial distribution.

This story translates directly into the likelihood function. The parameters appear naturally from the story, and the prior encodes what we know about them.

# Simulate data from the globe-tossing story
# True proportion of water
p_true <- 0.7

# Simulate 9 tosses

set.seed(42)
tosses <- rbinom(9, size = 1, prob = p_true)
cat("Simulated tosses (1=water, 0=land):", tosses, "\n")
cat("Number of waters:", sum(tosses), "\n")

# Now infer p from the simulated data

p_grid <- seq(0, 1, length.out = 100)
likelihood <- dbinom(sum(tosses), size = 9, prob = p_grid)
posterior <- likelihood / sum(likelihood)

cat("Posterior mean:", round(sum(p_grid * posterior), 3), "\n")
cat("True p:", p_true, "\n")

Notice that the posterior mean does not exactly equal the true $p$. This is expected — the posterior reflects what we can learn from the data, not the truth. With more tosses, the posterior would concentrate around the true value.

2.2.5 Priors as models

A prior is not just a technical necessity. It is a model of what we know. Different priors represent different states of knowledge:

• Uniform prior: We know nothing about $p$; all values are equally plausible.
• Informative prior: We have background knowledge that constrains $p$.
• Skeptical prior: We concentrate probability near a null value (e.g., $p = 0.5$).

# Compare priors as models of knowledge
p_grid <- seq(0, 1, length.out = 200)
likelihood <- dbinom(6, size = 9, prob = p_grid)

# Uniform: "I know nothing"

prior_unif <- rep(1, 200)

# Beta(2,2): "I weakly believe p is near 0.5"

prior_beta22 <- dbeta(p_grid, 2, 2)

# Beta(5,5): "I strongly believe p is near 0.5"

prior_beta55 <- dbeta(p_grid, 5, 5)

# Beta(1,1) = Uniform (for reference)

prior_beta11 <- dbeta(p_grid, 1, 1)

par(mfrow = c(2, 2))

for (prior in list(prior*unif, prior_beta22, prior_beta55, prior_beta11)) {
post <- (likelihood * prior) / sum(likelihood _ prior)
plot(p_grid, post, type = "l", lwd = 2, col = "steelblue",
xlab = "p", ylab = "posterior",
main = sprintf("Posterior mean: %.3f", round(sum(p_grid * post), 3)))
abline(v = 6/9, lty = 2, col = "gray50")
}

par(mfrow = c(1, 1))

The dashed line in each panel shows the observed proportion $6/9 \approx 0.667$. The uniform prior lets the data speak freely. The Beta(2,2) prior gently pulls the posterior toward 0.5. The Beta(5,5) prior pulls more strongly. This is regularization — the prior prevents the model from overfitting to small samples.

Summary of Section 2.2

• A Bayesian model has three ingredients: likelihood, parameters, and prior.
• The data story translates a narrative about how the data arise into a mathematical model.
• The posterior is proportional to the likelihood times the prior: $p(\theta \mid D) \propto p(D \mid \theta) \cdot p(\theta)$.
• Grid approximation computes the posterior at discrete parameter values; more grid points yield a smoother posterior.
• Priors are models of knowledge, not subjective biases. Different priors represent different states of information.
• The Beta distribution is the conjugate prior for the binomial likelihood, yielding a closed-form posterior.