2.3 Components of the model

A Bayesian model has three components: the likelihood, the parameters, and the prior. Each plays a distinct role, and confusing them leads to common mistakes. In this section, we examine each component in detail and show how they work together.

2.3.1 Variables

Variables in a Bayesian model come in several types:

Observed variables ( $D$ ): The data we actually collect. These are fixed and known.
Unobserved variables ( $\theta$ ): The parameters we want to estimate. These are unknown and are described by probability distributions.
Derived quantities: Functions of parameters, such as predictions or fitted values. These inherit uncertainty from the parameters.

The key distinction is between observed and unobserved variables. Observed variables are fixed; unobserved variables have distributions. The posterior distribution describes what we know about the unobserved variables after seeing the data.

2.3.2 The likelihood

The likelihood function $P(D \mid \theta)$ tells us how plausible the data are for each value of the parameters. It is not a probability distribution over $\theta$ — it is a function of $\theta$ given fixed $D$ .

For the globe-tossing example, the likelihood is:

$P(D \mid p) = \binom{n}{w} p^w (1-p)^{n-w}$

where $w$ is the number of waters and $n$ is the number of tosses. The binomial coefficient $\binom{n}{w}$ counts the number of ways to arrange $w$ waters among $n$ tosses.

# Visualize the likelihood as a function of p
# For 6 waters in 9 tosses
p_grid <- seq(0, 1, length.out = 200)
likelihood <- dbinom(6, size = 9, prob = p_grid)

plot(p_grid, likelihood, type = "l", lwd = 2, col = "steelblue",
xlab = "Proportion of water (p)",
ylab = "Likelihood P(D | p)",
main = "Likelihood function for globe tossing")
abline(v = 6/9, lty = 2, col = "firebrick")
text(0.72, max(likelihood) * 0.9, "MLE = 6/9", col = "firebrick", pos = 4)

Click Run to execute this code in your browser with WebR

The likelihood peaks at $p = 6/9 \approx 0.667$ , the maximum likelihood estimate (MLE). But the likelihood is not a probability distribution over $p$ — it does not integrate to 1. To get a probability distribution, we need the prior and Bayes' theorem.

A common mistake is to treat the likelihood as a probability over parameters. It is not. The likelihood is $P(D \mid \theta)$ , a probability over data given parameters. The posterior $P(\theta \mid D)$ is the probability over parameters given data.

2.3.3 Parameters

Parameters are the unknown quantities that index the model. In the globe-tossing example, the only parameter is $p$ , the proportion of water. In more complex models, there may be many parameters:

Mean parameters ( $\mu$ ): The expected value of the outcome.
Scale parameters ( $\sigma$ ): The spread or variability.
Regression coefficients ( $\beta$ ): The effect of predictors.
Variance components ( $\sigma_\alpha$ ): The variability of group-level effects.

Parameters are not "the truth." They are convenient quantities that index a family of possible data-generating processes. Different models may use different parameterizations of the same process, and the "true" parameter value is only meaningful relative to a specific model.

# Parameters in a simple normal model
# Model: y ~ Normal(mu, sigma)
# Parameters: mu (mean), sigma (standard deviation)

# Simulate some height data

set.seed(42)
heights <- rnorm(50, mean = 170, sd = 10)

# The parameters we want to estimate

cat("Sample mean (estimate of mu):", round(mean(heights), 2), "\n")
cat("Sample SD (estimate of sigma):", round(sd(heights), 2), "\n")
cat("True mu: 170, True sigma: 10\n")

# The likelihood surface for (mu, sigma) is a function of both parameters

# For a fixed sigma, the likelihood as a function of mu is:

mu_grid <- seq(160, 180, length.out = 100)
likelihood_mu <- sapply(mu_grid, function(m)
sum(dnorm(heights, mean = m, sd = sd(heights), log = TRUE)))

plot(mu_grid, exp(likelihood_mu - max(likelihood_mu)), type = "l", lwd = 2,
col = "steelblue", xlab = "mu", ylab = "Relative likelihood",
main = "Likelihood profile for mu (sigma fixed at sample SD)")
abline(v = mean(heights), lty = 2, col = "firebrick")

Click Run to execute this code in your browser with WebR

2.3.4 The prior

The prior $P(\theta)$ encodes what we know about the parameters before seeing the data. It is a model of our pre-data knowledge, not a bias or a prejudice.

The prior can be:

Informative: Based on previous studies, theory, or expert knowledge.
Weakly informative: Constrains parameters to reasonable ranges without strongly influencing the posterior.
Diffuse (uninformative): Spreads probability broadly, letting the data dominate.

The marginal likelihood (evidence) is:

$P(D) = \int P(D \mid \theta) \cdot P(\theta) \, d\theta$

This integral averages the likelihood over the prior, producing the probability of the data under the model. It is usually hard to compute analytically, which is why we work with the unnormalized posterior $P(D \mid \theta) \cdot P(\theta)$ and normalize numerically.

# Demonstrate the role of the prior
# Globe tossing: 6 waters in 9 tosses

p_grid <- seq(0, 1, length.out = 500)
likelihood <- dbinom(6, size = 9, prob = p_grid)

# Three different priors

priors <- list(
"Uniform" = rep(1, 500),
"Beta(2,2)" = dbeta(p_grid, 2, 2),
"Beta(10,10)" = dbeta(p_grid, 10, 10)
)

par(mfrow = c(1, 3))
for (name in names(priors)) {
prior <- priors[[name]]
post <- (likelihood _ prior) / sum(likelihood _ prior)

# Plot prior and posterior together

plot(p_grid, post / max(post), type = "l", lwd = 2, col = "steelblue",
xlab = "p", ylab = "Relative density",
main = name)
lines(p_grid, prior / max(prior), lwd = 2, col = "firebrick", lty = 2)
legend("topright", legend = c("Posterior", "Prior"),
col = c("steelblue", "firebrick"), lty = c(1, 2), lwd = 2, cex = 0.7)
}
par(mfrow = c(1, 1))

Click Run to execute this code in your browser with WebR

2.3.5 Putting it all together

The complete Bayesian model for the globe-tossing example can be written as:

$y_i \sim \text{Binomial}(n, p)$ $p \sim \text{Uniform}(0, 1)$

The first line is the likelihood: each observation $y_i$ is drawn from a binomial distribution with parameters $n$ and $p$ . The second line is the prior: $p$ is drawn from a uniform distribution on $[0, 1]$ .

This notation, called the model formula, is the standard way to specify Bayesian models. We will use it throughout the book.

Rethinking

The likelihood is not a probability over parameters. This is the most common source of confusion in Bayesian statistics. The likelihood $P(D \mid \theta)$ is a function of $\theta$ given fixed $D$ . It tells us how plausible the data are for each parameter value. But it is not a probability distribution over $\theta$ — it does not integrate to 1 over $\theta$ . The posterior $P(\theta \mid D)$ is the probability distribution over $\theta$ . It is obtained by multiplying the likelihood by the prior and normalizing. Confusing the likelihood with the posterior leads to errors like interpreting the MLE as the "most probable parameter value," which it is not — it is only the parameter value that makes the data most probable.

Overthinking

Why the likelihood is not a probability over parameters. Mathematically, the likelihood function $L(\theta) = P(D \mid \theta)$ is a function of $\theta$ for fixed $D$ . For it to be a probability density over $\theta$ , we would need $\int L(\theta) \, d\theta = 1$ . But in general, $\int P(D \mid \theta) \, d\theta \neq 1$ . For example, with the binomial likelihood $P(w \mid p) = \binom{n}{w} p^w (1-p)^{n-w}$ :

$\int_0^1 \binom{n}{w} p^w (1-p)^{n-w} \, dp = \binom{n}{w} B(w+1, n-w+1) = \binom{n}{w} \frac{w!(n-w)!}{(n+1)!} = \frac{1}{n+1}$

This is not 1 (unless $n = 0$ ). So the likelihood is not a probability density over $p$ . It is only after multiplying by the prior and normalizing that we get a proper probability distribution — the posterior.

Summary of Section 2.3

A Bayesian model has three components: likelihood, parameters, and prior.
The likelihood $P(D \mid \theta)$ is a probability over data, not over parameters.
Parameters are unknown quantities that index the model; they are not "the truth."
The prior $P(\theta)$ encodes pre-data knowledge; it is a model, not a bias.
The evidence $P(D)$ normalizes the posterior but is often computed numerically.
The model formula notation ( $y \sim \text{Binomial}(n, p)$ , $p \sim \text{Uniform}(0,1)$ ) is the standard way to specify Bayesian models.

Building a model

Making the model go