5.4 Categorical variables

Categorical predictors are common in statistical models. This section compares dummy variables and index variables, explaining why index variables are often more natural and easier to interpret in Bayesian models.

5.4.1 Dummy vs index variables

There are two ways to encode a categorical variable:

1. Dummy variables: Create $K-1$ binary indicators for $K$ categories. One category is the reference (baseline). Each coefficient is the difference between that category and the reference.

2. Index variables: Assign an integer index to each category. Use a vector of parameters $\alpha = [\alpha_1, \alpha_2, \ldots, \alpha_K]$, one per category. Each $\alpha_k$ is the expected value for category $k$.

# Compare dummy and index variable encoding
set.seed(42)
n <- 100

# Simulate data with 3 groups

group <- sample(1:3, n, replace = TRUE)
true_means <- c(150, 160, 170)
y <- true_means[group] + rnorm(n, sd = 5)

# Dummy variable encoding (group 1 is reference)

m_dummy <- lm(y ~ factor(group))
cat("Dummy variable encoding:\n")
cat(" Intercept (group 1):", round(coef(m_dummy)[1], 2), "\n")
cat(" Group 2 - Group 1:", round(coef(m_dummy)[2], 2), "\n")
cat(" Group 3 - Group 1:", round(coef(m_dummy)[3], 2), "\n")

# Index variable encoding (separate means)

m_index <- lm(y ~ 0 + factor(group))
cat("\nIndex variable encoding:\n")
cat(" Group 1 mean:", round(coef(m_index)[1], 2), "\n")
cat(" Group 2 mean:", round(coef(m_index)[2], 2), "\n")
cat(" Group 3 mean:", round(coef(m_index)[3], 2), "\n")

# Plot

boxplot(y ~ group, col = c("steelblue", "firebrick", "darkgreen"),
main = "Height by group",
xlab = "Group", ylab = "Height (cm)")
abline(h = true_means, lty = 2, col = "gray50")

5.4.2 Why index variables are better for Bayesian models

Index variables are often preferable in Bayesian models because:

1. No arbitrary reference: Dummy coding requires choosing a reference category, which affects the interpretation of all coefficients. Index variables treat all categories symmetrically.
2. Natural priors: With index variables, the prior applies to each category's mean directly. With dummy variables, the prior applies to differences, which is less intuitive.
3. Extends to multilevel models: Index variables generalize naturally to multilevel models (Chapter 12), where each category gets its own parameter drawn from a common distribution.
4. Easier to interpret: The index variable directly gives you the expected value for each category, without needing to add the intercept.

5.4.3 Multiple categories with predictors

We can combine categorical and continuous predictors:

$\mu_i = \alpha_{\text{group}[i]} + \beta x_i$

This gives each group its own intercept but a common slope.

# Model with categorical intercept and continuous predictor
set.seed(42)
n <- 150

# Three groups, each with different intercept but same slope

group <- sample(1:3, n, replace = TRUE)
x <- runif(n, 0, 10)
true_intercepts <- c(150, 155, 160)
y <- true_intercepts[group] + 2 * x + rnorm(n, sd = 3)

# Fit with index variable for group and continuous predictor

fit <- lm(y ~ 0 + factor(group) + x)
cat("Model: y = alpha[group] + beta * x\n")
cat(" alpha[1]:", round(coef(fit)[1], 2), "\n")
cat(" alpha[2]:", round(coef(fit)[2], 2), "\n")
cat(" alpha[3]:", round(coef(fit)[3], 2), "\n")
cat(" beta:", round(coef(fit)[4], 2), "\n")

# Plot

colors <- c("steelblue", "firebrick", "darkgreen")
plot(x, y, pch = 16, col = colors[group],
main = "Categorical intercepts with common slope",
xlab = "x", ylab = "y")
for (g in 1:3) {
abline(a = coef(fit)[g], b = coef(fit)[4], col = colors[g], lwd = 2)
}
legend("topleft", legend = paste("Group", 1:3), col = colors, lwd = 2)

Summary of Section 5.4

• Dummy variables encode categories as differences from a reference category.
• Index variables assign a separate parameter to each category.
• Index variables are preferable in Bayesian models: no arbitrary reference, natural priors, and they extend to multilevel models.
• Categorical and continuous predictors can be combined: $\mu_i = \alpha_{\text{group}[i]} + \beta x_i$.
• Categories are just a mapping from observations to parameters — the model treats them like any other parameter.