5.1 Spurious association

A strong bivariate association can disappear or even reverse when we account for other variables. This is the problem of spurious association. Multiple regression helps us estimate the direct association between two variables while controlling for confounders.

5.1.1 The divorce data

The classic example uses US state-level data on divorce rate, marriage rate, and median age at marriage. The question: does marriage rate cause divorce rate, or is the association spurious?

# Simulate the divorce data (WaffleDivorce-style)
set.seed(42)
n_states <- 50

# Median age at marriage influences both marriage rate and divorce rate

median_age <- rnorm(n_states, mean = 26, sd = 1.5)

# Marriage rate is influenced by median age

marriage_rate <- 20 - 0.5 * median_age + rnorm(n_states, sd = 3)

# Divorce rate is influenced by median age (not directly by marriage rate)

divorce_rate <- 12 - 0.4 * median_age + rnorm(n_states, sd = 1.5)

# Standardize

divorce_s <- (divorce_rate - mean(divorce_rate)) / sd(divorce_rate)
marriage_s <- (marriage_rate - mean(marriage_rate)) / sd(marriage_rate)
median_age_s <- (median_age - mean(median_age)) / sd(median_age)

# Bivariate correlations

cat("Correlation matrix:\n")
print(round(cor(cbind(divorce_s, marriage_s, median_age_s)), 3))

# Plot

par(mfrow = c(1, 3))
plot(marriage_s, divorce_s, pch = 16, col = "steelblue",
xlab = "Marriage rate (standardized)", ylab = "Divorce rate (standardized)",
main = "Bivariate: marriage vs divorce")
abline(lm(divorce_s ~ marriage_s), lwd = 2, col = "firebrick")

plot(median_age_s, divorce_s, pch = 16, col = "steelblue",
xlab = "Median age (standardized)", ylab = "Divorce rate (standardized)",
main = "Bivariate: age vs divorce")
abline(lm(divorce_s ~ median_age_s), lwd = 2, col = "firebrick")

plot(median_age_s, marriage_s, pch = 16, col = "steelblue",
xlab = "Median age (standardized)", ylab = "Marriage rate (standardized)",
main = "Bivariate: age vs marriage")
abline(lm(marriage_s ~ median_age_s), lwd = 2, col = "firebrick")
par(mfrow = c(1, 1))

Click Run to execute this code in your browser with WebR

The bivariate correlation between marriage rate and divorce rate is positive and strong. But median age at marriage is also correlated with both. Is the marriage-divorce association real, or is it spurious?

5.1.2 The multiple regression model

The multiple regression model includes both predictors:

$D_i \sim \text{Normal}(\mu_i, \sigma)$ $\mu_i = \alpha + \beta_M M_i + \beta_A A_i$ $\alpha \sim \text{Normal}(0, 0.2)$ $\beta_M \sim \text{Normal}(0, 0.5)$ $\beta_A \sim \text{Normal}(0, 0.5)$ $\sigma \sim \text{Exponential}(1)$

where $D_i$ is divorce rate, $M_i$ is marriage rate, and $A_i$ is median age at marriage (all standardized).

# Multiple regression: divorce ~ marriage + age
# Using simulated data from the previous block

# Bivariate model: divorce ~ marriage only

m_biv <- lm(divorce_s ~ marriage_s)
cat("Bivariate model (divorce ~ marriage):\n")
cat(" beta_M:", round(coef(m_biv)[2], 3), "\n")

# Multiple regression: divorce ~ marriage + age

m_multi <- lm(divorce_s ~ marriage_s + median_age_s)
cat("\nMultiple regression (divorce ~ marriage + age):\n")
cat(" beta_M:", round(coef(m_multi)[2], 3), "\n")
cat(" beta_A:", round(coef(m_multi)[3], 3), "\n")

cat("\nAfter controlling for median age,\n")
cat("the marriage rate coefficient shrinks toward zero.\n")
cat("This suggests the marriage-divorce association is spurious.\n")

Click Run to execute this code in your browser with WebR

5.1.3 Interpreting the coefficients

The coefficient $\beta_M$ in the multiple regression tells us the partial association between marriage rate and divorce rate, holding median age constant. If $\beta_M$ shrinks to near zero after adding age, the bivariate association was spurious — age was the common cause.

# Visualize the spurious association
# Plot the coefficients from bivariate vs multivariate models

# Bivariate coefficients

biv_M <- coef(m_biv)[2]

# Multivariate coefficients

multi_M <- coef(m_multi)[2]
multi_A <- coef(m_multi)[3]

# Plot

par(mfrow = c(1, 2))

# Coefficient comparison

barplot(c(biv_M, multi_M), names.arg = c("Bivariate", "Multivariate"),
col = c("steelblue", "firebrick"), border = "white",
main = "beta_M: marriage rate effect",
ylab = "Coefficient")
abline(h = 0, lty = 2)

# Predictor effects

barplot(c(multi_M, multi_A), names.arg = c("beta_M", "beta_A"),
col = c("steelblue", "firebrick"), border = "white",
main = "Multivariate coefficients",
ylab = "Coefficient")
abline(h = 0, lty = 2)

par(mfrow = c(1, 1))

cat("The marriage effect (beta_M) nearly disappears\n")
cat("after controlling for median age.\n")
cat("The age effect (beta_A) remains strong.\n")
cat("This is the signature of a spurious association.\n")

Click Run to execute this code in your browser with WebR

5.1.4 The causal story

The spurious association arises because median age at marriage is a common cause of both marriage rate and divorce rate. The causal graph is:

$A \rightarrow M$ $A \rightarrow D$

There is no direct arrow from $M$ to $D$ . The bivariate association between $M$ and $D$ is entirely due to the common cause $A$ . Multiple regression estimates the direct effect of $M$ on $D$ by conditioning on $A$ , which removes the spurious path.

Rethinking

Regression is not causation. Multiple regression can estimate direct effects, but only if the causal graph is correct. If we condition on the wrong variables, we can create or amplify bias. The key is to draw the causal graph (a DAG) and use it to decide which variables to include. This is the approach of structural causal models, which we study in more detail in later chapters. The regression is a tool; the causal graph is the strategy.

Overthinking

The formula for the partial coefficient. In a multiple regression $y = \alpha + \beta_1 x_1 + \beta_2 x_2 + \epsilon$ , the coefficient $\beta_1$ is the effect of $x_1$ on $y$ after removing the linear effect of $x_2$ from both $x_1$ and $y$ . Mathematically:

$\beta_1 = \frac{\text{Cov}(y, x_1) - \text{Cov}(y, x_2) \cdot \text{Cov}(x_1, x_2) / \text{Var}(x_2)}{\text{Var}(x_1) - \text{Cov}(x_1, x_2)^2 / \text{Var}(x_2)}$

This is the Frisch-Waugh-Lovell theorem: the partial coefficient can be obtained by (1) regressing $x_1$ on $x_2$ and taking residuals, (2) regressing $y$ on $x_2$ and taking residuals, (3) regressing the $y$ -residuals on the $x_1$ -residuals. This shows that the partial coefficient measures the association between $x_1$ and $y$ that is not explained by $x_2$ .

Summary of Section 5.1

Spurious associations arise when a common cause creates a bivariate association that disappears after controlling for the confounder.
Multiple regression estimates the partial (direct) association between each predictor and the outcome.
The divorce example shows that the marriage-divorce association is spurious — median age is the common cause.
Regression is not causation: the causal graph determines which variables to include.
The Frisch-Waugh-Lovell theorem explains how partial coefficients work.

Practice

Masked relationship