5.6 Summary
Chapter 5 in review
Chapter 5 extended the linear model to include multiple predictors. The key lesson is that multiple regression is not a magic wand — adding variables can reveal, mask, or distort associations. The modeler must reason about causal structure, not just statistical significance.
We covered:
-
Spurious associations (Section 5.1): A bivariate association disappears when a common cause is added. The divorce example showed that the marriage-divorce association is spurious — median age is the common cause.
-
Masked relationships (Section 5.2): A real association is hidden by an indirect path. The milk energy example showed that body mass has a strong negative direct effect on milk energy, masked by the positive indirect path through brain size.
-
When adding variables hurts (Section 5.3): Multicollinearity inflates standard errors, and post-treatment bias blocks causal paths. The decision to add variables must be guided by the causal graph.
-
Categorical variables (Section 5.4): Index variables are preferable to dummy variables in Bayesian models — they treat categories symmetrically and extend to multilevel models.
-
OLS and
lm(Section 5.5): OLS is the Bayesian model with flat priors. Informative priors usually improve the estimates.
Key lessons
- Draw the causal graph before fitting the model.
- Confounders should be added; post-treatment variables should not.
- Multicollinearity inflates uncertainty but does not bias point estimates.
- Index variables are the natural encoding for categorical predictors in Bayesian models.
- OLS is a special case of the Bayesian model with flat priors.
The causal graph is the strategy; the regression is the tool. This is the central message of Chapter 5. Multiple regression is a powerful tool, but it is only as good as the causal reasoning behind it. Without a causal graph, you cannot know whether adding a variable will help or hurt. With a causal graph, you can make principled decisions and interpret the results correctly. This is the foundation of structural causal models, which we will study in more detail in later chapters.