1.1 Statistical golems

Statistical models are powerful but mindless tools. Like the golems of Jewish folklore — clay statues brought to life by inscribing words on their foreheads — statistical models do exactly what they are told, but they have no wisdom about whether those instructions are sensible. They cannot judge whether the question they answer is the one we meant to ask. This is the central metaphor of the book: statistical procedures are golems. They are not oracles. They are not judges. They are tools that must be engineered with care, understood deeply, and never trusted blindly.

The golem analogy

In the legend of the Golem of Prague, Rabbi Judah Loew ben Bezalel created a clay creature to protect the Jewish community. The golem was powerful and obedient, but it lacked judgment. It followed its instructions literally, sometimes with disastrous consequences. The golem could not question its orders. It could not adapt to context. It could only execute.

Statistical models are the same. A linear regression does not know whether the data are appropriate for a linear model. A $p$ -value does not know whether the null hypothesis is meaningful. A confidence interval does not know whether the parameter it estimates is the one we should care about. These procedures are golems — they do what they are told, but they cannot tell us whether the instructions are wise.

This means that the responsibility for using statistical tools wisely falls entirely on the modeler. We must:

Understand the machinery: Know what each procedure does, what assumptions it makes, and what it cannot do.
Question the assumptions: Check whether the model's assumptions are reasonable for the data at hand.
Compare alternatives: Never rely on a single model. Always compare with plausible alternatives.
Interpret with humility: A model's output is not truth; it is a conditional answer to a specific question.

Science before statistics

Statistics is not a substitute for scientific thinking. Before we build a statistical model, we need a scientific model — a causal story about how the data arose. The statistical model is a tool for reasoning about that story, not for replacing it.

The most common abuse of statistics is to treat it as a machine for producing "significant" results. This leads to:

Overfitting: Models that fit the sample data but fail to generalize.
Confounding: Models that estimate associations rather than causal effects.
Multiple testing: Running many tests and reporting only the significant ones.
Garden of forking paths: Making data-dependent decisions that inflate false positive rates.

Each of these abuses arises from treating the golem as an oracle rather than a tool.

# A simple demonstration: the golem does what it is told
# Fit a line through data that is clearly not linear
set.seed(42)
x <- runif(30, 0, 10)
y <- 2 * x^2 + rnorm(30, sd = 5)

# The golem (lm) fits a straight line without questioning

fit <- lm(y ~ x)
cat("Linear model coefficients:\n")
cat("Intercept:", round(coef(fit)[1], 2), "\n")
cat("Slope:", round(coef(fit)[2], 2), "\n")
cat("R-squared:", round(summary(fit)$r.squared, 3), "\n")

# Plot the data and the golem's answer

plot(x, y, pch = 16, col = "steelblue",
main = "The golem fits a line to curved data",
xlab = "x", ylab = "y")
abline(fit, lwd = 2, col = "firebrick")
legend("topleft", legend = c("Data", "Linear fit (golem)"),
col = c("steelblue", "firebrick"), pch = c(16, NA), lty = c(NA, 1), lwd = 2)

Click Run to execute this code in your browser with WebR

The linear model fits — it produces coefficients, $R^2$ , and $p$ -values. But the model is wrong: the true relationship is quadratic, not linear. The golem does not know this. It does what it is told. It is our job to recognize that a linear model is inappropriate and to try a better one.

The problem with procedural statistics

Much of statistical practice is procedural: apply a test, check a threshold, draw a conclusion. This approach treats statistics as a recipe — follow the steps and get the answer. But this is exactly how golems cause trouble. The procedure does not encode scientific judgment. It encodes a mechanical rule.

For example, the $p$ -value is the probability of observing data as extreme as ours, assuming the null hypothesis is true. But the $p$ -value does not tell us:

Whether the null hypothesis is plausible.
Whether the effect is practically important.
Whether the model that produced the $p$ -value is correct.
Whether a different model would give a different answer.

$p\text{-value} = P(\text{data as extreme} \mid H_0)$

This is a conditional probability. It tells us about the data given the hypothesis, not about the hypothesis given the data. Confusing these two directions is the most common statistical error.

# Demonstrate how p-values can mislead
# Simulate 100 datasets under the null hypothesis (no effect)
set.seed(123)
p_values <- replicate(100, {
x <- rnorm(50)
y <- rnorm(50)
summary(lm(y ~ x))$coefficients[2, 4]  # p-value for slope
})

cat("Number of 'significant' results (p < 0.05):", sum(p_values < 0.05), "out of 100\n")
cat("Expected by chance: 5 out of 100\n")
cat("This is the false positive rate\n")

hist(p_values, breaks = 20, col = "steelblue", border = "white",
main = "P-values under the null hypothesis",
xlab = "p-value")
abline(v = 0.05, lty = 2, col = "firebrick", lwd = 2)
legend("topright", legend = "p = 0.05", col = "firebrick", lty = 2, lwd = 2)

Click Run to execute this code in your browser with WebR

Under the null hypothesis, about 5% of tests will be "significant" at $p < 0.05$ — purely by chance. This is the false positive rate, and it is a property of the procedure, not of the data. The golem does not know which significant results are real and which are noise. We do.

Rethinking

Models are golems, but modelers are not. The golem analogy is not a counsel of despair. It is a call to responsibility. Statistical models are powerful tools, but they require judgment, context, and scientific understanding to use well. The goal of this book is to teach you how to build better golems — models that are transparent, critiquable, and aligned with scientific questions. The three tools — Bayesian inference, multilevel models, and information criteria — are the engineering practices that make this possible.

What this book does differently

This book takes a different approach from standard statistics textbooks:

Bayesian inference replaces frequentist testing. Instead of $p$ -values, we compute posterior distributions that tell us what to believe about parameters given the data.
Multilevel models replace fixed-effect models. Instead of assuming that all groups are identical or completely independent, we model the variation across groups.
Model comparison replaces hypothesis testing. Instead of testing a single null hypothesis, we compare multiple models and estimate which will predict new data best.

These three tools are not independent. They reinforce each other. Bayesian inference provides the foundation, multilevel models provide the structure, and model comparison provides the critique. Together, they form a coherent workflow for statistical modeling.

Overthinking

Why Bayesian? The Bayesian approach is not just about using priors or computing posteriors. It is a framework for coherent reasoning under uncertainty. The key theorem is Bayes' theorem:

$P(\theta \mid D) = \frac{P(D \mid \theta) \cdot P(\theta)}{P(D)}$

This theorem tells us how to update our beliefs about parameters $\theta$ after seeing data $D$ . The prior $P(\theta)$ encodes what we know before seeing the data. The likelihood $P(D \mid \theta)$ encodes what the data tell us. The posterior $P(\theta \mid D)$ is the result: what we know after seeing the data.

The Bayesian framework is not the only way to do statistics, but it has several advantages:

It produces distributions, not just point estimates, so uncertainty is always explicit.
It allows prior information to be incorporated formally.
It generalizes naturally to hierarchical models and model comparison.
It is coherent: the rules of probability theory ensure that the conclusions are internally consistent.

The main cost is computational: Bayesian inference often requires numerical methods (MCMC) rather than analytical formulas. But with modern computing, this is no longer a barrier.

Summary of Section 1.1

Statistical models are golems: powerful but mindless tools that do what they are told.
The responsibility for using models wisely falls on the modeler, not the model.
Procedural statistics (applying tests mechanically) is the most common source of statistical error.
This book teaches three tools: Bayesian inference, multilevel models, and model comparison.
These tools form a coherent workflow: specify, fit, critique, and compare.

Statistical rethinking