9.2 Generalized linear models

Generalized linear models (GLMs) extend the linear model to non-Gaussian outcomes by adding a link function. The link function transforms the linear predictor into the natural scale of the outcome.

9.2.1 The GLM framework

A GLM has three components:

Likelihood: The outcome distribution from the exponential family (Gaussian, binomial, Poisson, etc.).
Linear predictor: $\eta_i = \alpha + \beta x_i$ (same as before).
Link function: $g(\mu_i) = \eta_i$ , where $\mu_i = E[y_i]$ .

$g(\mathbb{E}[y_i]) = \alpha + \beta x_i$

The link function $g$ maps from the parameter's natural scale to the real line:

| Outcome | Likelihood | Link | Name | | ------------------- | ---------- | -------------- | ------------------- | | Continuous | Gaussian | Identity | Linear regression | | Binary | Binomial | Logit | Logistic regression | | Count | Poisson | Log | Poisson regression | | Positive continuous | Gamma | Log or inverse | Gamma regression |

# Demonstrate different GLM link functions
set.seed(42)
n <- 200
x <- runif(n, 0, 10)

# 1. Gaussian (identity link)

y_gauss <- 2 + 1.5 * x + rnorm(n, sd = 2)
m_gauss <- lm(y_gauss ~ x)

# 2. Binomial (logit link)

p_true <- 1 / (1 + exp(-(0 + 0.5 * x)))
y_bin <- rbinom(n, 1, p_true)
m_logistic <- glm(y_bin ~ x, family = binomial)

# 3. Poisson (log link)

lambda_true <- exp(0.2 + 0.15 * x)
y_pois <- rpois(n, lambda_true)
m_pois <- glm(y_pois ~ x, family = poisson)

cat("Gaussian (identity link):\n")
cat(" Intercept:", round(coef(m_gauss)[1], 3), " Slope:", round(coef(m_gauss)[2], 3), "\n")

cat("\nBinomial (logit link):\n")
cat(" Intercept:", round(coef(m_logistic)[1], 3), " Slope:", round(coef(m_logistic)[2], 3), "\n")
cat(" (on logit scale; use plogis() for probabilities)\n")

cat("\nPoisson (log link):\n")
cat(" Intercept:", round(coef(m_pois)[1], 3), " Slope:", round(coef(m_pois)[2], 3), "\n")
cat(" (on log scale; use exp() for counts)\n")

Click Run to execute this code in your browser with WebR

9.2.2 The logit link (logistic regression)

For binary outcomes, the logit link maps probabilities to the real line:

$\text{logit}(p_i) = \log\frac{p_i}{1 - p_i} = \alpha + \beta x_i$

The inverse is the logistic function:

$p_i = \text{logistic}(\alpha + \beta x_i) = \frac{1}{1 + e^{-(\alpha + \beta x_i)}}$

# Demonstrate logistic regression
set.seed(42)
n <- 200
x <- runif(n, 0, 10)

# True relationship

p_true <- 1 / (1 + exp(-(-2 + 0.5 * x)))
y <- rbinom(n, 1, p_true)

# Fit logistic regression

m <- glm(y ~ x, family = binomial)

cat("Logistic regression:\n")
cat(" Intercept:", round(coef(m)[1], 3), "\n")
cat(" Slope:", round(coef(m)[2], 3), "\n")

# Convert to probability scale

cat("\nPredicted probabilities at x = 0, 5, 10:\n")
for (x_val in c(0, 5, 10)) {
logit_p <- coef(m)[1] + coef(m)[2] * x_val
p_val <- 1 / (1 + exp(-logit_p))
cat(sprintf(" x = %2d: p = %.3f\n", x_val, p_val))
}

# Plot

x_seq <- seq(0, 10, length.out = 200)
logit_pred <- coef(m)[1] + coef(m)[2] * x_seq
p_pred <- 1 / (1 + exp(-logit_pred))

plot(x, y, pch = 16, col = "steelblue",
main = "Logistic regression: binary outcome",
xlab = "x", ylab = "p(y = 1)")
lines(x_seq, p_pred, lwd = 2, col = "firebrick")
abline(h = 0.5, lty = 2, col = "gray50")

Click Run to execute this code in your browser with WebR

9.2.3 The log link (Poisson regression)

For count outcomes, the log link ensures the expected count is always positive:

$\log(\lambda_i) = \alpha + \beta x_i$

$\lambda_i = e^{\alpha + \beta x_i}$

# Demonstrate Poisson regression
set.seed(42)
n <- 200
x <- runif(n, 0, 10)

# True relationship: lambda = exp(0.5 + 0.2 * x)

lambda_true <- exp(0.5 + 0.2 * x)
y <- rpois(n, lambda_true)

# Fit Poisson regression

m <- glm(y ~ x, family = poisson)

cat("Poisson regression:\n")
cat(" Intercept:", round(coef(m)[1], 3), "\n")
cat(" Slope:", round(coef(m)[2], 3), "\n")

# Convert to count scale

cat("\nPredicted counts at x = 0, 5, 10:\n")
for (x_val in c(0, 5, 10)) {
log_lambda <- coef(m)[1] + coef(m)[2] * x_val
cat(sprintf(" x = %2d: lambda = %.3f\n", x_val, exp(log_lambda)))
}

# Plot

x_seq <- seq(0, 10, length.out = 200)
log_pred <- coef(m)[1] + coef(m)[2] * x_seq
lambda_pred <- exp(log_pred)

plot(x, y, pch = 16, col = "steelblue",
main = "Poisson regression: count outcome",
xlab = "x", ylab = "y (count)")
lines(x_seq, lambda_pred, lwd = 2, col = "firebrick")

Click Run to execute this code in your browser with WebR

Rethinking

The link function is the bridge. The linear model lives on the real line — it can produce any value from $-\infty$ to $+\infty$ . But outcomes live on different scales: probabilities are in $[0, 1]$ , counts are in $[0, \infty)$ . The link function bridges these scales. It transforms the linear predictor to the outcome's natural scale. This is not a hack — it is the natural consequence of choosing the maximum entropy likelihood and then connecting it to the linear model. The link function is what makes GLMs general.

Summary of Section 9.2

GLMs extend linear models with a link function: $g(E[y_i]) = \alpha + \beta x_i$ .
Logit link for binary outcomes: $\log(p/(1-p)) = \alpha + \beta x$ .
Log link for count outcomes: $\log(\lambda) = \alpha + \beta x$ .
Identity link for continuous outcomes: $\mu = \alpha + \beta x$ (ordinary linear regression).
The link function bridges the linear predictor to the outcome's natural scale.

Maximum entropy

Maximum entropy priors