3.4 Summary

Chapter 3 in review

Chapter 3 showed that sampling is the universal solvent of Bayesian computation. Once we can draw samples from the posterior, every question about parameters becomes an empirical question: compute the function on the sample and take the average.

We learned three uses of sampling:

1. Sampling to summarize (Section 3.2): Compute credible intervals, posterior means, medians, modes, and posterior probabilities from samples. The percentile interval and the HPD interval are two ways to summarize uncertainty. Point estimates (mean, median, mode) are useful but discard information.

2. Sampling to simulate prediction (Section 3.3): Draw from the posterior predictive distribution by sampling parameters and then simulating new data. This lets us check model fit and forecast future observations.

3. Sampling as the foundation of MCMC (Chapter 8): Markov Chain Monte Carlo produces samples from any posterior, no matter how complex. Once we have samples, the rest is arithmetic.

Key formulas from Chapter 3

• Monte Carlo estimate: $\hat{E}[f(\theta)] \approx \frac{1}{S} \sum_{s=1}^{S} f(\theta_s)$
• Percentile interval: $[\hat{\theta}_{\alpha/2}, \hat{\theta}_{1-\alpha/2}]$ from posterior quantiles
• Posterior probability: $P(\theta \in A \mid D) \approx \frac{1}{S} \sum \mathbf{1}[\theta_s \in A]$
• Posterior predictive: $p(y_\text{new} \mid D) = \int p(y_\text{new} \mid \theta) \, p(\theta \mid D) \, d\theta$