Bayesian Basics: A conceptual introduction with application in R and ...

MICHAEL CLARK C E N T E R F O R S TAT I S T I C A L C O N S U LTAT I O N A N D R E S E A R C H UNIVERSITY OF MICHIGAN

B AY E S I A N B A S I C S A C O N C E P T U A L I N T R O D U C T I O N W I T H A P P L I C AT I O N I N R A N D S TA N

Bayesian Basics

Contents

5

Preface

6

Introduction

7

Bayesian Probability

Conditional probability & Bayes theorem

8

A Hands-on Example

Prior, likelihood, & posterior distributions Prior

7

8

9 10

Likelihood

10

Posterior

Posterior predictive distribution

12

Regression Models

Example: Linear Regression Model Setup

11

12

13

Stan Code

15 16

Running the Model

Model Checking & Diagnostics

20

20

Monitoring Convergence

Visual Inspection: Traceplot & Densities 21

Statistical Measures 21

Autocorrelation

Model Checking

20

22

Sensitivity Analysis

22

Predictive Accuracy & Model Comparison

22

Posterior Predictive Checking: Statistical

23

Posterior Predictive Checking: Graphical

24

2

3

Bayesian Basics

25

Summary

25

Model Enhancements

Generating New Variables of Interest

25

27

Robust Regression

28

Generalized Linear Model

28

Issues

29

Debugging

30

Choice of Prior

Noninformative, Weakly Informative, Informative Conjugacy

31 32

Sensitivity Analysis 32

Summary

32

Sampling Procedure Metropolis Gibbs

32

33

Hamiltonian Monte Carlo

34

Other Variations and Approximate Methods

Number of draws, thinning, warm-up 35

Model Complexity

Summary

35

Appendix

37

Maximum Likelihood Review Example Linear Model

38 40

37

34

34

30

Bayesian Basics

Binomial Likelihood Example Modeling Languages Bugs

43

JAGS

43

Stan

43

R

42

43

43

General Statistical Package Implementations 44

Other Programming Languages Summary

44

BUGS Example

45

JAGS Example

47

Metropolis Hastings Example

49

Hamiltonian Monte Carlo Example

53

43

4

5

Bayesian Basics

Preface The following serves as a practical and applied introduction to Bayesian estimation methods for the uninitiated. The goal is to provide just enough information in a brief format to allow one to feel comfortable exploring Bayesian , lower=c(sigma=0)) mlnorm ## ## ## ## ## ## ## ## ##

42

An alternative approach would be to work with the log of σ which can take on negative values, and then convert it back to the original scale.

Call: mle2(minuslogl = LL, method = "L-BFGS-B", lower = c(sigma = 0)) Coefficients: mu sigma 4.946809 1.993676 Log-likelihood: -2108.92

# compare to an intercept only regression model summary(lm(y~1)) ## ## ## ## ## ## ## ## ## ## ## ## ## ## ##

Call: lm(formula = y ~ 1) Residuals: Min 1Q Median -6.7389 -1.2933 -0.0264

3Q 1.2848

Max 6.4450

Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 4.94681 0.06308 78.42 |t|) (Intercept) 5.03998 0.07751 65.02