Bayesian Statistics in Environmental Modeling What ...

Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One?

Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Song Qian Department of Environmental Sciences The University of Toledo

June 18, 2018 University of Toronto

Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Outline

1 Introduction 2 Stein’s Paradox and Empirical Bayes 3 Prior as Among Group Distribution 4 Modeling Approach 5 Examples Cryptosporidium Contamination in US Drinking Water Systems Nitrogen Criteria for Small Streams in Ohio Lake Erie Food Web Model 6 Discussion

Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction

Scientists versus Engineers Questions asked: “Why?” versus “How?”


Scientists versus Engineers Questions asked: “Why?” versus “How?” Reasoning mode: induction versus deduction


Scientists versus Engineers Questions asked: “Why?” versus “How?” Reasoning mode: induction versus deduction An uncertain answer versus a definite answer


Scientists versus Engineers Questions asked: “Why?” versus “How?” Reasoning mode: induction versus deduction An uncertain answer versus a definite answer Limited versus “unlimited” chances of experimentation


Two Branches of Statistics Definition of probability: long-run frequency versus degree of belief Neyman and Pearson (1933) – two approaches of testing statistical hypotheses Thomas Bayes – “probabilities a posteriori of the possible causes of a given event” Bertrant and Borel – hypothetical deduction “no test of this kind could give reliable result” useful with properly selected en quelque sorte remarquable character of data

Frequentist’s versus Bayesian statistics


https://xkcd.com/1132/


Bayesian Statistics Compared to the long-run frequency view of statistics, Bayesian – is coherent: everything under one equation p(θ|y) ∝ p(θ)p(y |θ)


Bayesian Statistics Compared to the long-run frequency view of statistics, Bayesian – is coherent: everything under one equation p(θ|y) ∝ p(θ)p(y |θ) is consistent with how we think about chance – “Pr(Trump will bully Trudeau again)” is meaningful


Bayesian Statistics Compared to the long-run frequency view of statistics, Bayesian – is coherent: everything under one equation p(θ|y) ∝ p(θ)p(y |θ) is consistent with how we think about chance – “Pr(Trump will bully Trudeau again)” is meaningful results are easy to interpret – no more p-value, nor “statistical significance”


Bayesian Statistics Compared to the long-run frequency view of statistics, Bayesian – is coherent: everything under one equation p(θ|y) ∝ p(θ)p(y |θ) is consistent with how we think about chance – “Pr(Trump will bully Trudeau again)” is meaningful results are easy to interpret – no more p-value, nor “statistical significance” but needs a prior distribution.


https://xkcd.com/1236/


What is a Prior? The non-normative definition: A prior of an uncertain quantity is the probability distribution that would express one’s beliefs about the quantity – Wikipedia


What is a Prior? The non-normative definition: A prior of an uncertain quantity is the probability distribution that would express one’s beliefs about the quantity – Wikipedia We do not usually define our uncertainty using probability distribution


What is a Prior? The non-normative definition: A prior of an uncertain quantity is the probability distribution that would express one’s beliefs about the quantity – Wikipedia We do not usually define our uncertainty using probability distribution We use probability, but we are bad at getting the probability right (Human judgment relies on heuristics and is biased. Tversky and Kahneman (1974) Science 185:1124-1131).


When Prior Is Known The Bayes estimator is the best with respect to Bayes risk


When Prior Is Known The Bayes estimator is the best with respect to Bayes risk The difficulty used to be computation


When Prior Is Known The Bayes estimator is the best with respect to Bayes risk The difficulty used to be computation MCMC resolved the computation problem


When Prior Is Known The Bayes estimator is the best with respect to Bayes risk The difficulty used to be computation MCMC resolved the computation problem Adrian F.M. Smith (now Sir Adrian Smith) quit statistics because “all the Bayesian problems are solved.”


But How Do We Derive a Prior? The personal belief definition is of no help


But How Do We Derive a Prior? The personal belief definition is of no help There are numerous papers on how to derive a prior that does not contain real information


But How Do We Derive a Prior? The personal belief definition is of no help There are numerous papers on how to derive a prior that does not contain real information Jeffery’s prior – π(µ, σ 2 ) ∝ 1/σ 2


But How Do We Derive a Prior? The personal belief definition is of no help There are numerous papers on how to derive a prior that does not contain real information Jeffery’s prior – π(µ, σ 2 ) ∝ 1/σ 2 Uninformative (flat) prior – N(0, 100), gamma(0.001, 0.001)


But How Do We Derive a Prior? The personal belief definition is of no help There are numerous papers on how to derive a prior that does not contain real information Jeffery’s prior – π(µ, σ 2 ) ∝ 1/σ 2 Uninformative (flat) prior – N(0, 100), gamma(0.001, 0.001) Reference prior –



They are either difficult to derive or informative in someways.



They are either difficult to derive or informative in someways. For example, Gelman (2004)


An Informative Prior is Essencial Prior times likelihood is proportional to the posterior Without the prior, the posterior is just the likelihood Updating is the key

Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Stein’s Paradox and Empirical Bayes

Stein’s Paradox Efron and Morris (1977) “Stein’s Paradox in Statistics” Scientific American A class of shrinkage estimators out perform both MLE and Bayes estimator An empirical Bayes interpretation of Stein’s paradox The batting average example

Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Stein’s Paradox and Empirical Bayes

A Normative Definition of a Prior Prior distribution of an uncertain parameter for a population is the distribution of the same parameter across similar (exchangeable) populations Examples: Prior of a baseball player’s batting average – distribution of batting averages of all players in the same league; Prior of the mean phosphorus concentration of Highland Creek on the Campus of University of Toronto – distribution of mean phosphorus concentrations in all similar sized urban streams in the same region; Prior of next year’s walleye recruitment is the distribution of all previous years recruitment; Expert opinion: a summary of life long observation on the same parameter.

Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Prior as Among Group Distribution

Prior Distribution as an “Among Group” Distribution Group can be spatial, temporal, or organizational; Deriving prior is a process of assemble and analysis of data from similar “groups”

Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Prior as Among Group Distribution

Prior Distribution as an “Among Group” Distribution Group can be spatial, temporal, or organizational; Deriving prior is a process of assemble and analysis of data from similar “groups” Models based on cross-sectional data are basis for informative prior for a specific site

Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Modeling Approach

Outline Bayesian hierarchical models for “simple” models A Network-based modeling approach for complicated models


A Bayesian Network Modeling Approach Establish causal links – a conceptual causal model Develop a graphical model using directed acyclic graphs (DAG) EDA – Isolate empirical models Fit these models to data

Connect individual models using a conditional modeling approach (Gibbs sampler) Updating the model through repeated use of the Bayes theorem


Modeling – from a conceptual model to a DAG A Bayesian Network modeling approach for continuous variables (a) X1

X3

X2

Y1

after observing data

(b) x1

x3

x2

µ1

y1

Y2 µ2

β

σ22

α

y2

σ12


Inference Monte Carlo simulation: Scenario development – changes in predictor variables Propagating through the network to derive distributions of variables of interest Basis for risk assessment and other inferences

Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Cryptosporidium Contamination in US Drinking Water Systems

Cryptosporidium in US Drinking Water Systems Rose, et al (2002) Cryptosporidium – a protozoa that can cause diarrhea disease Oocysts in water → sporozoites in host gut → oocysts → exit host We assume that crypto oocysts are everywhere 1993 Milwaukee cryptosporidiosis outbreak – about 1.61 million residents exposed and 104 deaths attributed to the outbreak Milwaukee outbreak was featured in at least two TV series (Monsters Inside Me and Forensic Files)


Regulations Safe Drinking Water Act (1996 amendment) added microbial contaminants to the list of regulated drinking water contaminants and emphasized risk-based standard setting EPA developed Information Collection Rule (18 month monitoring) and Data Collection and Tracking System (24 month monitoring) Data from ICR and DCTS were intended for developing risk-based standards Water systems with elevated risk are required to use UV disinfection


Data Analysis Challenges Crypto is rare in drinking water – test results are mostly 0s


Data Analysis Challenges Crypto is rare in drinking water – test results are mostly 0s Even present, the current method is far from perfect, adding more 0s


Data Analysis Challenges Crypto is rare in drinking water – test results are mostly 0s Even present, the current method is far from perfect, adding more 0s Certified labs often have different results in standard tests


Data Analysis Challenges Crypto is rare in drinking water – test results are mostly 0s Even present, the current method is far from perfect, adding more 0s Certified labs often have different results in standard tests Results from EPA’s analysis represent national average


A Conceptual Diagram – Spiked Samples j

Cryptojs

λsj

ysj

rj

zj

pj


A Conceptual Diagram – Field Samples ri

pi

Ci

zi

λi

yi

εi

vi


A Conceptual Diagram – Combined

Cryptojs

j

λsj

ysj

rj

zj

pj

zj[i]

Ci

vi

λi

εi

yi


Lab performance assessment 0.25

0.8

0.08

Zero Recovery Rate

Zero Recovery Rate

Recovery Rate

0.20 0.6

0.06

0.15

0.4

0.04

0.10

0.02

0.05

0.2

0.00 0

10

20

30 Labs

40

50

0

10

20

30

40

50

0.20

Labs

Figure: Large between lab variances

0.30

0.40

Recovery Rate

0.50


Geometric means

Crypto Occurrences

0.5 0.1 0.05 0.01 0.005 0.001 0.0005 0.0001 .00005 .00001 0

500 1000 Sampling Points

Figure: Geometric means for each sampling point

1500


Current Practice Data from ICR and DCTS (many systems) were pooled and stratified (by source water type and system size) A Bayesian hierarchical model is used The model is used to determine which systems should be subject to UV treatment requirement


A Bayesian Approach The model developed using nationwide data → priors System-specific risk assessment be conducted using data from the system Developing system-specific model and update over time

Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Nitrogen Criteria for Small Streams in Ohio

From Nutrient to Macroinvertebrates Qian and Miltner (2015) DIN

TP

Chla

DO range

Min DO

EPT

ICI

Habitat Quality


Modeling – The OH cBN DOmin , DOrange

2 σDOR

TP

Canopy

DIN

µDOR

µchla

βDOR ’s

βchla ’s

Chla

βICI ’s

µICI

2 σICI

ICI

QHEI

Ag

µEPT

βEPT ’s

EPT

2 σEPT

2 σchla


Computation EDA for individual model structure


Computation EDA for individual model structure Individual empirical model – prior distribution of the network model


Computation EDA for individual model structure Individual empirical model – prior distribution of the network model Gibbs sampler for the networks model


State-Wide Nitrogen Criteria? 0.4

Density

0.3

0.2

0.1

0.0 0.05

0.5

1

DIN (mg/L)

5

10

25


Streams Are Different 3.0 2.5

DIN (mg/L)

2.0 1.5

5 55

0.

65

0.4

35 0.

0.5

0.1

0.0 5

5

0.2 5

1.0

0. 0.1 0.2

0.0

0.7

80

85

90 QHEI

0.3 0.4 0.75

0.5

0.85

95

0.6

0.8 0.95

0.9

100


Bayesian Updating for Individual Rivers

River 02-500

600

River 02-500 800

500 600

400 300

400

200 200 100 0

0 2.0

2.5

3.0 DIN

3.5

4.0

4.5

30

40

50

60 70 QHEI

80

90

Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Lake Erie Food Web Model

Lake Erie Multiple Gradients Whole lake – East to west Increasing depth Decreasing temperature Decreasing nutrient input and productivity

Local – Nearshore to offshore Increasing depth Decreasing productivity and nutrient levels


USGS 2014 CSMI Study

Understanding food web structure along productivity gradient

Sampling all three basins to capture the west to east gradient Sampling within basin sites to capture the nearshore to offshore gradient


Data Sampling design: 5 transects; 3 distances to shore; 2 replicates, 3 months (May, July, September) Lower trophic sampling of benthic invertebrates, zooplankton, water quality Walleye, yellow perch, and pray fish. Emphasizing the nearshore to offshore gradient


The Conceptual Food Web Structure TP

Zoo

TN

Temp

chla

DO

Visibility

Bentho

Fish


Component Models Regression analysis used to develop functional models connecting the conceptual diagram Chlorophyll a: log(chla) = α0 + α1 log(TP) + α2 log(TN) + α3 Temp + ε Zooplankton: log(Zoo) = β0 + log(chla) + ε √ Visibility: 1/ Secchi = γ0 + γ1 log(chla) + ε DO: DO λ = δ0 + δ1 log(chla) + δ2 Temp + ε Benthic macroinvertebrates: log(Benthos) = a0 + a1 DO λ + ε Fish biomass: log(Fish) √= b0 + b1 log(Zoo) + b2 / Secchi + b3 log(Bentos) + ε


Graphical Presentation The chlorophyll a model TP

TN

βchla ’s

µchla

2 σchla

Chla

Temp


Graphical Presentation Linking chlorophyll a and DO TP

TN

βchla ’s

µchla

Temp

µDO

βDO ’s

2 σDO

2 σchla

Chla

DO


Graphical Presentation The full model TP

TN

βchla ’s

µchla

Temp

µDO

βDO ’s

2 σDO

2 σchla

Chla

βZOO ’s

µZOO ’s

2 σZOO ’s

ZOO

DO

µVis

βVis ’s

µBen ’s

βBen ’s

ben

Vis

2 ’s σBen 2 ’s σVis

Fish


Simulation for model evaluation Total Fish Biomass

Model

Data

0.01

0.1

1

10

100

Total Fish Biomass

1000


Simulation for model evaluation Yellow Perch composition

Model

Data

0.01

0.1

1

10

100 1000

Yellow Perch Biomass


Monte Carlo Simulation for Inference – Total Fish Biomass Effect of nutrient loading reduction (40% reduction in TP)

Reduced P

Model

Data

0.01

0.1

1

10

100 1000



Monte Carlo Simulation for Inference – Y Perch Biomass Effect of nutrient loading reduction (40% reduction in TP)

Reduced P

Model

Data

0.01

0.1

1

10

100 1000



Bayesian Updating The current model is based on a small data set with limited spatial and temporal coverage Uncertainty in model coefficients are high Model has limited spatial and temporal specificity

But the model is Bayesian – it can be used as a prior for sampling design combining spatially focused monitoring data from elsewhere


Bayesian Updating Updating is not limited to similar data collection methods Updating can be sequential – we can gradually refine the model Future development should focus on regional models for the three sub-basins Combining data from multiple sources to include unique seasonal patterns in each regional model Maintain a robust sampling program for model updating over time – Improving the model detecting temporal changes

Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Discussion

Summary Bayesian method is always better, provided we have the right prior


Summary Bayesian method is always better, provided we have the right prior James-Stein estimator and empirical Bayes – prior distribution is “among-group” distribution


Summary Bayesian method is always better, provided we have the right prior James-Stein estimator and empirical Bayes – prior distribution is “among-group” distribution The Bayesian hierarchical modeling


Summary Bayesian method is always better, provided we have the right prior James-Stein estimator and empirical Bayes – prior distribution is “among-group” distribution The Bayesian hierarchical modeling Models based on cross-sectional data should be considered as prior models


Summary Bayesian method is always better, provided we have the right prior James-Stein estimator and empirical Bayes – prior distribution is “among-group” distribution The Bayesian hierarchical modeling Models based on cross-sectional data should be considered as prior models Inference about individual “group” should be based on the posterior derived using group-specific data


cBN – A modeling framework A network-based modeling approach enabled by the Gibbs sampler Component models can be empirical and/or mechanistic – a framework for combining information from multiple sources A flexible tool for analyzing cross-sectional data Bayesian prior for “group”-specific model


A beginning, rather than the end Developing a prior – the first step of a modeling work Group-specific inference – posterior A practice dictated by the Simpson’s paradox


Acknowledgment USGS for funding Craig Stow, Laura Steinberg, Mike Messner, Bob Miltner, Casey Yanos, Chris Mayer, Mark Rogers MSU Quantitative Fisheries Center