Jun 18, 2018 - Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Outline. 1 Introduction. 2 Stein's Paradox and Empirical ...
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One?
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Song Qian Department of Environmental Sciences The University of Toledo
June 18, 2018 University of Toronto
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Outline
1 Introduction 2 Stein’s Paradox and Empirical Bayes 3 Prior as Among Group Distribution 4 Modeling Approach 5 Examples Cryptosporidium Contamination in US Drinking Water Systems Nitrogen Criteria for Small Streams in Ohio Lake Erie Food Web Model 6 Discussion
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
Scientists versus Engineers Questions asked: “Why?” versus “How?”
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
Scientists versus Engineers Questions asked: “Why?” versus “How?” Reasoning mode: induction versus deduction
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
Scientists versus Engineers Questions asked: “Why?” versus “How?” Reasoning mode: induction versus deduction An uncertain answer versus a definite answer
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
Scientists versus Engineers Questions asked: “Why?” versus “How?” Reasoning mode: induction versus deduction An uncertain answer versus a definite answer Limited versus “unlimited” chances of experimentation
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
Two Branches of Statistics Definition of probability: long-run frequency versus degree of belief Neyman and Pearson (1933) – two approaches of testing statistical hypotheses Thomas Bayes – “probabilities a posteriori of the possible causes of a given event” Bertrant and Borel – hypothetical deduction “no test of this kind could give reliable result” useful with properly selected en quelque sorte remarquable character of data
Frequentist’s versus Bayesian statistics
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
https://xkcd.com/1132/
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
Bayesian Statistics Compared to the long-run frequency view of statistics, Bayesian – is coherent: everything under one equation p(θ|y) ∝ p(θ)p(y |θ)
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
Bayesian Statistics Compared to the long-run frequency view of statistics, Bayesian – is coherent: everything under one equation p(θ|y) ∝ p(θ)p(y |θ) is consistent with how we think about chance – “Pr(Trump will bully Trudeau again)” is meaningful
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
Bayesian Statistics Compared to the long-run frequency view of statistics, Bayesian – is coherent: everything under one equation p(θ|y) ∝ p(θ)p(y |θ) is consistent with how we think about chance – “Pr(Trump will bully Trudeau again)” is meaningful results are easy to interpret – no more p-value, nor “statistical significance”
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
Bayesian Statistics Compared to the long-run frequency view of statistics, Bayesian – is coherent: everything under one equation p(θ|y) ∝ p(θ)p(y |θ) is consistent with how we think about chance – “Pr(Trump will bully Trudeau again)” is meaningful results are easy to interpret – no more p-value, nor “statistical significance” but needs a prior distribution.
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
https://xkcd.com/1236/
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
What is a Prior? The non-normative definition: A prior of an uncertain quantity is the probability distribution that would express one’s beliefs about the quantity – Wikipedia
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
What is a Prior? The non-normative definition: A prior of an uncertain quantity is the probability distribution that would express one’s beliefs about the quantity – Wikipedia We do not usually define our uncertainty using probability distribution
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
What is a Prior? The non-normative definition: A prior of an uncertain quantity is the probability distribution that would express one’s beliefs about the quantity – Wikipedia We do not usually define our uncertainty using probability distribution We use probability, but we are bad at getting the probability right (Human judgment relies on heuristics and is biased. Tversky and Kahneman (1974) Science 185:1124-1131).
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
When Prior Is Known The Bayes estimator is the best with respect to Bayes risk
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
When Prior Is Known The Bayes estimator is the best with respect to Bayes risk The difficulty used to be computation
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
When Prior Is Known The Bayes estimator is the best with respect to Bayes risk The difficulty used to be computation MCMC resolved the computation problem
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
When Prior Is Known The Bayes estimator is the best with respect to Bayes risk The difficulty used to be computation MCMC resolved the computation problem Adrian F.M. Smith (now Sir Adrian Smith) quit statistics because “all the Bayesian problems are solved.”
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
But How Do We Derive a Prior? The personal belief definition is of no help
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
But How Do We Derive a Prior? The personal belief definition is of no help There are numerous papers on how to derive a prior that does not contain real information
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
But How Do We Derive a Prior? The personal belief definition is of no help There are numerous papers on how to derive a prior that does not contain real information Jeffery’s prior – π(µ, σ 2 ) ∝ 1/σ 2
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
But How Do We Derive a Prior? The personal belief definition is of no help There are numerous papers on how to derive a prior that does not contain real information Jeffery’s prior – π(µ, σ 2 ) ∝ 1/σ 2 Uninformative (flat) prior – N(0, 100), gamma(0.001, 0.001)
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
But How Do We Derive a Prior? The personal belief definition is of no help There are numerous papers on how to derive a prior that does not contain real information Jeffery’s prior – π(µ, σ 2 ) ∝ 1/σ 2 Uninformative (flat) prior – N(0, 100), gamma(0.001, 0.001) Reference prior –
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
But How Do We Derive a Prior? The personal belief definition is of no help There are numerous papers on how to derive a prior that does not contain real information Jeffery’s prior – π(µ, σ 2 ) ∝ 1/σ 2 Uninformative (flat) prior – N(0, 100), gamma(0.001, 0.001) Reference prior –
They are either difficult to derive or informative in someways.
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
But How Do We Derive a Prior? The personal belief definition is of no help There are numerous papers on how to derive a prior that does not contain real information Jeffery’s prior – π(µ, σ 2 ) ∝ 1/σ 2 Uninformative (flat) prior – N(0, 100), gamma(0.001, 0.001) Reference prior –
They are either difficult to derive or informative in someways. For example, Gelman (2004)
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Introduction
An Informative Prior is Essencial Prior times likelihood is proportional to the posterior Without the prior, the posterior is just the likelihood Updating is the key
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Stein’s Paradox and Empirical Bayes
Stein’s Paradox Efron and Morris (1977) “Stein’s Paradox in Statistics” Scientific American A class of shrinkage estimators out perform both MLE and Bayes estimator An empirical Bayes interpretation of Stein’s paradox The batting average example
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Stein’s Paradox and Empirical Bayes
A Normative Definition of a Prior Prior distribution of an uncertain parameter for a population is the distribution of the same parameter across similar (exchangeable) populations Examples: Prior of a baseball player’s batting average – distribution of batting averages of all players in the same league; Prior of the mean phosphorus concentration of Highland Creek on the Campus of University of Toronto – distribution of mean phosphorus concentrations in all similar sized urban streams in the same region; Prior of next year’s walleye recruitment is the distribution of all previous years recruitment; Expert opinion: a summary of life long observation on the same parameter.
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Prior as Among Group Distribution
Prior Distribution as an “Among Group” Distribution Group can be spatial, temporal, or organizational; Deriving prior is a process of assemble and analysis of data from similar “groups”
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Prior as Among Group Distribution
Prior Distribution as an “Among Group” Distribution Group can be spatial, temporal, or organizational; Deriving prior is a process of assemble and analysis of data from similar “groups” Models based on cross-sectional data are basis for informative prior for a specific site
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Modeling Approach
Outline Bayesian hierarchical models for “simple” models A Network-based modeling approach for complicated models
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Modeling Approach
A Bayesian Network Modeling Approach Establish causal links – a conceptual causal model Develop a graphical model using directed acyclic graphs (DAG) EDA – Isolate empirical models Fit these models to data
Connect individual models using a conditional modeling approach (Gibbs sampler) Updating the model through repeated use of the Bayes theorem
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Modeling Approach
Modeling – from a conceptual model to a DAG A Bayesian Network modeling approach for continuous variables (a) X1
X3
X2
Y1
after observing data
(b) x1
x3
x2
µ1
y1
Y2 µ2
β
σ22
α
y2
σ12
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Modeling Approach
Inference Monte Carlo simulation: Scenario development – changes in predictor variables Propagating through the network to derive distributions of variables of interest Basis for risk assessment and other inferences
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Cryptosporidium Contamination in US Drinking Water Systems
Cryptosporidium in US Drinking Water Systems Rose, et al (2002) Cryptosporidium – a protozoa that can cause diarrhea disease Oocysts in water → sporozoites in host gut → oocysts → exit host We assume that crypto oocysts are everywhere 1993 Milwaukee cryptosporidiosis outbreak – about 1.61 million residents exposed and 104 deaths attributed to the outbreak Milwaukee outbreak was featured in at least two TV series (Monsters Inside Me and Forensic Files)
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Cryptosporidium Contamination in US Drinking Water Systems
Regulations Safe Drinking Water Act (1996 amendment) added microbial contaminants to the list of regulated drinking water contaminants and emphasized risk-based standard setting EPA developed Information Collection Rule (18 month monitoring) and Data Collection and Tracking System (24 month monitoring) Data from ICR and DCTS were intended for developing risk-based standards Water systems with elevated risk are required to use UV disinfection
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Cryptosporidium Contamination in US Drinking Water Systems
Data Analysis Challenges Crypto is rare in drinking water – test results are mostly 0s
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Cryptosporidium Contamination in US Drinking Water Systems
Data Analysis Challenges Crypto is rare in drinking water – test results are mostly 0s Even present, the current method is far from perfect, adding more 0s
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Cryptosporidium Contamination in US Drinking Water Systems
Data Analysis Challenges Crypto is rare in drinking water – test results are mostly 0s Even present, the current method is far from perfect, adding more 0s Certified labs often have different results in standard tests
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Cryptosporidium Contamination in US Drinking Water Systems
Data Analysis Challenges Crypto is rare in drinking water – test results are mostly 0s Even present, the current method is far from perfect, adding more 0s Certified labs often have different results in standard tests Results from EPA’s analysis represent national average
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Cryptosporidium Contamination in US Drinking Water Systems
A Conceptual Diagram – Spiked Samples j
Cryptojs
λsj
ysj
rj
zj
pj
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Cryptosporidium Contamination in US Drinking Water Systems
A Conceptual Diagram – Field Samples ri
pi
Ci
zi
λi
yi
εi
vi
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Cryptosporidium Contamination in US Drinking Water Systems
A Conceptual Diagram – Combined
Cryptojs
j
λsj
ysj
rj
zj
pj
zj[i]
Ci
vi
λi
εi
yi
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Cryptosporidium Contamination in US Drinking Water Systems
Lab performance assessment 0.25
0.8
0.08
Zero Recovery Rate
Zero Recovery Rate
Recovery Rate
0.20 0.6
0.06
0.15
0.4
0.04
0.10
0.02
0.05
0.2
0.00 0
10
20
30 Labs
40
50
0
10
20
30
40
50
0.20
Labs
Figure: Large between lab variances
0.30
0.40
Recovery Rate
0.50
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Cryptosporidium Contamination in US Drinking Water Systems
Geometric means
Crypto Occurrences
0.5 0.1 0.05 0.01 0.005 0.001 0.0005 0.0001 .00005 .00001 0
500 1000 Sampling Points
Figure: Geometric means for each sampling point
1500
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Cryptosporidium Contamination in US Drinking Water Systems
Current Practice Data from ICR and DCTS (many systems) were pooled and stratified (by source water type and system size) A Bayesian hierarchical model is used The model is used to determine which systems should be subject to UV treatment requirement
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Cryptosporidium Contamination in US Drinking Water Systems
A Bayesian Approach The model developed using nationwide data → priors System-specific risk assessment be conducted using data from the system Developing system-specific model and update over time
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Nitrogen Criteria for Small Streams in Ohio
From Nutrient to Macroinvertebrates Qian and Miltner (2015) DIN
TP
Chla
DO range
Min DO
EPT
ICI
Habitat Quality
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Nitrogen Criteria for Small Streams in Ohio
Modeling – The OH cBN DOmin , DOrange
2 σDOR
TP
Canopy
DIN
µDOR
µchla
βDOR ’s
βchla ’s
Chla
βICI ’s
µICI
2 σICI
ICI
QHEI
Ag
µEPT
βEPT ’s
EPT
2 σEPT
2 σchla
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Nitrogen Criteria for Small Streams in Ohio
Computation EDA for individual model structure
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Nitrogen Criteria for Small Streams in Ohio
Computation EDA for individual model structure Individual empirical model – prior distribution of the network model
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Nitrogen Criteria for Small Streams in Ohio
Computation EDA for individual model structure Individual empirical model – prior distribution of the network model Gibbs sampler for the networks model
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Nitrogen Criteria for Small Streams in Ohio
State-Wide Nitrogen Criteria? 0.4
Density
0.3
0.2
0.1
0.0 0.05
0.5
1
DIN (mg/L)
5
10
25
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Nitrogen Criteria for Small Streams in Ohio
Streams Are Different 3.0 2.5
DIN (mg/L)
2.0 1.5
5 55
0.
65
0.4
35 0.
0.5
0.1
0.0 5
5
0.2 5
1.0
0. 0.1 0.2
0.0
0.7
80
85
90 QHEI
0.3 0.4 0.75
0.5
0.85
95
0.6
0.8 0.95
0.9
100
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Nitrogen Criteria for Small Streams in Ohio
Bayesian Updating for Individual Rivers
River 02-500
600
River 02-500 800
500 600
400 300
400
200 200 100 0
0 2.0
2.5
3.0 DIN
3.5
4.0
4.5
30
40
50
60 70 QHEI
80
90
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Lake Erie Food Web Model
Lake Erie Multiple Gradients Whole lake – East to west Increasing depth Decreasing temperature Decreasing nutrient input and productivity
Local – Nearshore to offshore Increasing depth Decreasing productivity and nutrient levels
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Lake Erie Food Web Model
USGS 2014 CSMI Study
Understanding food web structure along productivity gradient
Sampling all three basins to capture the west to east gradient Sampling within basin sites to capture the nearshore to offshore gradient
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Lake Erie Food Web Model
Data Sampling design: 5 transects; 3 distances to shore; 2 replicates, 3 months (May, July, September) Lower trophic sampling of benthic invertebrates, zooplankton, water quality Walleye, yellow perch, and pray fish. Emphasizing the nearshore to offshore gradient
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Lake Erie Food Web Model
The Conceptual Food Web Structure TP
Zoo
TN
Temp
chla
DO
Visibility
Bentho
Fish
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Lake Erie Food Web Model
Component Models Regression analysis used to develop functional models connecting the conceptual diagram Chlorophyll a: log(chla) = α0 + α1 log(TP) + α2 log(TN) + α3 Temp + ε Zooplankton: log(Zoo) = β0 + log(chla) + ε √ Visibility: 1/ Secchi = γ0 + γ1 log(chla) + ε DO: DO λ = δ0 + δ1 log(chla) + δ2 Temp + ε Benthic macroinvertebrates: log(Benthos) = a0 + a1 DO λ + ε Fish biomass: log(Fish) √= b0 + b1 log(Zoo) + b2 / Secchi + b3 log(Bentos) + ε
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Lake Erie Food Web Model
Graphical Presentation The chlorophyll a model TP
TN
βchla ’s
µchla
2 σchla
Chla
Temp
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Lake Erie Food Web Model
Graphical Presentation Linking chlorophyll a and DO TP
TN
βchla ’s
µchla
Temp
µDO
βDO ’s
2 σDO
2 σchla
Chla
DO
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Lake Erie Food Web Model
Graphical Presentation The full model TP
TN
βchla ’s
µchla
Temp
µDO
βDO ’s
2 σDO
2 σchla
Chla
βZOO ’s
µZOO ’s
2 σZOO ’s
ZOO
DO
µVis
βVis ’s
µBen ’s
βBen ’s
ben
Vis
2 ’s σBen 2 ’s σVis
Fish
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Lake Erie Food Web Model
Simulation for model evaluation Total Fish Biomass
Model
Data
0.01
0.1
1
10
100
Total Fish Biomass
1000
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Lake Erie Food Web Model
Simulation for model evaluation Yellow Perch composition
Model
Data
0.01
0.1
1
10
100 1000
Yellow Perch Biomass
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Lake Erie Food Web Model
Monte Carlo Simulation for Inference – Total Fish Biomass Effect of nutrient loading reduction (40% reduction in TP)
Reduced P
Model
Data
0.01
0.1
1
10
100 1000
Yellow Perch Biomass
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Lake Erie Food Web Model
Monte Carlo Simulation for Inference – Y Perch Biomass Effect of nutrient loading reduction (40% reduction in TP)
Reduced P
Model
Data
0.01
0.1
1
10
100 1000
Yellow Perch Biomass
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Lake Erie Food Web Model
Bayesian Updating The current model is based on a small data set with limited spatial and temporal coverage Uncertainty in model coefficients are high Model has limited spatial and temporal specificity
But the model is Bayesian – it can be used as a prior for sampling design combining spatially focused monitoring data from elsewhere
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Examples Lake Erie Food Web Model
Bayesian Updating Updating is not limited to similar data collection methods Updating can be sequential – we can gradually refine the model Future development should focus on regional models for the three sub-basins Combining data from multiple sources to include unique seasonal patterns in each regional model Maintain a robust sampling program for model updating over time – Improving the model detecting temporal changes
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Discussion
Summary Bayesian method is always better, provided we have the right prior
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Discussion
Summary Bayesian method is always better, provided we have the right prior James-Stein estimator and empirical Bayes – prior distribution is “among-group” distribution
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Discussion
Summary Bayesian method is always better, provided we have the right prior James-Stein estimator and empirical Bayes – prior distribution is “among-group” distribution The Bayesian hierarchical modeling
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Discussion
Summary Bayesian method is always better, provided we have the right prior James-Stein estimator and empirical Bayes – prior distribution is “among-group” distribution The Bayesian hierarchical modeling Models based on cross-sectional data should be considered as prior models
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Discussion
Summary Bayesian method is always better, provided we have the right prior James-Stein estimator and empirical Bayes – prior distribution is “among-group” distribution The Bayesian hierarchical modeling Models based on cross-sectional data should be considered as prior models Inference about individual “group” should be based on the posterior derived using group-specific data
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Discussion
cBN – A modeling framework A network-based modeling approach enabled by the Gibbs sampler Component models can be empirical and/or mechanistic – a framework for combining information from multiple sources A flexible tool for analyzing cross-sectional data Bayesian prior for “group”-specific model
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Discussion
A beginning, rather than the end Developing a prior – the first step of a modeling work Group-specific inference – posterior A practice dictated by the Simpson’s paradox
Bayesian Statistics in Environmental Modeling What Is a Prior and How to Derive One? Discussion
Acknowledgment USGS for funding Craig Stow, Laura Steinberg, Mike Messner, Bob Miltner, Casey Yanos, Chris Mayer, Mark Rogers MSU Quantitative Fisheries Center
