Adaptive Spatially-varying Variance Inflation in an Ensemble Filter

Adaptive Spatially-varying Variance Inflation in an Ensemble Filter Kevin Raeder, Jeff Anderson, Tim Hoar, Nancy Collins, Hui Liu NCAR Data Assimilation Research Section (DAReS)

Raeder et al.: AGU Spring 2007

1

5/18/07

Assimilation System Data Assimilation Research Testbed

Forecast Model Community Atmosphere Model

20-member ensemble filter Sondes, ACARS, Drift Winds 0.2 Gaspari-Cohn Localization

T85L26 Climatological I.C.s January, 2003

Raeder et al.: AGU Spring 2007

2

5/18/07

Six-hour Forecast RMS Error and Spread: No Inflation Observation Space for 500 hPa North America ACARS Temperatures. mean w/o inflation 1.5791 2.5

Spread (dashed) gets too small.

2

Confidence in prior too great.

1.5

Observations given too little weight.

1

0.5

0

01/04

01/09

Raeder et al.: AGU Spring 2007

01/14

01/19

01/24

3

01/29

RMS error (solid) grows after a few days.

5/18/07

Sources of insufficient prior spread. 1. Model error. 2. Ensemble sampling error.

*

* * * *

Spread

* * Posterior *

*

Expected RMS

t1

1

Correct Prior

Expected |Sample Correlation|

t2

Model Prior

0.6 0.4 0.2 0 0

10 Members 20 Members 40 Members 80 Members 0.5 True Correlation

1

Expected value of |sample correlation| vs. true correlation is too large for small ensemble sizes.

Prior model spread doesn’t account for errors (including representativeness). Result can be treated as lack of spread. Models have too little error growth, too.

Raeder et al.: AGU Spring 2007

0.8

Observations systematically reduce spread of state variables too much. 4

5/18/07

One Solution: Prior State Space Inflation Prior State Ensemble

Mean

Inflated Prior Ensemble

*

*

Linearly expand around mean. Spread is increased. Mean is unchanged. Correlation with other inflated state variables unchanged.

Applied independently to each state variable.

Raeder et al.: AGU Spring 2007

5

5/18/07

Spatially-varying temporally-adaptive inflation: Hierarchical Bayesian Prior Obs. Space Estimate

Observed Value and Likelihood from Instrument

Obs. Apply h Forward Obs. Operator

h

h

* Model Advance

Raeder et al.: AGU Spring 2007

Is spread smaller than expected? Yes: increase inflation. No: decrease inflation.

* * * * * Inflate Prior 6

Use joint prior of obs. and each state variable to regress inflation increment.

Each state variable has its own inflation distribution (Gaussian).

5/18/07

Adaptive Inflation Applied to DART/CAM January, 2003 Observation Space for 500 hPa North America ACARS Temperatures. Fewer observations rejected. (If prior error is more than 3 times expected, obs. are rejected). RMS Error is Reduced. RMS does not increase with time. Spread is increased.

Raeder et al.: AGU Spring 2007

7

5/18/07

RMS Error Reduced in General: Number of Obs. Rejected Reduced

Obs. Space North America ACARS Temperatures vertical profile. Raeder et al.: AGU Spring 2007

8

5/18/07

Spatial and Temporal Structure of Adaptive Inflation Zonal Wind Inflation, 266 hPa

Raeder et al.: AGU Spring 2007

9

5/18/07

Why Does Inflation Get so BIG? Correct Evolution

t2

Observed Value

t3

t1

Spread

Model Evolution

Assimilating Dense Obs. Leaves Tiny Spread. Raeder et al.: AGU Spring 2007

10

Expected RMS

*

Spread still small after advance. Big inflation needed to handle model error. 5/18/07

Conclusions (PLUS SEE OUR POSTER: A31B-06) 1. Ensemble Assimilations have too little variance. 2. Inflation can correct for this. 3. Hierarchical Bayesian algorithm automatically gives good inflation. 4. Applied to CAM, WRF, other large models without tuning. 5. Error is reduced, spread increased. 6. Available as part of DART from www.image.ucar.edu/DAReS/.

Raeder et al.: AGU Spring 2007

11

5/18/07

After 6 hours. NCEP

CAM starts with climatology! Nearly zonal.

DART/CAM

Difference.

Raeder et al.: AGU Spring 2007

12

5/18/07

After 1 day. NCEP

DART/CAM

Difference.

Raeder et al.: AGU Spring 2007

13

5/18/07

After 3 days. NCEP

CAM gains zonal structure.

DART/CAM

Difference.

Raeder et al.: AGU Spring 2007

14

5/18/07

After 7 days. NCEP

NH converged. SH poorly observed.

DART/CAM

Difference.

Raeder et al.: AGU Spring 2007

15

5/18/07

6-Hour Forecast and Analysis Observation Space Temperature RMS RMS Error: Tropics

RMS Error: Northern Hemisphere

100

100

200

200

NCEP Opnl. Analysis

300

300

400

T85 CAM Analysis T85 CAM Guess

500 600 700

Pressure (hPa)

NCEP Opnl. Guess 400

T85 CAM Analysis 600

900

900

0.9

1

1.1

1.2

1.3

1.4

1000 0.8

1.5

TEMPERATURE K

1

1.2

1.4

1.6

1.8

2

2.2

2.4

2.6

TEMPERATURE K

Tropics

Raeder et al.: AGU Spring 2007

T85 CAM Guess

700 800

0.8

NCEP Opnl. Guess

500

800

1000 0.7

NCEP Opnl. Analysis

Northern Hemisphere

16

5/18/07

:

6-Hour Forecast and Analysis Observation Space Wind RMS RMS Error: Tropics

RMS Error: Northern Hemisphere 100

200

200

300

300

400

400

Pressure (hPa)

100

500 600 700

500 600 700

NCEP Opnl. Analysis

NCEP Opnl. Analysis 800

T85 CAM Analysis

900

NCEP Opnl. Guess

800

NCEP Opnl. Guess

T85 CAM Analysis

900

T85 CAM Guess

T85 CAM Guess 1000 1

2

3

4

5

6

7

8

1000 2

VECTOR WIND M/S

3

3.5

4

4.5

5

5.5

6

6.5

VECTOR WIND M/S

Tropics

Raeder et al.: AGU Spring 2007

2.5

Northern Hemisphere

17

5/18/07

Adding new models, new observations is simple. 1. Incorporating existing model requires handful of interfaces. A. No need for linear tangents or adjoints. B. Finite Volume CAM added in 1-month by postdoc. 2. Adding observations also straightforward. A. Only need forward operator (map state to expected observation). B. No linear tangents or adjoints. C. Several different GPS operators added in weeks. GPS and other novel observations may help detect climate model bias. GPS provides soundings of temperature and water world-wide.

Raeder et al.: AGU Spring 2007

18

5/18/07

Assimilating GPS Radio Occultation Observation Assimilated as refractivity along beam path. Complicated function of T, Q, P and ionospheric electric field.

Get a sounding as GPS satellite sets relative to low earth satellite. Raeder et al.: AGU Spring 2007

19

5/18/07

DART compliant models (largest set ever with assim system?) 1. Many low-order models (Lorenz63, L84, L96, L2004,...). 2. Global 2-level PE model (from NOAA/CDC). 3. CGD’s CAM 2.0, 3.0, 3.1 (global spectral model) 3a. CGD’s CAM 3.1 FV (global finite volume model) with chemistry. 4. GFDL AM GCM (global grid point model). 5. MIT GCM (from Jim Hansen; configured for annulus). 6. WRF model (regional prediction grid point). 6a. WRF column physics model. 7. NCEP GFS (operational global spectral; assisted by NOAA/CDC). 8. GFDL MOM3/4 (global grid point ocean model). 9. ACD’s ROSE model (upper atmosphere with chemistry). 10. Cane-Zebiak 5 (tropical ocean/atmosphere model). Also models from outside geophysics. This allows for a hierarchical approach to filter development. Raeder et al.: AGU Spring 2007

20

5/18/07

DART compliant Forward Operators and Datasets Many linear and non-linear forward operators for low-order models. U, V, T, Ps, Q, dewpoint for realistic models. Radar reflectivity, doppler velocity, GPS refractivity for realistic models. Mopitt CO retrievals. Can ingest observations from reanalysis or operational BUFR files. Can create synthetic (perfect model) observations for any of these.

Raeder et al.: AGU Spring 2007

21

5/18/07

Additional enhancements available for Earth System Analysis 1. Smoother: uses observations in past and future to estimate state. 2. High performance parallel implementations. Filter algorithms are naturally scalable. Run many copies of models. Impact of observations on model variables can be done in parallel. Algorithm has natural reformulations for different platforms. 3. Parameter estimation in large models. Use data to constrain parameters. Implementation is trivial. Interpretation is still VERY tricky. Did this for gravity wave drag efficiency in CAM.

Raeder et al.: AGU Spring 2007

22

5/18/07

Climate Model Parameter Estimation via Ensemble Data Assimilation T21 CAM assimilation of gravity wave drag efficiency parameter.

45 0

Oceanic values are noise (should be 0).

−45 0 −10

90 −5

180

270

0

5

0< efficiency< ~4 suggested by modelers. 10

Positive values over NH land expected. Problem: large negative values over tropical land near convection. May reduce wind bias in tropical troposphere, but for ‘Wrong Reason’.

Assimilation tries to use free parameter to fix ALL model problems

Raeder et al.: AGU Spring 2007

23

5/18/07