Interaction between bias correction and quality control

QUARTERLY JOURNAL OF THE ROYAL METEOROLOGICAL SOCIETY Q. J. R. Meteorol. Soc. 133: 643–653 (2007) Published online in Wiley InterScience (www.interscience.wiley.com) DOI: 10.1002/qj.57

Interaction between bias correction and quality control T. Aulign´e* and A. P. McNally European Centre for Medium-Range Weather Forecasts, Reading, UK

ABSTRACT: An interaction between the quality control (QC) and the bias correction of satellite radiances has been identified. If the bias correction is recalibrated intermittently, or if it is adaptive, a feedback process is possible. Indeed, the bias is calculated over a population of quality-controlled observations. Since QC usually acts upon bias-corrected observed-minus-first-guess departures, the value of the bias correction influences the next population that passes the QC, and so on. Two situations that can trigger a feedback are described: residual outliers that have not been detected by the QC; and an asymmetric QC that selects a sub-population of the good dataset. In both cases, the bias correction is strongly influenced by the feedback, and the performance of the QC is also degraded. The use of a new metric for the bias correction (called the ‘pseudo-mode’), approximating the mode of the radiancedeparture distribution, significantly reduces the feedback due to outliers and asymmetric QC. The variational bias correction scheme VarBC, which updates the bias inside the analysis, also constrains the feedback triggered by an asymmetric QC, but it has limited skill for outliers in a population of infrared window channel observations. A combination of VarBC with the pseudo-mode benefits from the advantages of both approaches. The bias correction is less sensitive to the QC, and more robust with respect to residual outliers. Copyright  2007 Royal Meteorological Society KEY WORDS

adaptive bias correction; feedback process; pseudo-mode; variational bias correction

Received 26 September 2006; Revised 26 January 2007; Accepted 6 February 2007

1.

Introduction

Observations are imperfect and therefore prone to error. The data with errors that are not described by the assimilation system through the error covariance matrices need to be eliminated prior to the analysis. In order to detect these data, a quality control (QC) is applied. Infrared and microwave radiances are also sensitive to clouds, aerosols and hydrometeors. When only the clearsky radiances are assimilated, a cloud detection is used; this can be considered as a particular QC focusing on the elimination of cloud-contaminated data. Most QC and cloud-detection schemes act upon observed-minusfirst-guess departures (so called ‘first-guess checks’) to discriminate between good and bad data. However, in some cases, to be useful, these departures in some cases have to be bias-corrected before the check. The bias correction aims at removing the systematic errors attributed to the observations, the radiative transfer and the pre-processing steps. The estimation of the biases requires a population of quality-controlled observations representative of those we ultimately intend to assimilate. There is thus a fundamental link between bias correction and QC: a different choice of QC threshold will result in a different estimate of the bias.

The above applies to a bias correction scheme applied statically. However, if this static bias correction is recalibrated intermittently, or if an adaptive bias correction scheme (i.e. a scheme that updates the bias estimate for each assimilation cycle) is used, there is a potential feedback mechanism. The value of the bias correction influences the population that successfully passes the QC, which, in turn, is used to define the new bias estimate. If not controlled, this feedback mechanism can substantially degrade the numerical weather prediction (NWP) analysis. Observations from the Atmospheric Infra-Red Sounder (AIRS) (Aumann et al., 2003) are used here to illustrate this problem. However, the problem is general, and can apply to any satellite observation. AIRS data are currently assimilated operationally or pre-operationally in several NWP centres (Aulign´e et al., 2003; Collard, 2003; McNally et al., 2006). Although AIRS provides relatively few poor-quality data, like any other infrared instrument it is strongly affected by clouds. The feedback mechanism is explained in Section 2, where different situations are illustrated with theoretical examples. Sections 3 and 4 propose two solutions that greatly reduce this problem. The combination of these two solutions is studied in section 5. Conclusions are presented in Section 6.

* Correspondence to: T. Aulign´e, European Centre for Medium-Range Weather Forecasts, Shinfield Park, Reading, Berkshire RG2 9AX, UK. E-mail: [email protected]

2. Feedback mechanism We use a theoretical simulation to demonstrate two processes that trigger a feedback mechanism. The basis

Copyright  2007 Royal Meteorological Society

T. AULIGNE´ AND A. P. MCNALLY

2.1.

Influence of residual outliers

QC and cloud-detection schemes are never perfect, and residual outliers (e.g. poor-quality data or cloud residual) are inevitable. Their influence is studied with a theoretical simulation applied to a real population of radiance departures for AIRS window channel 787 (10.89 µm) issued from the data used in the ECMWF operational assimilation system over a 12-hour window. In order to increase the amount of cloud residual in the dataset, the ECMWF operational cloud detection scheme for AIRS (McNally and Watts, 2003) has been intentionally degraded (the threshold on the first-guess departures being relaxed from 0.5 K to 5 K). The distribution of radiance departures is shown in Figure 1. It contains clear observations dominated by random error and associated with a nearly-Gaussian distribution. The distribution also exhibits a well-known cold tail, which is the signature of cloud residual in the infrared band (and a less prominent warm tail, corresponding to warm clouds over cold surfaces). Whatever the box-car window width chosen a priori, the adaptive bias correction converges after a few iterations (Figure 2). However, both the final estimate of the bias correction and the speed of convergence depend strongly on the chosen width. The more stringent the cloud detection, the more inertia the system will have and the longer it will take for the bias correction to converge. Although this is a simple model, it does highlight the potential feedback between QC and bias correction. The main effect overlooked is that in a real system, the assimilation of the data being bias-corrected could cause the analysis itself to drift, which in turn affects the next update of the bias correction. 2.2.

Influence of an asymmetric QC

In the absence of residual outliers, the use of a QC that selects asymmetrically a part of the population can still trigger a feedback mechanism between QC and bias correction. The same simple model is applied to a theoretical population of departures defined by a Gaussian distribution with mean zero and standard deviation one. The box-car window is now applied asymmetrically around zero with an upper bound of +2 K Copyright  2007 Royal Meteorological Society

250

Active data count

is a population of radiance departures assumed to be representative of the departures in an NWP assimilation cycle (e.g. a 12-hour assimilation window with a firstguess issued from a short-range forecast). Depending on the trigger that is being studied, the population will be chosen from real or simulated data. For the sake of simplicity, the bias correction is limited to a constant offset, and it is calculated as the mean of the radiance departures over the active population. The active data are determined with a simple QC, a ‘first-guess check’ defined as a box-car window with a pre-defined width around zero applied to the bias-corrected departures. An adaptive system is simulated by updating the bias estimate and the QC iteratively.

200 150 100 50 0 −6 −5 −4 −3 −2 −1

0

1

2

3

4

5

Uncorrected first–guess departures (K)

Figure 1. Histogram of AIRS window channel 787 (10.89 µm) uncorrected first-guess departures. The population has been selected by a degraded cloud-detection scheme. 0

+/− 0.5 K +/− 1.0 K +/− 1.5 K +/− 2.0 K

−0.1 −0.2 Bias estimate (K)

644

−0.3 −0.4 −0.5 −0.6 −0.7 −0.8 −0.9 −1

0

5

10

15 Iterations

20

25

30

Figure 2. Simulation of an adaptive bias correction scheme with residual outliers. The bias estimate is represented as a function of the number of iterations, for different QC window widths.

and a lower bound between −1.75 K and zero. This type of asymmetric QC is often used, for example for the detection of infrared cloudy radiances where warm departures are more likely to be clear than cold ones. Figure 3 shows the bias estimate and the number of active data as functions of the number of iterations of the bias correction scheme. For slightly asymmetric QC (with a lower bound of −1.75 K or −1.5 K), the bias estimate is very close to the real zero mean of the population. When the asymmetry in the QC is greater, the selected population for each iteration of the bias correction has a mean significantly different from the global zero mean. This influences the active population of the next iteration, and the number of active data decreases. This process is iterated, leading to a convergence state that is highly dependent on the thresholds used for the bounds of the QC. For extremely asymmetric QC (with a lower bound of −0.25 K), the bias estimate diverges significantly from the actual mean of the population, and the number of active observations decreases to almost zero (i.e. almost all observations are eventually thrown away). Q. J. R. Meteorol. Soc. 133: 643–653 (2007) DOI: 10.1002/qj

645

INTERACTION BETWEEN BIAS CORRECTION AND QUALITY CONTROL 3

2.5

Bias estimate (K)

2

−1.75 −> 2 K −1.50 −> 2 K −1.25 −> 2 K −1.00 −> 2 K −0.75 −> 2 K −0.50 −> 2 K −0.25 −> 2 K

1.5

1

0.5

0 5

10

(a)

15 Iterations

20

25

30

10000 9000 8000

Active data count

7000 6000 5000 4000 −1.75 −> 2K −1.50 −> 2K −1.25 −> 2K −1.00 −> 2K −0.75 −> 2K −0.50 −> 2K −0.25 −> 2K

3000 2000 1000 0

5

10

(b)

15 Iterations

20

25

30

Figure 3. Simulation of an adaptive bias correction scheme with an asymmetric QC. The bias estimate (upper panel) and the active data count (lower panel) are represented as functions of the number of iterations, for different QC window widths.

3.

Use of the mode to reduce feedback

It has been shown that feedback can be triggered by residual outliers (e.g. poor-quality data or contamination by cloud or rain) or asymmetric QC. There is an advantage to considering a bias correction based on the mode (peak) of the distribution of first-guess departures, instead of its mean, since the mode is expected to be less sensitive to outliers. Let us take the example of observations contaminated by clouds. The first-guess departures for cloudy data are much more heterogeneous Copyright  2007 Royal Meteorological Society

than those for clear data, given the variety of cloud types, depths and altitudes, and their corresponding radiative impact. Assuming that the errors from the NWP model and the clear observations are dominated by Gaussian noise, the population of departures provided to the analysis can be represented by a combination of a relatively Gaussian population for the clear data and a more widely-spread cloud residual. The mode of the distribution is expected to be less influenced by the cloudy data than the mean. Since for a Gaussian distribution the mean is equal to the mode, this new bias Q. J. R. Meteorol. Soc. 133: 643–653 (2007) DOI: 10.1002/qj

T. AULIGNE´ AND A. P. MCNALLY

calculation should provide an estimate of the mean of the clear population. An adaptive bias correction using this formulation will converge immediately to a single value regardless of the QC thresholds. The mode is applicable to a constant offset, but a more complex representation of the bias requires a different metric. Furthermore, the mode is not differentiable, and is not practical to implement in a variational system. A new metric approximating the mode, which we will call the ‘pseudo-mode’, is introduced. During the bias update, instead of considering all observations equally and averaging their contributions, we apply different weights to the observation contributions. These weights are derived from the normalized distribution of the uncorrected first-guess departures. Conceptually, this is equivalent to applying, for the bias calculation, a confidence to the observations according to their relative position within the distribution of departures. Outliers will therefore contribute less to the evolution of the bias than data close to the peak of the distribution. This formulation is similar to the M-estimator, also called the Huber norm (Huber, 2003), which defines a weight for each observation in the analysis on the basis of the first-guess departures. However, the proposed method only applies to the bias update, and does not require any assumptions about the shape of the distribution; nor does it assume that the population is unbiased. A parallel can also be drawn with the radiosonde mask described in (Aulign´e et al., 2007). This mask retains for the bias update only the observations in the vicinity of the radiosondes: this is equivalent to applying a weight of one near the radiosondes and zero elsewhere. The proposed metric can be considered as a mask that is not binary but has a smooth transition from one to zero with a sharpness that is defined by the magnitude of the uncorrected first-guess departures. We apply this metric to the two theoretical examples described in Section 2. The results from the simulation of a feedback triggered by outliers are shown in Figure 4 (to be compared with Figure 2). The new bias correction is less sensitive to the value of the box-car window limits than that based on the mean of the distribution, and the scheme converges faster. It can also be concluded, for this example, that the pseudo-mode is a reasonable approximation to the mode of the distribution (shown as a solid line). The results from the simulation of a feedback triggered by an asymmetric QC are shown in Figure 5 (to be compared with Figure 3). The feedback process is generally less significant with the bias correction based on the pseudo-mode, compared with that based on the mean of the distribution. For a slightly asymmetric QC (with a lower bound of −1.75 K, −1.5 K or −1.25 K), the bias estimate with the pseudo-mode does not diverge significantly from the actual mode of the global Gaussian distribution. The number of active data is also generally larger with the pseudo-mode. However, the use of the pseudo-mode does not eliminate the feedback, especially for highly asymmetric QC (with a lower bound of −0.25 K or zero), where the bias estimate still diverges Copyright  2007 Royal Meteorological Society

0

+/− 0.5 K +/− 1.0 K +/− 1.5 K +/− 2.0 K Mode

−0.1 −0.2 Bias estimate (K)

646

−0.3 −0.4 −0.5 −0.6 −0.7 −0.8 −0.9 −1

5

10

15

20

25

30

Iterations

Figure 4. As Figure 2, but using the proposed pseudo-mode metric. The bias estimate from the mode (i.e. the peak of the distribution of radiance departures) is shown by the black solid line.

from the mean of the population and the number of active data decreases dramatically. 4. Use of variational bias correction to reduce feedback The feedback process is now studied in a real NWP environment. The cloud-detection scheme used operationally at ECMWF for AIRS is more complicated than the firstguess check used in the above examples. It is described in detail in (McNally and Watts, 2003). In (Dahoui et al., 2005) the scheme is compared with other cloud detections and with collocated MODIS cloud information, and found to perform well for NWP purposes. However, significant NWP model humidity errors can result in large first-guess departures in the water-vapour band (7 µm), with a signature similar to cloud contamination. An artefact of the cloud-detection scheme is that it is more likely to reject negative departures than positive ones. First-guess departure histograms for AIRS mid-tropospheric water-vapour channel 1545 (7.23 µm) are shown in Figure 6, for the total population and the population retained by the clouddetection scheme. It can be seen that the cloud detection induces a discrepancy between the bias estimated from the active population and the actual bias of the population. This example can be classified as an asymmetric QC, and simulations have shown that it can trigger a feedback with the bias correction. An assimilation experiment has been run using an adaptive bias correction scheme that updates the bias before each analysis cycle. Figure 7 shows the evolution of the bias correction, departure statistics and data counts for AIRS channel 1545. The bias correction drifts (by more than 3 K) as the number of data accepted by the cloud detection decreases dramatically, nearly reaching zero. This is a real example of the ‘runaway’ feedback process triggered by an asymmetric QC, as described and simulated in Section 2.2. The variational bias correction scheme VarBC (Dee, 2004; Dee, 2005) is an adaptive scheme applied to Q. J. R. Meteorol. Soc. 133: 643–653 (2007) DOI: 10.1002/qj

647

INTERACTION BETWEEN BIAS CORRECTION AND QUALITY CONTROL 10000 9000 8000

Active data count

7000 6000 5000 4000 −1.75 −> 2K −1.50 −> 2K −1.25 −> 2K −1.00 −> 2K −0.75 −> 2K −0.50 −> 2K −0.25 −> 2K

3000 2000 1000 0

5

10

(a)

15 Iterations

20

25

30

10000 9000 8000

Active data count

7000 6000 5000 4000 −1.75 −> 2K −1.50 −> 2K −1.25 −> 2K −1.00 −> 2K −0.75 −> 2K −0.50 −> 2K −0.25 −> 2K

3000 2000 1000 0

5

10

(b)

15 Iterations

20

25

30

Figure 5. As Figure 3, but using the proposed pseudo-mode metric.

satellite observations. VarBC updates the bias inside the assimilation system by finding corrections that minimize the systematic differences between the satellite data and the model while simultaneously preserving (or improving) the fit to other observed data inside the analysis. This is achieved by including the bias correction parameters in the control vector of the analysis, so that the bias estimates are adjusted simultaneously with the NWP model, using all the information available to the analysis. The fitting is optimal in that it respects the uncertainty of the observations and any background or inertia Copyright  2007 Royal Meteorological Society

constraints we wish to impose on changes to the satellite bias estimation. The scheme implicitly constrains the calculation of the bias correction with the fit of the analysis to all other observations (e.g. radiosondes) (Aulign´e et al., 2007). We might expect that any potential feedback process would modify the analysis, and thus its fit to all data. Provided that a subset of observations is not bias-corrected adaptively, the system is expected to reach an equilibrium when it becomes too costly for the assimilation scheme to further update the bias parameters. Q. J. R. Meteorol. Soc. 133: 643–653 (2007) DOI: 10.1002/qj

T. AULIGNE´ AND A. P. MCNALLY

648 1600

All data Active data

1400

Data count

1200 1000 800 600 400 200 0 −5

−4

−3

−2

−1

0

1

2

3

4

5

Uncorrected first–guess departures (K)

Figure 6. Histograms of uncorrected first-guess departures for AIRS water-vapour channel 1545 (7.23 µm) for all data (grey) and data declared active (black) in the analysis.

Another assimilation experiment, in which the adaptive bias correction scheme described above is replaced by VarBC, has been run. Figure 8 shows the bias correction for AIRS water-vapour channel 1545 (7.23 µm). The bias now drifts by approximately 0.45 K (instead of more than 3 K with the other scheme). The number of active data is also more stable, at around 400. It can be concluded that in VarBC the extra constraint on the bias estimate

from all other observations greatly reduces the feedback process due to an asymmetric QC. An example with AIRS window channel 787 (10.89 µm) using a degraded cloud detection is now used to demonstrate the influence of outliers on the feedback process. The evolution of the bias estimate is represented in Figure 9 for two assimilation experiments, one using VarBC and one using an adaptive bias correction scheme (‘Offline’) performed before the assimilation. In both schemes, the bias evolves from −0.47 K to −1.27 K over 35 assimilation cycles (17.5 days). In this example, VarBC does not show any skill in preventing the bias correction from drifting away. This window channel is poorly constrained by the other data in the VarBC system. Since the bias correction becomes more negative, the cloud detection (which is mainly driven by a firstguess check for this channel) will select more negative departures. A change in the performance of the cloud detection is therefore expected. The AIRS Visible/Near Infrared instrument (VISNIR), a collocated imager, represents an independent source of information about the cloud amount. During daytime only, a proportion of cloud within the AIRS field of view is provided, as described in (Gautier et al., 2003). We consider two assimilation experiments, one using the VarBC scheme and one using a bias correction scheme (‘Static’) applied statically. The cloud amount

eq1e (DA): AIRS_NASA-1_Tb Ch 1545 Northern Hemisphere, used data, st. dev. and bias (K) 4 3

OB-FG OB-AN BIASCOR (mean)-0.66

2 1 0 28 29 30 1 JUN

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 2004 JUL

eq1e (DA): AIRS_NASA-1_Tb Ch 1545 Northern Hemisphere, used data 400

Daily used observations 4 days MA

350 300 250 200 150 100 50 28 29 30 1 JUN

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 2004 JUL

Figure 7. Upper panel: bias correction (black broken line) for AIRS mid-tropospheric water-vapour channel 1545 (7.23 µm), calculated with an offline adaptive bias correction scheme. An initial offset of 0.66 K is applied for better visualization. The standard deviation and bias for the first-guess departures (grey) and analysis departures (black) are represented by solid and dashed lines respectively. Lower panel: number of active data in the analysis (grey line), and 4-value moving average (black line). Copyright  2007 Royal Meteorological Society

Q. J. R. Meteorol. Soc. 133: 643–653 (2007) DOI: 10.1002/qj

649

INTERACTION BETWEEN BIAS CORRECTION AND QUALITY CONTROL epo0 (DA): AIRS_NASA-1_Tb Ch 1545, Northern Hemisphere, used data, st. dev. and bias (K) 1.20 1.05 0.90 0.75 0.60 0.45 0.30 0.15 0.00

OB-FG

28 29 30 1 JUN

OB-AN

2

3

4

5

BIASCOR (mean)-0.60

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 2004 JUL

epo0 (DA): AIRS_NASA-1_Tb Ch 1545, Northern Hemisphere, used data

500 400 300 Daily used observations 4 days MA

200 28 29 30 1 JUN

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 JUL 2004

Figure 8. As Figure 7, but using the variational bias correction scheme VarBC (with an initial offset of 0.60 K for the bias). −0.4

700

VarBC Offline

−0.5

Static VarBC

600 Active data count

Bias estimate (K)

−0.6 −0.7 −0.8 −0.9 −1 −1.1 −1.2 −1.3

400 300 200 100

5

10

15

20

25

30

35

Assimilation cycle

Figure 9. Bias estimate as a function of the number of assimilation cycles, for the Offline adaptive bias correction scheme (white) and the VarBC scheme (black).

according to VISNIR for the AIRS window channel 787 population retained by the cloud detection is represented in Figure 10. The VarBC experiment shows fewer data than the Static experiment with low cloud cover (20% or less) and more data with higher cloud cover (more than 20%). It can therefore be concluded that the update of the bias with VarBC has degraded the performance of the cloud-detection scheme. 5.

500

Combination of the pseudo-mode with VarBC

The pseudo-mode metric has been implemented for the bias calculation in combination with the variational bias Copyright  2007 Royal Meteorological Society

0

0

10

20

30

40

50

60

70

80

90 100

Cloud cover for AIRS/VISNIR imager (%)

Figure 10. Histograms of cloud cover from the VISNIR collocated imager for a population where AIRS window channel 787 (10.89 µm) is active. Two assimilation experiments have run for 7 weeks, one using the Static bias correction scheme (white) and one using the VarBC scheme (black).

correction scheme VarBC. This combination is described theoretically, and illustrated with a simple model, in Appendix A. The effect of the pseudo-mode metric on the shape of the cost function is studied. It can be concluded that the pseudo-mode slightly alters the a priori ambiguity between an update in the bias correction and meteorological increments (for temperature and humidity) to correct the radiance departures. The ability of VarBC to partially distinguish between observation bias and systematic NWP model error, demonstrated in Q. J. R. Meteorol. Soc. 133: 643–653 (2007) DOI: 10.1002/qj

T. AULIGNE´ AND A. P. MCNALLY

650

(Aulign´e et al., 2007), also needs to be assessed for the combination of VarBC with the pseudo-mode metric. The two assimilation experiments ‘VarBC’ and ‘VarBC-pseudo-mode’ use, respectively, VarBC alone and VarBC combined with the pseudo-mode. Figure 11 shows the evolution of the bias in these two experiments for the previous example of outliers in an AIRS window channel 787 (10.89 µm) population that trigger a feedback. Starting from an initial bias of −0.47 K, VarBC needs about 8 cycles to converge to a value around −1.25 K. The system updates the bias correction to balance globally the effect of cloud contamination on the departures. VarBC-pseudo-mode converges in only 2 cycles to a bias estimate that is much less influenced by the cloud contamination of the population. As before, the dramatic difference in the bias correction evolution results in different performances of the clouddetection scheme. Figure 12 shows the cloud amount −0.4

VarBC VarBC + pseudo–mode

−0.5 Bias estimate (K)

−0.6 −0.7 −0.8 −0.9 −1 −1.1 −1.2 −1.3

0

5

10 15 20 Assimilation cycle

25

Figure 11. Bias estimate as a function of the number of assimilation cycles, for VarBC (black) and VarBC combined with the pseudo-mode metric (white). 900 VarBC VarBC + pseudo–mode

800

Active data count

700 600 500 400 300 200 100 0

0

10

20

30

40

50

60

70

80

90 100

Cloud cover for AIRS/VISNIR imager (%)

Figure 12. Histograms of cloud cover from the VISNIR collocated imager for a population where AIRS window channel 787 (10.89 µm) is active. Two assimilation experiments have run for 13 days, one using VarBC (black) and one using VarBC combined with the pseudo-mode metric (white). Copyright  2007 Royal Meteorological Society

within the AIRS pixel from the AIRS VISNIR imager. The VarBC-pseudo-mode experiment retains more data than the VarBC experiment with a cloud cover under 20%. It also keeps fewer data with a cloud cover greater than 20%.

6.

Conclusion and perspectives

A feedback mechanism between QC and iterated bias correction has been identified. The bias correction is usually calculated from a quality-controlled dataset, and therefore the a priori definition of the QC has a direct influence on the bias estimation. This might already result in a sub-optimal assimilation system when the bias correction is applied statically. In the case of a bias correction recalibrated intermittently, or of an adaptive bias correction, there is the additional risk of degrading the analysis through a feedback mechanism. The update of the bias modifies the QC for the next assimilation cycle, and the process is iterated. Two situations that can trigger a feedback have been described: residual outliers that have not been detected by the QC (e.g. first-guess check or cloud contamination), and an asymmetric QC that selects a sub-population of the good dataset (e.g. an over-conservative cloud detection). In both cases, the bias correction is strongly influenced by the feedback, and the performance of the QC is also degraded. The use of the mode instead of the mean reduces the feedback process. Since this is not practical to implement in an NWP system, a new metric, called the pseudomode, has been introduced: it consists in weighting the observations by their departures from the NWP model during the bias calculation. The bias calculated with this metric is an approximation to the bias from the mode. The use of the pseudo-mode significantly reduces the influence of outliers and asymmetric QC, both of which trigger feedback. The variational bias correction scheme VarBC updates the bias inside the analysis, and therefore constrains the bias values by the fit to all other observations. VarBC manages to reduce the feedback in the case of an asymmetric QC. It has not demonstrated particular skill in eliminating feedback triggered by residual outliers in the case of AIRS window channel 787 (10.89 µm) observations. For this channel, the bias calculation is weakly constrained by other observations, and therefore VarBC performs comparably to an offline adaptive scheme. A combination of VarBC with the pseudo-mode benefits from the advantages of both approaches: the bias is constrained by the fit to all other data, and is less sensitive to outliers and asymmetric QC. Implementation of this scheme in an NWP system shows an increased robustness of the bias correction with respect to cloud residual. By reducing the dependency of the bias correction on the QC, it reduces the feedback process. However, an approach that tried to make the QC independent of the bias correction would also be very Q. J. R. Meteorol. Soc. 133: 643–653 (2007) DOI: 10.1002/qj

651

INTERACTION BETWEEN BIAS CORRECTION AND QUALITY CONTROL

useful. This could be achieved by defining QC checks that are not related to the NWP model, and thus do not require bias correction (but could rely on ancillary data). Furthermore, a QC that is updated regularly (or adaptively) has the potential to sustain the QC performance even if the bias correction is evolving. Further work on adaptive QC methods and their impact on the feedback process would therefore be beneficial.

In order to implement the new metric for the bias correction calculation, we introduce a vector of weights w, with its components wi defined for each observation i as a function of the uncorrected first-guess departure yi − x, normalized in such a way that

Acknowledgements

The implementation of the pseudo-mode bias correction is equivalent to modifying the expression of the cost function in Equation (3) to:

The authors wish to thank Dick Dee for the implementation of VarBC at ECMWF and for his constant reliable support. Elias Holm and Yannick Tr´emolet are acknowledged for fruitful discussions about the Huber norm used in data assimilation. Lars Isaksen must be thanked for developing priceless diagnostic tools. Jean-Noel Th´epaut provided helpful comments to improve the manuscript.

1
wi = 1. n i=1 n

Jo (x, b) =

n


(yi − x) + 2

i=1

−

n


n


wi (yi − b)2

i=1

yi2 + 2nbx.

(A.4)

i=1

A simple model is introduced to illustrate the VarBC mechanism. A series of n satellite observations is considered. For the sake of simplicity, the NWP model state and the observations represent the same meteorological quantity (e.g. temperature) at the same time and location (i.e. the observation operator is reduced to the identity matrix). Every observation is assumed to be active, and the corresponding error statistics are set to be constant, with a variance of one and no correlation between observational errors. The bias is characterized by a global offset b that can be calculated either as the mean of the departures for the active population (the mean bias) or by using the proposed metric (the pseudo-mode bias). Provided that the error distributions for the NWP model and the observations are normal, the best linear unbiased estimate can be found in a variational framework (Talagrand, 1992) by minimizing a cost function J (x) = Jb + Jo ,

(A.1)

where x is the model state, Jb is the part of the cost function with respect to the NWP model background (e.g. a short-range forecast) and Jo is the part with respect to the observations. In the VarBC framework, the bias model will influence the Jo (Dee, 2004), which can be simply expressed as Jo (x, b) =

n


(yi − x − b)2 ,

(A.2)

i=1

Jo (x, b) =

(yi − xi )2 +

i=1

−

n


500 450 400 350 300 250 200 150

where yi is the ith observation and x is the NWP model state. This expression is equivalent to the formula: n


For a Gaussian distribution of departures, the two cost functions derived from the mean bias model and the pseudo-mode bias correction are identical to the discretization accuracy. A theoretical population of observed-minus-background departures (first-guess departures) for AIRS window channel 787 (10.89 µm) is generated. It is defined as a combination of a clear population with white noise (corresponding to a Gaussian distribution) and a cloudy residual with random departures within a range of negative values (the cold tail in the histogram displayed in Figure 13). The cost function Jo within the VarBC framework of Equation (3) is displayed as a function of the state vector and the bias in Figure 14 (top panel). The range of pairs (x, b) that minimize the cost function for the clear population is overlaid. VarBC displays a symmetry in the shape of the corresponding cost function, since there is no information in the departures to distinguish between the state vector and the observation bias. This equally allows VarBC to adjust the state vector or the bias to improve the global fit to other observations and to

Data count

A. Appendix: Theoretical description of the combination of the pseudo-mode metric with the variational bias correction

n


50

(yi − b)2

0 −6

i=1

yi2 + 2nbx.

i=1

Copyright  2007 Royal Meteorological Society

100

−4

−2

0

2

4

6

8

10

First–guess departures (K)

(A.3)

Figure 13. Theoretical histogram of first-guess departures for the active population of an infrared instrument window channel. Q. J. R. Meteorol. Soc. 133: 643–653 (2007) DOI: 10.1002/qj

T. AULIGNE´ AND A. P. MCNALLY

652 8

60 0

30 0 20 0

7

40 0

16 0 14 0

18 0

70 0

80 0

90 0

50 0

11 00

20 0

10 00

Temperature increment (K)

6

60 0

30 0 20 0

5

7 70 5

13 00

12 00

10 0

80

12 0

40 0

16 0 14 0

18 0

70 0

80 0

90 0

50 0

20 0

4

3

60 0

30 0

65

20 0

70 75

7 70 5

80

10 0

80

65

12 0

40 0

16 0 14 0

18 0

50 0

20 0

2 30 0

65

1

70 75

12 0

14 16 0 0

20 0

10 0

18 0

1

80

65

2

3

7 70 5

4

12 0

10 0

80

5

16 0 14 0

18 0

6

20 0

8

7

Bias increment (K)

(a) 8

30 0

50

0

40

60

0

202 00

0

12

90

80

0

0

70

00

7 18 0

16

0

5

30 0

14

120

0

100

4 70

30 0

14 0

0

202 00

0

2

45 70

0

10

0

12

35

14

30

0

14

0

30

10 0 75

16

40

45

5

20

0

12

50

4

18

0

55

15

3

0

202 00

25

40 45 50 60

2

0

55

75 80

0 180 0 20

65

16

1

18

12 0 8 65 70 0 60

0

0

16

50

50

40

60

(b)

0

0

0

55

1

80

70

0

18

75

65

60

0

10 0

80

0

40

16 0 12

3

50

202 00

75 80

Temperature increment (K)

6

00

10

0

00

11

0

65 7080 60

0

75

30 35

6

14

7

10

0

8

Bias increment (K)

Figure 14. Objective function Jo as a function of the bias and temperature increments, with VarBC (upper panel) and VarBC combined with the pseudo-mode metric (lower panel). The crosses represent the minimum of Jo with VarBC. The circles represent the minimum of Jo if only the clear population is considered.

the background. The area of low values for the cost function is shifted relative to the minimum Jo for the clear population because of the cloud-contaminated data. Figure 14 (lower panel) displays the cost function for VarBC combined with the pseudo-mode, corresponding to Equation (4). The shift due to cloud contamination is Copyright  2007 Royal Meteorological Society

much smaller than with the standard VarBC, showing that the proposed metric is less sensitive to outliers. Nevertheless, the symmetry of Jo is broken, and one source of systematic departures – either the model state or the observation bias, depending on the normalization factor for the histogram – is slightly favoured. Aulign´e et al. Q. J. R. Meteorol. Soc. 133: 643–653 (2007) DOI: 10.1002/qj

INTERACTION BETWEEN BIAS CORRECTION AND QUALITY CONTROL

(2007) have demonstrated that VarBC is able to partially distinguish observation bias from systematic NWP model error. This ability is expected to be slightly degraded when VarBC is combined with this weighted mean metric. Therefore a combined approach should only be used when it is believed that the bias model efficiently distinguish between the sources of bias, or when outliers (such as cloud-contaminated observations) constitute a major concern for the analysis. References Aulign´e T, McNally AP, Dee DP. 2007. Adaptive bias correction for satellite data in a numerical weather prediction system. Q. J. R. Meteorol. Soc. 133: 631–642. Aulign´e T, Rabier F, Lavanant L, Dahoui M. 2003. First results of the assimilation of AIRS data in Meteo-France Numerical Weather Prediction model. In: Proceedings of the thirteenth International TOVS Study Conference. Sainte Adele, Canada, 29 October to 4 November 2003: 74–79. Aumann H, Chahine M, Gautier C, Goldberg M, Kalnay E, McMillin L, Revercomb H, Rosenkranz P, Smith W, Staelin D, Strow L, Susskind J. 2003. AIRS/AMSU/HSB on the AQUA mission: Design, science

Copyright  2007 Royal Meteorological Society

653

objectives, data products and processing systems. IEEE T. Geosci. Remote 41: 253–264. Collard A, Saunders R, Cameron J, Harris B, Takeuchi Y, Horrocks L. 2003. Assimilation of data from AIRS for improved numerical weather prediction. In: Proceedings of the thirteenth International TOVS Study Conference. Sainte Adele, Canada, 29 October to 4 November 2003: 99–106. Dahoui M, Lavanant L, Rabier F, Aulign´e T. 2005. Use of the MODIS imager to help deal with AIRS cloudy radiances. Q. J. R. Meteorol. Soc. 610: 2559–2579. Dee DP. 2005. Bias and data assimilation. Q. J. R. Meteorol. Soc. 131: 3323–3343. Dee DP. 2004. Variational bias correction of radiance data in the ECMWF system. In: Proceedings of the ECMWF Workshop on Assimilation of High Spectral Resolution Sounders in NWP, Reading, UK, 28 June to 1 July 2004. pp. 97–112. Gautier C, Shiren Y, Hofstadter MD. 2003. AIRS/Vis Near IR instrument. IEEE T. Geosci. Remote. 41(2): 330–342. Huber PJ. 2003. Robust Statistics. Wiley. McNally AP, Watts PD. 2003. A cloud detection algorithm for highspectral-resolution infrared sounders. Q. J. R. Meteorol. Soc. 129: 3411–3423. McNally AP, Watts PD, Smith LA, Engelen R, Kelly G, Th´epaut JN, Matricardi M. 2006. The assimilation of AIRS radiance data at ECMWF. Q. J. R. Meteorol. Soc. 132: 935–957. Talagrand O. 1992. Assimilation of observations: An introduction. J. Meteorol. Soc. Jpn 75: 191–209.

Q. J. R. Meteorol. Soc. 133: 643–653 (2007) DOI: 10.1002/qj