AN INVESTIGATION INTO THE PERFORMANCE OF RETRIEVALS OF TEMPERATURE AND HUMIDITY FROM IASI 1
2
3
3
4
5
Fiona Hilton , Paolo Antonelli , Xavier Calbet , Tim Hultberg , Lydie Lavanant , Xu Liu , Guido 6 1 1 6 5 Masiello , Stuart Newman , Jonathan Taylor , Carmine Serio , Daniel Zhou (1) Met Office, FitzRoy Road, Exeter, EX1 3PB, United Kingdom (2) University of Wisconsin-Madison (3) EUMETSAT (4) Météo-France/CMS (5) NASA Langley Research Center (6) DIFA/Universita degli Studi della Basilicata
Abstract The meteorological community has had access to operational data from the Infrared Atmospheric Sounding Interferometer (IASI) since July 2007. The data are now successfully assimilated at many operational meteorological centres, which have demonstrated good impact on forecast skill from the use of the data for temperature and humidity sounding. The next stage in increasing the impact from IASI data is to understand the limitations of our processing methodology, and thus determine where modifications to processing or further research could enable us to extract more information from IASI. One way of investigating where the limitations of a particular methodology might be is to perform a careful intercomparison of the results of different retrieval schemes for the same observations. For this study, nine different retrieval schemes are applied to twenty five clear sky cases which form part of the Joint Airborne IASI Validation Experiment (JAIVEx) campaign of April-May 2007. The resulting retrievals are verified using near-coincident dropsonde data collected during the campaign. It is hoped that the similarities and differences in the retrievals from the different schemes shown here will guide future research into the best methods of using IASI data.
INTRODUCTION The Joint Airborne Infrared Atmospheric Sounding Interferometer (IASI) Validation Experiment (JAIVEx) (Newman et al., 2008; Smith et al., 2008) was a field campaign based in Houston, Texas during April and May 2007. The campaign was conducted to provide in-situ measurements giving the best possible understanding of the atmosphere at the time of certain MetOp overpasses of the Gulf of Mexico and the Atmospheric Radiation Measurement (ARM) Southern Great Plains site. A dataset of carefully selected clear sky IASI footprints from the campaign was made available to researchers in July 2008. This dataset combines the IASI footprints with collocated airborne hyperspectral infrared radiance measurements and near-coincident measurements of the atmospheric state from dropsondes, airborne and ground-based instrumentation. The availability of a good quality atmospheric validation dataset prompted this study, which uses the clear sky IASI footprints from the JAIVEx flights over the Gulf of Mexico to compare the performance of different retrieval schemes, and to validate them with the near-coincident dropsonde data. Data from three flights were analysed: 29 April 2007 with 6 IASI footprints and 2 dropsondes; 30 April 2007 with 16 IASI footprints and 4 dropsondes; and 4 May with 3 IASI footprints and 3 dropsondes. The nine different retrieval schemes shown in Table 1 have been included in the intercomparison and a brief description of each, along with the main features of the retrieval system, is provided in Table 2. All are physically-based retrieval systems – in other words they explicitly relate an estimate of the state of the atmosphere to the IASI observation through the use of a radiative transfer model rather than through a regression relationship or neural network approach. All the retrieval systems are also based on techniques derived from optimal estimation theory, and all perform a simultaneous retrieval of the atmospheric constituents analysed.
Researcher
Affiliation and Scheme Name
Fiona Hilton Paolo Antonelli Xavier Calbet Tim Hultberg
Met Office Operational 1D-Var Pre-processor University of Wisconsin-Madison UWPHYSRET EUMETSAT MTG-IRS Prototype EUMETSAT IASI Level 2 Processor
Lydie Lavanant Xu Liu Guido Masiello Jon Taylor Dan Zhou
Météo-France CMS Retrieval Scheme NASA Langley PCRTM University of Basilicata δ-IASI Met Office HT-FRTC NASA Langley
Reference Hilton et al. (2009) Contact
[email protected] Contact
[email protected] EUMETSAT IASI Level 2 Product Guide http://oiswww.eumetsat.org/WEBOPS/epspg/IASI-L2/IASIL2-PG-index.htm Contact
[email protected] Liu et al. (2009) Carissimo et al. (2005),Grieco et al. (2007), Masiello et al. (2009) Havemann (2006) Zhou et al. (2009)
Table 1: The retrieval schemes used in this intercomparison
Researcher Fiona Hilton Paolo Antonelli Xavier Calbet
Tim Hultberg
Lydie Lavanant Xu Liu Guido Masiello
Jon Taylor Dan Zhou
Scheme Description 1D-Var on biascorrected data, obs errors NEDT+0.2K. Optimal Estimation, obs errors NEDT*1.7 Optimal Estimation Similar to IASI L2 code. Optimal Estimation. Bias-corrected, noise-filtered data, full error covariance. 1D-Var on biascorrected data. Obs errors NEDT 1D-Var. EOFcompressed state vector. Statistical Regularisation; bias corrected data; obs error NEDT 1D-Var, obs errors NEDT+spectroscopy error 1D-Var with Minimum information regularization
Channels/PCs 183 Channels
Radiative Transfer RTTOV7
Retrieved Levels 43
5200 Channels
LBLRTM
51
4419 Channels
OSS
90
252 Channels
RTIASI-4
90
150 Channels
RTTOV9
43
100 PCs
PCRTM
101
2865 Channels
σ-IASI
60
EOF regression to climatology
100 PCs
HT-FRTC
43
ECMWF 6 hour forecast
1697 Channels
SARTA+Cloud model
101
EOF regression to Climatology
Table 2:The main characteristics of the inputs into the retrieval schemes used in the study
A Priori Met Office 6 hour forecast Local Climatology EOF regression to ECMWF Climatology ECMWF Climatology
ECMWF 12-18 hour forecast Climatology
The researchers were particularly interested in whether fine vertical structure and sharp gradients could be seen in the retrievals and whether such features could be introduced if they were not present in the a priori profile information used by the retrieval system. Two examples of features investigated were a dry layer at around 750hPa and a temperature inversion near the surface (see Figure 1). Although the schemes all share the similarities outlined above, the researchers were also interested to see whether the differences between them had any systematic bearing on the quality of the retrievals. Firstly, the proportion of the IASI spectrum used in the retrievals varies widely between the different schemes, from operational near-real-time retrieval schemes which use a subset of the channels chosen for use in numerical weather prediction (NWP) by Collard (2007), through to principal component (PC)-based schemes which use information from the whole spectrum. Secondly, all of the models used different radiative transfer code (for example the PC-RTM and HT-FRTC retrievals used PC radiative transfer models) and the retrievals are carried out on different numbers of atmospheric levels. Finally, some of the models use NWP short-range forecasts as a priori; the others use climatology, often via an intermediate empirical orthogonal function (EOF) regression to IASI.
VALIDATION AGAINST DROPSONDE DATA The dropsonde profiles included in the JAIVEx dataset did not pass through any cloud layers, and were all within 20 minutes in time of the MetOp overpass. The distance between the profiles and the retrievals was variable, but generally less than 0.5° latitude/longitude, equating to roughly 50km. Pougatchev et al. (2009) give an assessment of the spatial and temporal non-coincidence error for comparisons of IASI retrievals with radiosonde profiles at Lindenberg. Assuming their results are relevant for JAIVEx, which may not be true given the difference in latitude, the spatial non-coincidence error should be less than 0.8K in temperature at 600mb and the temporal error less than 0.4K. For humidity, the temporal error could be up to 8% in relative humidity at this pressure, with a similar or lower spatial error. The measurement error associated with the dropsonde is assumed to be smaller than this. For each of the three case studies, more than one dropsonde profile is available. The coincidence is generally better for some particular combination of footprints and sonde profiles than others, but in order to provide some estimate of the magnitude of the non-coincidence errors, all of the sonde profiles have been used in comparisons of the retrieval schemes for each footprint. Sonde data is only available below approximately 400hPa and no attempt has been made to verify above this. A qualitative comparison between the schemes is possible using the high-resolution dropsonde profiles, which allow some assessment of the vertical scales of variations in temperature and humidity present in the atmosphere. For a quantitative comparison, for example of the mean and standard deviation of the differences between the retrievals and the measured atmospheric profile, the sonde data need to be converted to the atmospheric levels used in each retrieval. Furthermore, it is not possible to compare the retrievals with each other directly as each is performed on a different set of levels. Various methods could be used to provide an estimate of the atmospheric profile on the retrieval levels, ranging from a linear interpolation of the dropsonde profile to a layer average, or to convolution with the averaging kernels. The latter method (following Rodgers and Connor, 2003) is preferred theoretically, because it takes account of the smoothing error inherent in the way the retrieval system would represent a true atmospheric profile with high-resolution vertical structure. The smoothing error arises because the retrieval is performed on a finite number of levels using satellite data which observes the atmosphere with broad weighting functions. However, convolution with the averaging kernel is problematic because, prior to convolution, the profile still needs to be interpolated to the levels on which the kernels were calculated, and because the kernels are broad and therefore sensitive to the atmosphere above the top of the sonde profile. The sharpness of vertical gradients in the dropsonde profiles, particularly in humidity, result in these different methods of resolution reduction giving quite different results. In the interests of simplicity, comparisons have been made between the retrievals and profiles linearly interpolated in pressure. For humidity at least, there is some justification for this method, as operational preprocessing of radiosonde data at the Met Office uses linear interpolation (Lorenc et al., 1996). However, due to the uncertainties in the interpolated profiles, no quantitative comparison has been attempted in this study.
Figure 1: Retrieval intercomparison for one IASI footprint, together with the full resolution dropsonde profiles. The lefthand plot shows temperature with a constant lapse rate subtracted, and the right-hand plot water vapour mass mixing ratio. The features of particular interest are the temperature inversion at approx. 920hPa and the dry layer at approx. 720hPa
THE INTERCOMPARISON It is not possible to present here the results from each of the IASI footprints in the case studies, so a sample is shown which is representative of the results in general. The different retrieval schemes are labelled by the first name of the researcher who provided the results (see Table 1). Figure 1 shows an intercomparison of one of the retrievals and the full-resolution dropsonde data. Despite the similarities in the schemes, the results are very variable. Some profiles show quite smooth variations with pressure, whereas others have unrealistic oscillations. The number of channels used in the retrieval has no obvious effect on the quality of the retrievals for these profiles. The variation in the retrieval of humidity in the lowest layers in particular is very high, with almost a factor of two variation in mass mixing ratio. None of the retrieval schemes successfully retrieve the temperature inversion around 920hPa or the dry layer at 720hPa. The gradients in humidity with height are generally not correct. Overall, the schemes which used an NWP forecast as a priori tended to produce retrievals with small biases and reasonably correct temperature structure, although the height of any inversion was generally unaffected by the retrieval process (for example, see Fiona in Figure 1, where the inversion is at approximately 880hPa as it was in the a priori profile). Figure 2 shows how schemes which started from climatology (e.g. Paolo, Xavier, Xu in Figure 2) were generally able to correct gross biases, particularly for temperature, but the presence of realistic structure in the temperature retrievals (e.g. Jon in Figure 2) was generally influenced to a great extent by their presence in the a priori profile. The retrieval schemes sometimes show compensation, improving the retrievals at some altitudes and introducing errors at others. There may also compensation between temperature and humidity – in some cases, the NWP temperature forecast can be found to fit the dropsonde better than the retrieval which could be a result of the temperature profile being adjusted to fit the observation on account of changes to the humidity profile.
Figure 2: A comparison of temperature a priori (background/first guess) in blue/green and retrieval in red for two of the retrieval schemes. The error bars plotted are the stated background and retrieval errors for each level where available. The dropsonde profiles in black have been interpolated to the retrieval levels. From left to right: Paolo, Xavier, Xu, Jon.
Figure 3: A comparison of water vapour a priori (background) and retrieval for two of the retrieval schemes. The error bars plotted are the stated background and retrieval errors for each level. The dropsonde profiles have been interpolated to the retrieval levels. Lydie’s retrieval, on the left, uses an NWP forecast as background whereas Tim’s retrieval, on the right, uses climatology.
Figure 4: Comparison of the results for all IASI footprints for two of the schemes, showing both retrieval and a priori (background) profiles with the full resoltion dropsonde data. From left to right: Guido, Fiona
Figure 5: Spectral residuals from the a priori (first guess) and retrieval using their respective forward models, along with those of the best fit dropsonde profile forward modelled with RTTOV. For Fiona, only the 314 channels from Collard (2007) are available. From top to bottom: Fiona, Guido, Lydie, Jon
While, in general, only small adjustments were made to NWP temperature profiles in the retrievals, larger increments were added to NWP humidity profiles, except in the lowest layers where the water vapour retrieval was strongly constrained by the a priori. Figure 3 compares two humidity retrievals one of which uses an NWP forecast and the other climatology as a priori. The two retrievals show similar fits to the dropsonde data above 850hPa, but while the NWP forecast allows a good retrieval of the boundary layer humidity, the climatology is far too dry which results in a retrieval which is also too dry. The averaging kernels (see below) indicate that the retrieval schemes generally show little sensitivity to the observations in the lowest layers for these atmospheric profiles. The forward model Jacobians show that the observations are not sensitive to the lowest layers of the atmosphere. Although none of the retrievals were able to successfully retrieve a dry layer at 720hPa for the 5 May case (Figure 1), the retrievals did show some skill in a similar situation on 29 April (Figure 4). In this case, most retrievals moved towards a drier state at the altitude of the dry layer or towards a moister atmosphere above, depending respectively on whether the a priori profile was relatively too moist or too dry. The δ-IASI scheme (Guido) produced a particularly good retrieval of the moist layer above the dry layer despite this structure not being present in the a priori. However, the retrieval was too moist below 880hPa, and the only retrievals to retrieve a dry profile down to 980hPa and a sharp humidity gradient in the lowest levels used NWP forecasts as a priori (e.g. Fiona in Figure 4). Again, the IASI observations had little influence on the retrieval for the lower levels and so the height of the sharp gradient, which was slightly too high in the a priori, was not corrected in the retrieval. The retrieval schemes were also compared in terms of their spectral residuals (Figure 5) although only four researchers were able to provide these. The retrievals showed, for most parts of the spectrum, a better fit than the a priori profiles to the IASI observations. The Met Office scheme showed particular improvements in the tropospheric temperature sounding and window channels, while the schemes using ECMWF forecasts as first guess showed good initial fits in the temperature sounding channels, and so less improvement. However, these schemes did show marked improvements in fit to the water vapour sounding channels. The best-fit dropsonde profile (extrapolated to the top of the atmosphere with the ECMWF analysis) showed a better fit to the IASI observations than the retrievals did in most cases, a notable exception being the δ-IASI scheme retrieval (Guido) for the window region. Two of -1 the schemes (Fiona, Lydie) do not retrieve ozone so the fit to the ozone channels (1000cm ) is poor.
Figure 6: Averaging kernels for temperautre (top) and water vapour (bottom) for one profile on 29 April for five of the retrieval schemes. Only kernels peaking below 400hPa have been plotted. The colour indicates which level the kernel is for, with blue being the lowest level and red being the highest. From left to right, Fiona, Lydie, Xu, Guido, Jon.
AVERAGING KERNELS Figure 6 shows the averaging kernels (following Rodgers, 2000) for five of the retrieval schemes. They differ widely in form, although all are broad. It is interesting to note that the two retrievals using PCs of the IASI spectrum have very different averaging kernels; for water vapour in particular, HT-FRTC (Jon) actually resembles the CMS scheme (Lydie) more closely than the PCRTM scheme (Xu). This suggests that the form of the background error covariance matrix is driving the form of the averaging kernel as HT-FRTC and CMS both use ECMWF forecasts as background and therefore the same background error covariances. The retrieval schemes used by the Met Office (Fiona) and CMS are very similar but there are differences in the averaging kernels, in particular for temperature where the Met Office background error covariance matrix seems to produce particularly broad kernels. Only δIASI (Guido) seems to show much sensitivity at the lowest levels (although the Met Office scheme in this instance shows a marked sensitivity in water vapour to the second and third levels from the surface where the background profile has particularly high humidity). Following this intercomparison, the reserachers operating schemes on 43 levels (which were historically used with RTTOV) expressed the opinion that these levels were insufficient to make full use of the information content of IASI data.
RESULTS AND INFERENCES This study compares the performance of nine IASI retrieval schemes for 25 footprints validated against near-coincident dropsonde data and as such does not provide a valid statistical estimate of the quality of the retrieval schemes. It does illustrate differences in performance of state-of-the-art temperature and humidity retrievals from hyperspectral data in one type of atmospheric environment and allows some general inferences to be made. The retrievals show that gross error in a first guess from climatology can be successfully corrected, but that structure in the retrieval often comes primarily from the first guess. A good quality first guess is therefore of high importance for a good retrieval. Other differences between schemes, such as the number of channels used, did not appear to have a
systematic effect on the retrievals. Some schemes do add correct structure that was not present in the first guess, but temperature and humidity retrievals may compensate in ways which add incorrect structure elsewhere. Most of the schemes can, for some footprints at least, reproduce broad features seen in the dropsonde data, but the sharpness of gradients and small-scale fluctuations are still not well retrieved. No scheme does a universally better job than the others. The results of this study will be used to inform future work on improving the information content of IASI retrievals.
REFERENCES Carissimo A., De Feis I., Serio C. (2005) The physical retrieval methodology for IASI: the δ-IASI code. Env Modelling & Software, 20, 9, pp 1111-1126. DOI 10.1016/j.envsoft.2004.07.003 Collard, A.D. (2007) Selection of IASI channels for use in numerical weather prediction. Q.J.R. Meteorol. Soc, 133, pp 1977-1991 Grieco, G., Masiello G., Matricardi M., Serio C., Summa D., Cuomo V. (2007) Demonstration and Validation of the φ-IASI Inversion Scheme with NAST-I Data. Q.J.R. Meteorol. Soc, 133, pp 217-232. DOI 10.1002/qj.162 Havemann, S. (2006) The development of a fast radiative transfer model based on an Empirical Orthogonal Functions (EOF) technique. Proc SPIE 6405, art. no. 64050M Hilton, F., Atkinson, N.C., English, S.J. and Eyre, J.R (2009) Assimilation of IASI at the Met Office and assessment of its impact through observing system experiments. Q.J.R. Meteorol. Soc, 135, 639, pp 495-505. DOI 10.1002/qj.379 Liu, X., Zhou, D.K. , Larar, A.M., Smith, W.L., Schluessel, P., Newman, S.M., Taylor, J.P. and Wu, W. (2009) Retrieval of atmospheric profiles and cloud properties from IASI spectra using super-channels. Atmos. Chem. Phys. Discuss., 9, pp 8683-873 Lorenc, A.C., Barker, D., Bell, R.S., Macpherson, B., and Maycock, A.J. (1996) On the use of radiosonde humidity observations in mid-latitude NWP. Met. And Atmos. Physics, 60, pp 3-17 Masiello G., Serio C., Carissimo A., Grieco G. (2009) Application of φ-IASI to IASI: retrieval products evaluation and radiative transfer consistency. Atmos. Chem. Phys. Discuss., 9, pp 9647-9691 Newman, S.M., Hilton, F., Taylor, J.P., Collard, A.D., Smith, W.L. and Larar, A. (2008) Direct Radiance Validation of IASI – Results from JAIVEx. In Proceedings of the 16th International TOVS Study Conference, Angra dos Reis, Brazil, 6-13 May 2008. Available from: http://cimss.ssec.wisc.edu/itwg/itsc/itsc16/proceedings/index.html Pougatchev, N., August, T., Calbet, X., Hultberg, T., Oduleye, O., Schlüssel, P., Stiller, B., St. Germain, K. and Bingham, G. (2009) IASI temperature and water vapour retrievals – error assessment and validation. Atmos. Chem. Phys., 9, pp 6453-6458 Rodgers, C.D. (2000) Inverse Methods for Atmospheric Sounding: Theory and Practice. World Scientific Publishing Co. Pte. Ltd., Singapore Smith, W.L., Larar, A., Taylor, J., Revercomb, H., Kireev, S., Zhou, D., Liu, X., Tobin, D., Newman, S., Schluessel, P., Clough, A., Mango, S. and St. Germain, K. (2008) Joint Airborne IASI Validation Experiment (JAIVEx) - An Overview. In Proceedings of the 16th International TOVS Study Conference, Angra dos Reis, Brazil, 6-13 May 2008. Available from: http://cimss.ssec.wisc.edu/itwg/itsc/itsc16/proceedings/index.html Zhou, D., Smith, W.L., Larar, A.M., Liu, X., Taylor, J.P., Schlüssel, P., Strow, L.L. and Mango, S.A. (2009) All weather IASI single field-of-view retrievals: case study – validation with JAIVEx data. Atmos. Chem. Phys., 9, pp 2241-2255
