Performance comparison of some dynamical and empirical ...

INTERNATIONAL JOURNAL OF CLIMATOLOGY Int. J. Climatol. 29: 1535–1549 (2009) Published online 8 December 2008 in Wiley InterScience (www.interscience.wiley.com) DOI: 10.1002/joc.1766

Performance comparison of some dynamical and empirical downscaling methods for South Africa from a seasonal climate modelling perspective Willem A. Landman,a,b * Mary-Jane Kgatuke,a Maluta Mbedzi,a Asmerom Beraki,a Anna Bartmana and Annelise du Piesaniea b

a South African Weather Service, Pretoria, South Africa Department of Geography, Geoinformatics and Meteorology, University of Pretoria, South Africa

ABSTRACT: The ability of advanced state-of-the-art methods of downscaling large-scale climate predictions to regional and local scale as seasonal rainfall forecasting tools for South Africa is assessed. Various downscaling techniques and raw general circulation model (GCM) output are compared to one another over 10 December-January-February (DJF) seasons from 1991/1992 to 2000/2001 and also to a baseline prediction technique that uses only global sea-surface temperature (SST) anomalies as predictors. The various downscaling techniques described in this study include both an empirical technique called model output statistics (MOS) and a dynamical technique where a finer resolution regional climate model (RCM) is nested into the large-scale fields of a coarser GCM. The study addresses the performance of a number of simulation systems (no forecast lead-time) of varying complexity. These systems’ performance is tested for both homogeneous regions and for 963 stations over South Africa, and compared with each other over the 10-year test period. For the most part, the simulations method outscores the baseline method that uses SST anomalies to simulate rainfall, therefore providing evidence that current approaches in seasonal forecasting are outscoring earlier ones. Current operational forecasting approaches involve the use of GCMs, which are considered to be the main tool whereby seasonal forecasting efforts will improve in the future. Advantages in statistically post-processing output from GCMs as well as output from RCMs are demonstrated. Evidence is provided that skill should further improve with an increased number of ensemble members. The demonstrated importance of statistical models in operation capacities is a major contribution to the science of seasonal forecasting. Although RCMs are preferable due to physical consistency, statistical models are still providing similar or even better skill and should still be applied. Copyright  2008 Royal Meteorological Society KEY WORDS

downscaling; regional climate model; general circulation model; model output statistics; South Africa

Received 25 May 2007; Revised 7 July 2008; Accepted 12 August 2008

1.

Introduction

Droughts and floods have long been distinctive features of the climate of southern Africa (Tyson and Preston-Whyte, 2000). Variability of the climate has been accentuated by the occurrence of the El Ni˜no/southern oscillation (ENSO) phenomenon, but is by no means dominated by them (Mason and Jury, 1997). The need for providing improved seasonal rainfall forecasts, both temporally and spatially, is becoming more and more necessary in the region. Improvements to the prediction of southern Africa’s summer rainfall on a seasonal scale will be best achieved using General Circulation Models (GCMs) (Landman et al., 2001a). Although GCMs, commonly configured with an effective resolution of 200–300 km, have demonstrated skill at global or even continental scale, they are unable to represent local sub-grid features, subsequently producing rainfall over southern Africa that * Correspondence to: Willem A. Landman, South African Weather Service, Private Bag X097, Pretoria, 0001, South Africa. E-mail: [email protected] Copyright  2008 Royal Meteorological Society

is typically overestimated (Joubert and Hewitson, 1997; Mason and Joubert, 1997). Such systematic biases have created the need to downscale GCM simulations to regional level over southern Africa. Semi-empirical relationships exist between observed large-scale circulation and rainfall, and assuming that these relationships are valid under future climate conditions and also that the large-scale structure and variability is well characterized by GCMs, mathematical equations can be constructed to predict local precipitation from the simulated largescale circulation (Wilby and Wigley, 1997). Empirical remapping of GCM fields to regional rainfall at month and seasonal time scales has been demonstrated successfully over southern Africa (Landman and Goddard, 2002, 2005a; Landman and Tennant, 2000; Landman et al., 2001a,b, 2005a). Further increase in resolution may be required over a specific area, which could be resolved by nesting a higher resolution limited area regional model within the global GCM (Giorgi, 1990). The nested model is run over a limited-area domain and is driven by time-dependent

1536

W. A. LANDMAN ET AL.

large-scale meteorological fields, such as output from a GCM. The nesting will improve on the topographic features that are poorly resolved by global models and the simulation of significant synoptic features inadequately simulated by coarser GCM resolutions. A finer resolution should thus improve on the understanding of how seasonal predictability is related to spatial scale, even though the choice of the regional model domain may have a significant impact on simulation results (Giorgi et al., 1996; Seth and Giorgi, 1998). However, given the most suitable domain, physical parameterizations and lower boundary conditions, a multi-year integration is required in order to properly recognize anomalous behaviour relative to the regional model’s time-averaged behaviour (Goddard et al., 2001), which requires a large amount of computer time and storage space. However, nested modelling approaches are capable of simulating the details of regional climate over southern Africa with greater skill than global models (Joubert et al., 1999). Owing to the large expense involved when using a combination of GCMs and regional models and to a lesser extent a combination of GCMs and empirical downscaling methods, the GCM-based approaches to seasonal rainfall estimation should produce rainfall forecasts with skill that outscores that of simple empirical modelling. An empirical model may, for example, utilize a relationship between oceanic and rainfall variability to assess the expected rainfall distribution of a coming season. The main objective is therefore to assess whether dynamicalbased approaches to seasonal rainfall estimation can produce rainfall forecasts with skill that outscores that of simple empirical modelling. In addition, it is necessary to assess if the nested approach is more skillful than that of the computationally less expensive empirical downscaling approach. The main season of interest is the December-JanuaryFebruary (DJF) season when tropical influences start to dominate the atmospheric circulation over southern Africa. The DJF season is therefore the season of highest predictability for South African seasonal rainfall (e.g. Landman and Mason, 1999; Landman et al., 2005a), which makes this season a logical choice for model inter-comparison studies. Moreover, the study will assess seasonal total rainfall predictability and not address issues pertaining to the predictability sub-seasonal variability.

2. 2.1.

Data and models Observed rainfall data and homogenization

A total of 963 stations, more or less evenly distributed over South Africa, are used in the study (Figure 1). These stations are also grouped into regions where the stations within each region have the same interannual rainfall variability. Cluster analysis is used to group the stations into homogeneous rainfall regions based on the standardized monthly rainfall totals for the 45-year period of 1960–2004. The data are first standardized to eliminate problems of non-linearity that arise if the monthly rainfall Copyright  2008 Royal Meteorological Society

Figure 1. The eight homogenous rainfall regions of South Africa. South African Weather Service rainfall stations (963) are shown as dots.

totals do not have equal variance. It is evident that there are certain regions where the distribution of the stations is sparse, but knowledge of geographical and climatological characteristics are used to ‘extrapolate’ the borders of some regions. After the regionalization, monthly and 3-monthly indices are calculated for each region (Mason, 1998), using the raw monthly rainfall station data. The regions, numbered in Figure 1, are named as follows: Region 1: South-western Cape (SWC); Region 2: South coast (SOC); Region 3: Southern KwaZulu-Natal (SKN); Region 4: Northern KwaZulu-Natal (NKN); Region 5: Lowveld (LOW); Region 6: Highveld (HGH); Region 7: Central interior (CNT) and Region 8: Western interior (WNT). 2.2.

The general circulation model

All GCM integrations were performed by the International Research Institute for Climate and Society (IRI) using the ECHAM4.5 AGCM (Roeckner et al., 1996). An ensemble of 24 runs was forced with simultaneous observed sea-surface temperatures (SSTs) (SSTs occurring during the same season as the simulated rainfall season) (Reynolds and Smith, 1994; Smith et al., 1996) from 1950 to present. The SST data were made into daily values, linearly interpolated between the monthly values (assumed to represent the value of the day in the middle of the month). At initialization, ensemble members differ from each other by one model day at the beginning of the integration. No observed atmospheric conditions are inserted into the runs at any time. The resulting AGCM fields are referred to as simulation mode fields (no forecast lead time). The GCM is successful in simulating the overall pattern of maximum rainfall over the north-east of South Africa decreasing towards the south-west, but it displaces slightly the local maximum over these regions and simulates lower rainfall totals than found in the observed climatology. Skill measures of the ECHAM4.5 used in the study can be found at: http://iri.columbia.edu/forecast/climate/skill/ Skill indvl.html. The IRI website also provides model skill maps of a number of atmospheric GCMs. These Int. J. Climatol. 29: 1535–1549 (2009) DOI: 10.1002/joc

PERFORMANCE COMPARISON OF SOME DOWNSCALING METHODS FOR SOUTH AFRICA

models perform best over the central and western parts as well as the north-eastern parts of South Africa when simulating DJF rainfall. Models generally perform poorly over the HGH regions of South Africa where predominantly thunderstorm activity contributes significantly to summer rainfall totals. 2.3. The regional climate model Dynamical downscaling utilizes high resolution regional climate models (RCMs) to derive regional climate information on a selected domain that covers an area of interest. An RCM is nested within a GCM or global analyses of observations, which provide the required large-scale conditions (the initial conditions (ICs) and the time dependant lateral boundary conditions (LBCs)) (Fennessy and Shukla, 2000). The RegCM3 is the thirdgeneration RCM developed at the Abdus Salam International Centre for Theoretical Physics (ICTP) in Italy. The first-generation National Centre for Atmospheric Research (NCAR) RegCM was built upon the NCARPennsylvania State University Mesoscale version MM4 in the late 1980s. The dynamical component of the model originated from that of the MM4, which is a compressible, finite difference model with hydrostatic balance and vertical σ -coordinates. For application of the MM4 to climate studies, a number of physics parameterizations were replaced, mostly in the areas of radiative transfer and land surface physics, which led to the first-generation RegCM. The first major upgrade of the model physics and numerical schemes was documented by Giorgi et al. (1993a,b), and resulted in the second-generation RegCM (RegCM2). The physics of the RegCM2 was based on that of the NCAR Community Climate Model 2 (CCM2), and the mesoscale model MM5 (Grell et al., 1994). The model has 18 sigma levels in the vertical and the cumulus parameterization scheme used in this experiment is Grell with the Fritsh and Chappell closure (Grell, 1993). There have been several improvements and additions to the newest version of the RegCM3. In the last few years, some new physics schemes have become available for use in the RegCM, mostly based on physics schemes of the latest version of the Community Climate Model, (CCM3) (Kiehl et al., 1996). ICs and LBC fields are derived by standard interpolation procedures from the ECHAM4.5 data grid to the RegCM3 grid. The USGS Global Land Cover Characterization and Global 30 Arc-Second Elevation datasets are used to create the terrain files. The monthly optimum interpolation sea-surface temperature (OISST) analysis is used as surface boundary conditions on a 1° grid. For the purposes of this experiment, the RegCM3 model is run with a horizontal resolution of 60 km. The model is stable at this resolution. Moreover, the available computing infrastructure at the South African Weather Service where the experiments were conducted did not allow regional model runs at a much finer horizontal resolution. The period of interest in the study is DJF. The model is allowed to run for a period of about 1 month prior to Copyright  2008 Royal Meteorological Society

1537

the period of interest for spin-up purposes (Anthes et al., 1989). The atmosphere spins up rather quickly (within a few days), meaning the regional model dynamics equilibrate with the boundary forcing. The RCM climatology is determined by a dynamical equilibrium among the LBC forcing, the model-generated forcing from the interior of the domain, and the internal model physics and dynamics (Giorgi and Bi, 2000). Reinitialization improves the simulations because the simulations do not drift downstream (Qian et al., 2003). November is used for model spin-up and the ensemble members of the National Centers for Environmental Prediction (NCEP) (Kalnay et al., 1996) driving fields are determined using a lagged (24-h) average forecasting technique (Hoffman and Kalnay, 1983). For nesting into the GCM fields, 1 November is the first simulation date, and for the reanalyses fields, the nested runs are started on 1, 2, 3 and 4 November. The selection of the domain requires careful consideration. Domains should be put well outside the area of interest so as to give the RCM solution the opportunity to respond to the internal physics and dynamics of the RCM (Seth and Giorgi, 1998). The important sources that influence the climate systems of the area of interest should be included in the domain (Giorgi and Mearns, 1999; Fennessy and Shukla, 2000). Most of the moisture that provides the rainfall in South Africa is advected from the Atlantic Ocean, the Indian Ocean and the Tropics (D’Abreton and Lindesay, 1992; Cook et al., 2004). During synoptic wet spells, moisture flux from the tropical and sub-tropical south-western Indian Ocean increases, and the low pressure system over Angola/Namibia may act as a tropical source for major synoptic rain-producing systems such as tropical temperate troughs (Washington and Todd, 1999). The domain of interest for South Africa should include Madagascar because it affects the moisture flux from the Indian Ocean into South Africa and it also influences the migration of the cyclone-like vortices (Landman et al., 2005b). With the consideration of the above-mentioned facts, Figure 2 shows the model domain to cover the area from about the equator to 40 ° S and from Greenwich to about 70 ° E, while Figure 3 shows where the grid-points of the dynamical models are located over South Africa. Four ensemble members nested in ECHAM4.5 GCM large-scale fields (6-hourly data) and four ensemble members nested in NCEP reanalysis data (6-hourly data) are completed for the DJF seasons over the 10-year period from 1991/1992 to 2000/2001. The large-scale forcing fields are from both the GCM and reanalysis data so that the performance of the two nesting systems can be compared and conclusions drawn on how the RCM is constrained by the large-scale conditions provided by the GCM. 2.4. Interpolation to stations and regions An interpolation scheme is used to interpolate GCM and RCM output to the 963 stations and to eight homogeneous regions. The technique is a superior alternative to Int. J. Climatol. 29: 1535–1549 (2009) DOI: 10.1002/joc

1538

W. A. LANDMAN ET AL.

Figure 2. RegCM3 topography (metres above sea level) and domain.

Figure 3. Grid-point allocation of the (a) ECHAM4.5 and the (b) RegCM3 to the homogeneous rainfall regions.

those interpolation functions that assign the station value only from the nearest grid point. The algorithm, based on the inverse distance weighted interpolation method, detects four grid points nearest to a given station, and assumes that the interpolating surface is influenced by the nearby points (Shepard, 1968). The contribution from each grid point is evaluated using a weighting function, which assigns the respective merit of each participant grid point according to its relative proximity to the station. The method also considers the extreme case where a station falls perfectly on the model grid point. Under this condition, it directly assigns the value to the station. 2.5. Canonical correlation analysis The statistically based method used in constructing the base-line forecast model that uses SSTs as predictors (Landman and Mason, 1999) and the model output statistics (MOS) models (Landman and Goddard, 2002) is called canonical correlation analysis (CCA). CCA, which is often used as a forecast technique, is a multivariate statistical methodology to determine optimal linear combinations of two data sets (the predictor data set, e.g. SST, and the predictand data set, e.g. observed rainfall) that are highly correlated, and is at the top of the Copyright  2008 Royal Meteorological Society

regression modelling hierarchy (Barnett and Preisendorfer, 1987). Because the predictor and the predictand fields contain a large number of highly correlated variables and few observations, it is recommended that preorthogonalization (Barnston, 1994) using standard empirical orthogonal function (EOF) analysis (Jackson, 1991) be performed. The predictor and predictand data sets are first standardized, resulting in correlation matrices on which the EOF analysis is performed. The standardization ensures that all the grid points and rainfall indices have equal opportunity to participate in the prediction process (Jackson, 1991; Barnston, 1994). EOF analysis is performed separately for each of the predictor and predictand fields. The number of modes to be retained in the analysis (i.e. those that will be used in the CCA eigenanalysis problem) is determined using forecast skill sensitivity tests. This is performed by cross-validation with varying numbers of retained predictor and predictand EOF modes. The combination producing the highest area-average correlation is used as the best estimate of the number of predictor and predictand EOF modes. For the predictor and predictand, the retained EOF modes usually explain Int. J. Climatol. 29: 1535–1549 (2009) DOI: 10.1002/joc

PERFORMANCE COMPARISON OF SOME DOWNSCALING METHODS FOR SOUTH AFRICA

about 60–80% of the variances. The truncation for the number of CCA modes retained is determined by using the Guttman–Kaiser criterion (Jackson, 1991) where only those modes having an eigenvalue greater that the average eigenvalue are retained. In most cases, this results in two CCA modes. 2.6. Model output statistics MOS (Wilks, 2006; Glahn and Lowry, 1972) applied to GCM output can improve seasonal rainfall forecasts over southern Africa (Landman and Goddard, 2002). However, the question of whether or not the selection of a particular domain configuration has a significant impact on the simulation of seasonal climate rainfall anomalies over the region has remained largely unanswered. As domain selection using dynamical downscaling techniques has a definite influence on forecast skill, investigation into the domain configuration using statistical downscaling methods is warranted. ECHAM4.5 GCM simulation (simultaneously observed SSTs serve as the lower boundary forcing) rainfall data (24 ensemble member mean) for the DJF season are used in the analysis. MOS is subsequently applied to the raw GCM rainfall data for a number of domain configurations. The performance of the MOS system is judged deterministically using the area-averaged cross-validation correlation between MOS-simulated rainfall for each homogeneous region of Figure 1 and the relevant observed rainfall index. Simulations are obtained using a multiyear-out cross-validation procedure that resulted in a 33-year test period from 1965/1966 to 1997/1998. First, an optimal linear statistical MOS model (i.e. the MOS model producing the highest averaged correlation over the summer rainfall regions) is designed for each domain. Second, the best MOS model from each domain is subsequently compared and the domain with its MOS model producing the highest average correlation is considered to be the optimal domain of those considered here. A domain representing the equatorial Pacific Ocean is also considered on its own and in combination with the optimal southern African domain. The optimal African domain produced an MOS model that outperformed the MOS model that included the equatorial Pacific Ocean domain. The ability of the GCM to adequately simulate ENSO teleconnection patterns over southern Africa has therefore been demonstrated. The area-averaged crossvalidation correlation for the summer (DJF) rainfall regions shows the domain that includes the south-western Indian Ocean to produce the best results. This domain is large enough to include the part of the south-western Indian Ocean that has an effect on southern African austral summer rainfall (Reason, 2001). In this study the performance of both the ECHAM4.5MOS model and the SST baseline model are tested over a multi-decadal time period using a 3-year-out cross-validation approach. The 10-year test period of 1991/1992 to 2000/2001 is subsequently extracted from the long cross-validated period. Copyright  2008 Royal Meteorological Society

3.

1539

Results

3.1. Homogenous regions of the GCM and RCM The aim of designing homogeneous rainfall regions is to group the rainfall stations into regions with similar interannual rainfall variability. Within each region, the mechanisms responsible for the rainfall variability should then be similar (Mason, 1998). Such a grouping should reduce some of the noise associated with individual stations and may as a result improve on simulation and forecast skill over the area of interest (Gong et al., 2003). In this section, the simulation skill over the 10-year period from 1991/1992 to 2000/2001 for each of the different simulation methods is compared. Simulation skill may be considered as an upper boundary for the various systems’ operational forecasting potential as the SSTs do not have to be simulated. GCM and RCM output data were averaged over the grid-points located within each of the regions specified in Figure 1. The resulting time series were subsequently normalized over the 10-year test period. Figure 3 shows where the grid-points of the GCM and RCM are allocated for each region. The representation of the homogeneous rainfall regions is better for the finer resolution nesting system than for the GCM. This difference in the horizontal resolution of the model may have an effect on calculating the rainfall time series representing the defined regions, but this effect is considered to be negligible. 3.2. Simulation skill levels for the 10-year period With such a small ensemble (4 members for the GCMRCM nesting), it is unrealistic to expect to be able to derive useful conclusions based on the ensemble distribution. For this reason ensemble mean values are used and simulation skill levels compared by considering Pearson (‘ordinary’) and Spearman rank correlation values. In addition to these statistics, the root-mean-squared error (RMSE) of the simulated and observed normalized rainfall time series is also considered. Unfortunately, owing to the small ensemble of the RegCM3 and the short test period of 10 years, it might not be possible to demonstrate unequivocally that one simulation method is better than the other. Notwithstanding, similar short test periods have been considered elsewhere (Kidson and Thompson, 1998; Leung and Ghan, 1999; Murphy, 1999; Bromwich et al., 2005). Moreover, the results obtained should at least be able to demonstrate the relative potential of the methods used to simulate South African summer seasonal rainfall during the 1990s. As the skill comparison may be biased when simulation methods using different ensemble sizes are compared, a Monte Carlo procedure (Wilks, 2006) was applied to the raw ECHAM4.5 and ECHAM4.5-MOS ensemble in order to randomly select four ensemble members a large number of times (500) and then calculate the average ensemble mean over the large number of iterations. In this way, the possibility of selecting the best (or worst) set of four members to be used in the skill comparisons is Int. J. Climatol. 29: 1535–1549 (2009) DOI: 10.1002/joc

1540

W. A. LANDMAN ET AL.

eliminated. The full iteration set was subsequently used to calculate 95% confidence intervals using the normal distribution. The various simulation methods used are: 1. DJF SST to simulate DJF rainfall; 2. Raw ECHAM4.5 DJF rainfall simulation data using the mean of the full ensemble of 24 members; 3. Raw ECHAM4.5 DJF rainfall simulation data using a four-member ensemble mean derived from a large number (500) of Monte Carlo iterations; 4. ECHAM4.5-MOS DJF rainfall simulation data using the mean of the full ensemble of 24 members; 5. ECHAM4.5-MOS DJF rainfall simulation data using a four-member ensemble mean derived from a large number (500) of Monte Carlo iterations; and, 6. ECHAM4.5-RegCM3 nested DJF rainfall simulation data using a four-member ensemble mean. 3.3. Regional skill levels Pearson and Spearman correlation values and RMSE values between the observed and simulated rainfall over the ten summers for each of the eight homogeneous regions obtained from the six simulation methods are shown in Figure 4. In addition to the regions, performance estimates are also shown for an area-averaged value, calculated over all of the eight regions using the Fisher Z transformation (Wilks, 2006) for the correlations only. With such a small sample it proved very difficult to obtain results that are significant at the 95% level. However, this limit is superseded by some of the models, and,

in particular, for the LOW region where the 95% level is reached for both Pearson and Spearman correlations. Moreover, the baseline skill level of the SST-rainfall statistical model is outscored, even over the far north-eastern areas (NKN and the LOW) where the baseline model produces its highest skill. There is benefit to be seen in applying MOS to the raw GCM output, as MOS and raw GCM value simulations produced at least equally skillful estimates, with the MOS producing simulations specific to the region considered. This result is in agreement with what has been found when applying a MOS procedure to an older (Landman and Goddard, 2002) and a later (Shongwe et al., 2006) version of the ECHAM GCM where MOS skill has been demonstrated to be generally higher than raw model output skill. Notwithstanding the sample size, the performance of the simulation system with the larger ensemble is generally doing better than when using only a subset of the full ensemble (raw ECHAM4.5 (24) vs raw ECHAM4.5 (4); ECHAM4.5-MOS (24) vs. ECHAM4.5-MOS (4)). This result is to be expected as improvement in skill with larger ensemble sizes has been demonstrated (e.g. Krishnamurti et al., 2000). The area-averaged values of Figure 4 show that the ECHAM4.5-RegCM3 system is marginally outscored by the small-ensemble raw GCM and MOS simulations (the nested system has lower correlation values and higher RMSE values than the other two systems). However, with the exception of NKN (Region 4), the LOW (Region 5) and the HGH (Region 6) where negative or nearzero correlations are found, the nested system produces skill levels that are higher than those obtained from the

Figure 4. DJF rainfall simulations skill presented by (a) Pearson correlations, (b) Spearman rank correlations and (c) RMSE, for each of the eight regions using the six simulation methods. Area-averaged values are presented by AVE on the graphs. Horizontal lines indicate the 90% and 95% confidence levels. Asterisks show 95% confidence intervals. Copyright  2008 Royal Meteorological Society

Int. J. Climatol. 29: 1535–1549 (2009) DOI: 10.1002/joc

1541

PERFORMANCE COMPARISON OF SOME DOWNSCALING METHODS FOR SOUTH AFRICA

ECHAM4.5 and ECHAM4.5-MOS simulations using the small ensemble (four members). The negative regional model correlations found for KwaZulu-Natal and LOW may be attributed to steep topographic gradients in the area as the RegCM3 tends to overestimate rainfall over such gradients (Kgatuke et al., 2008). However, owing to the improvement in skill with larger ensemble sizes for the raw and MOS systems, the demonstrated simulation performance of the ECHAM4.5-RegCM3 may improve over all the regions when the ensemble size is increased. There is close agreement between the verification statistics presented in Figure 4. For example, take note of the high RMSE values of the nested system for NKN, the LOW and the HGH compared with the low correlation values for the same regions and system, and the generally low RMSE values for the LOW and high correlation values for the same region, to name but two examples. To demonstrate how similar the spatial distribution of skill is, the three performance statistics are compared in Table I. This table shows the association between Pearson and Spearman correlations and Pearson correlation and RSME values over the summer rainfall regions (excluding the SWC region, which predominantly is a winter rainfall region). The good agreement between the three statistics shown in the table demonstrates the robustness of the skill patterns presented. Although the Pearson correlation is neither robust (non-linear relationships cannot be recognized) nor resistant (sensitive to outliers), it has produced similar patterns to the Spearman correlations here, which are both robust and resistant. Moreover, a two-sample Kolmogorov–Smirnov goodness-offit hypothesis test (Wilks, 2006) was also performed on the forecast and simulated data for each region over the ten summers. It is found that at the 95% confidence level, for each simulation system the forecast and observed values are drawn from the same underlying population. The spatial distribution in simulation skill is demonstrated in Figure 4 above. Figure 5 shows the performance of the simulation methods over the 10-year test period averaged over the eight regions. The time series of the figure are standardized departures. Correlation values associated with the raw ECHAM4.5 and MOS methods are high and between 0.6 (above 90% confidence) and 0.7 (above 95% confidence), with the correlation associated with the nested system 0.5, followed by the SST baseline model of 0.2. Simulated anomalies are generally overestimated during the El Ni˜no season of 1997/1998 and the La Ni˜na season of 1998/1999, whereas the wet conditions during 1995/1996 are underestimated. The rainfall during the 1999/2000 La Ni˜na season is skillfully simulated by

Area−Averaged Rainfall Indices for REGIONS OBS SST Raw ECHAM (24) ECHAM−MOS (24) ECHAM−MOS (4) ECHAM−RCM (4)

3

2

1

0

−1

−2 1992

1994

1996

1998

2000

Figure 5. Simulated area-averaged (over eight regions) and then standardized DJF rainfall anomalies over the 10-year test period. The thick black line shows the observed normalized DJF rainfall.

most of the systems, especially by the ECHAM4.5-MOS system. The skill of the nested system may be adversely affected by the large-scale driving fields (Chouinard et al., 1994). The RegCM3 is for this reason also nested in the large-scale driving fields obtained from the NCEP reanalysis data set and its area-averaged simulation is compared with that of the GCM-RCM system. The two pairs of RegCM3 simulations compare well during the first half of the test period, but differences in nestedsystem performance are seen thereafter (not shown). The NCEP nested system produced area-averaged simulations that show a high correlation with that observed (0.8), and does not produce the large negative rainfall anomaly of 1997/1998 that is found with the ECHAM4.5-RegCM3 system. The NCEP nested system also produced a much better rainfall estimation for the two La Ni˜na seasons of 1999/2000 and 2000/2001. These results suggest that improvement in GCM forecast skill might lead to improved nested simulations.

Table I. Ordinary correlation values between Pearson correlations and Spearman rank correlations and between Pearson correlations and RMSE values, associated with each simulation system (SST, GCM with 24 and 4 members, MOS with 24 and 4 members and the nested system) calculated over the summer rainfall regions.

Pearson versus Spearman Pearson versus RMSE

SST

ECHAM(24)

ECHAM(4)

MOS(24)

MOS(4)

RCM

0.8935 −0.9990

0.8655 −0.9982

0.9035 −0.9980

0.9439 −0.9987

0.9601 −0.9987

0.9174 −0.9974

Copyright  2008 Royal Meteorological Society

Int. J. Climatol. 29: 1535–1549 (2009) DOI: 10.1002/joc

1542

W. A. LANDMAN ET AL.

3.4. Station skill levels Although spatial aggregation of model output may improve on model performance, grouping large numbers of stations together obscures some of the characteristics of the simulations or forecasts unique to particular stations (Taylor and Leslie, 2005). The simulation skill over the 10-year period from 1991/1992 to 2000/2001 for each of the different simulation methods is again assessed, but for the 963 stations instead of the eight regions. In addition, the various above-mentioned simulation methods are also used here to describe upper limits for the various systems’ operational forecasting potential for individual stations. Area-average Pearson correlation, Spearman rank correlation (both with Fisher Z transformation) and RMSE are calculated over the 963 stations. Figure 6 shows the results: almost identical area-averaged values are obtained for both the Pearson and Spearman correlations,

and a reverse association as again found between the correlations and RMSE values. As with Table I, Table II shows the association between Pearson and Spearman correlations and Pearson correlation and RSME values over the 963 stations. The good agreement between the three statistics shown in the table demonstrates once again the robustness of the skill patterns. As these patterns are so similar, the remainder of the paper will only discuss Pearson correlation fields. The description of the various simulation systems’ performance is described here in terms of their respective status in the hierarchy of the complexity of the systems. Complexity is defined here according to the number of tiers involved in the simulation system as well as the computing requirements to perform the simulation tiers. For example, a simulation system that uses only SST as predictor is less complex and hence has a lower status in the modelling hierarchy than a system involving post-processing of GCM large-scale fields. Moreover, when the same number of simulation tiers is involved, a regional modelling approach requires more computing resources than a statistical MOS approach and hence has a higher status here. The systems are next discussed according to their status, and skill differences are calculated by subtracting skill of the lower status system from the skill of the higher status system. Positive skill patterns therefore indicate that the more complex system outscores the less complex one. Based on the above definition of model hierarchy, the simulation methods are ranked from the highest to the lowest status as follows: 1. 2. 3. 4.

Figure 6. Area-averaged (a) Pearson correlation, (b) Spearman rank correlation and (c) RMSE between the observed and simulated DJF rainfall of the 963 stations. Asterisks show the 95% confidence intervals. Simulation systems used are SST, sea-surface temperatures; E24, ECHAM4.5 (24 members); E4, ECHAM4.5 (four members); M24, MOS (24 members); M4, MOS (four members) and RCM, ECHAM4.5-RegCM3.

GCM-RCM; GCM-MOS; Raw GCM; and, SST as predictor.

The station correlations between the simulations from the six systems and the observed rainfall are described here. The system with the highest status in the hierarchy is the GCM-RCM system, and its correlation map is shown in Figure 7. In general, the correlation values are low, with the majority of the station correlations less than 0.2. The low skill found here is in contrast to the results found for simulating the regional rainfall, where some of the regions had correlation values higher than 0.4. This result demonstrates the advantage that can sometimes occur when spatially aggregating the data. The GCM-MOS system (full ensemble) shows more

Table II. Ordinary correlation values between Pearson correlations and Spearman rank correlations and between Pearson correlations and RMSE values, associated with each simulation system (SST, GCM with 24 and four members, MOS with 24 and four members, and the nested system) calculated over the 963 rainfall stations.

Pearson versus Spearman Pearson versus RMSE

SST

ECHAM(24)

ECHAM(4)

MOS(24)

MOS(4)

RCM

0.8643 −0.9942

0.8932 −0.9909

0.9065 −0.9681

0.9025 −0.9928

0.9162 −0.9966

0.8229 −0.9938

Copyright  2008 Royal Meteorological Society

Int. J. Climatol. 29: 1535–1549 (2009) DOI: 10.1002/joc

PERFORMANCE COMPARISON OF SOME DOWNSCALING METHODS FOR SOUTH AFRICA

1543

Figure 7. Correlations between the ECHAM4.5-RegCM3 DJF rainfall simulation (four-member mean) and observed DJF rainfall over the 10-year test period. Correlation values less than 0 are masked out.

Figure 9. Correlations between the ECHAM4.5-MOS DJF rainfall simulation (24-member mean) and observed DJF rainfall over a 44-year cross-validation (3-year-out approach) period. Correlation values less than 0 are masked out.

encouraging results (Figure 8). The WNT and northeastern parts have the highest correlation values, but low values are found over the HGH region, which is in agreement with the regional skill assessment (3.3). Spatial correlation analysis of the GCM-MOS system using only four members produced almost identical patterns (not shown), although the correlation values on average are lower than that of the full ensemble of 24 members. Figure 8 also corresponds very well with the ECHAM4.5-MOS correlation patterns obtained over a 44-year cross-validation (3-year-out approach) period shown in Figure 9. Both test periods show maximum skill over the north-eastern interior, and central and WNT regions. Notwithstanding the higher skill found over Region 4 (Figure 1; NKN) during the 1990s (Landman and Goddard, 2002), the good agreement between the correlation patterns of the two test periods confirms a robust relationship between forecast and observed anomalies. The robustness of the skill pattern suggests that the skill levels associated with the GCM demonstrated here for the 10-year period are applicable over much longer test

periods. The raw ECHAM4.5 output (Figure 10) shows similar correlation patterns to the ECHAM4.5-MOS system of Figure 8. However, the raw GCM output produces higher skill over the south and over far north-eastern parts. Low skill is found in between. It seems that the MOS makes its best contribution over the western parts of the country, but for some isolated areas the MOS causes the simulation skill of the raw GCM output to deteriorate. The results of the raw ECHAM4.5 output using a four-member ensemble are similar to the correlation patterns shown in Figure 10, but with lower values (not shown) owing to the smaller ensemble size. Simulation skill obtained from the baseline system (Figure 11) is generally lower than the skill associated with the statistical post-processed simulations and the raw GCM output. The best skill is found over the far north-eastern parts and along the coastal regions of KwaZulu-Natal where the GCM and GCM-MOS systems also perform well. The figures presented earlier show the associations between simulated rainfall and observed rainfall and

Figure 8. Correlations between the ECHAM4.5-MOS DJF rainfall simulation (24-member mean) and observed DJF rainfall over the 10-year test period. Correlation values less than 0 are masked out.

Figure 10. Correlations between the raw ECHAM4.5 DJF rainfall simulation (24-member mean) and observed DJF rainfall over the 10-year test period. Correlation values less than 0 are masked out.

Copyright  2008 Royal Meteorological Society

Int. J. Climatol. 29: 1535–1549 (2009) DOI: 10.1002/joc

1544

W. A. LANDMAN ET AL.

GCM-MOS simulation skill is subtracted from GCMRCM values because the GCM-RCM has a higher rank. Positive correlation differences would then provide evidence that the more complex, higher ranked system is performing better than the less sophisticated system. Six cases are presented here: 1. 2. 3. 4. 5. 6.

Figure 11. Correlations between the baseline model (using SSTs as predictor) and observed DJF rainfall over the 10-year test period. Correlation values less than 0 are masked out.

present evidence that the highest skill is found over the north-eastern, and central and western parts. Skill over the HGH is the lowest. These results are in agreement with the results for the regional rainfall simulations (3.3). The GCM-MOS results presented here are not a function of a particular GCM. MOS applied to 42 years of DEMETER (Palmer et al., 2004) data, and in particular DJF forecast fields produced by the UKMO fully coupled system, show correlation skill patterns over South Africa (not shown) similar to the ECHAM4.5-MOS simulations presented here. In addition to the similarities in the spatial distribution of the correlation fields, additional verification parameters such as linear error in probability space (LEPS) (Potts et al., 1996) and relative operating characteristic (ROC) scores (Mason and Graham, 2002) show patterns similar to the correlation fields (not shown). Therefore, as stated earlier, the correlation maps presented here should be considered good approximations of model performance spatially. These skill patterns also show a strong association with the spatial pattern obtained when correlating DJF Nino3.4 SST and DJF South African rainfall. The robustness of the skill patterns is therefore a result of the equatorial Pacific being a strong forcing in both statistical (Landman and Mason, 1999; Landman and Goddard, 2002) and physical models (Goddard and Graham, 1999) when simulating seasonal rainfall over South Africa. Although the simulation systems are in agreement with regard to where the best skill is found, the main purpose of this paper is to compare the skill levels of the various downscaling systems. The next section shows maps of the correlation differences when comparing the simulation systems. 3.5.

Station skill differences

This section presents correlation difference maps for the simulation systems. Lower ranked system (lower in the hierarchy described earlier) correlations are subtracted from higher ranked system correlations. For example, Copyright  2008 Royal Meteorological Society

GCM-RCM minus GCM-MOS; GCM-RCM minus Raw GCM; GCM-RCM minus baseline model; GCM-MOS minus Raw GCM; GCM-MOS minus baseline model; and, Raw GCM minus baseline model.

The most complex simulation system considered here involves the nesting of the RCM into the GCM largescale fields. Figure 12(a) shows the correlation differences between the ECHAM4.5-RegCM3 system and the ECHAM4.5-MOS system using the full ensemble (24 members). Only isolated areas of positive correlation differences are found, with the largest concentration of positive differences restricted to the HGH and central parts. This pattern is repeated in the analysis of Monte Carlo-generated correlation (four-member ensemble) and shows that for the most part the statistical downscaling outscores the dynamical downscaling method. Similarly, comparing the ECHAM4.5-RegCM3 with the raw ECHAM4.5 skill levels shows that the greater benefit of using the GCM-RCM system compared to the raw GCM data is again mainly restricted to the HGH and western areas; the least benefit is seen over the far southern parts. Notwithstanding the GCM-RCM’s poor showing against these other systems, it outscores the baseline model over the larger part of South Africa (Figure 12(b)), except over the far north-eastern parts where high skill is found using the baseline model (Figure 11). The GCM-MOS system generally produces higher skill than the raw GCM simulations (Figure 12(c)), with the exception of the southern coastal and adjacent areas and the far north-eastern parts. In addition, the ECHAM4.5MOS system also outscores the baseline model over the larger part of South Africa, excluding parts of the far north-eastern regions where SSTs produce high levels of skill (Figure 12(d)). The baseline model is similarly outscored using raw ECHAM4.5 simulations (not shown). Except for the GCM–RCM system, the higher ranked systems seem to outscore the lower ranked systems over the larger part of the country. This result is in agreement with the results found for the regions (Figure 4) where on average the RCM system did not outscore the GCM or MOS systems. Skill levels (correlation) of areaaveraged simulations for the 963 stations (Figure 13) show results similar to Figure 5. Here, correlation values between area-averaged observed and simulated time series over the 10 years associated with the MOS and raw ECHAM4.5 systems are high and between 0.4 and 0.5. The correlation associated with the SST baseline model is only 0.2, followed by the nested system of Int. J. Climatol. 29: 1535–1549 (2009) DOI: 10.1002/joc

PERFORMANCE COMPARISON OF SOME DOWNSCALING METHODS FOR SOUTH AFRICA (a)

(b)

(c)

(d)

1545

Figure 12. Correlation differences between the (a) ECHAM4.5-RegCM3 system and the ECHAM4.5-MOS system (24-member mean), the (b) ECHAM4.5-RegCM3 system and the baseline model (using SSTs to simulate rainfall), the (c) ECHAM4.5-MOS and the raw ECHAM4.5 systems (24-member mean), and the (d) ECHAM4.5-MOS (24-member mean) and the baseline system (using SSTs to simulate rainfall) over the 10-year test period. Negative values are masked out.

0.1. The correlation value associated with the raw (24 member) ECHAM4.5 simulations is 0.4, and that of the ECHAM4.5-MOS simulations 0.5, which provides evidence that the MOS has generally improved raw model output. As with the simulations for regional rainfall, simulated anomalies are overestimated during the El Ni˜no season of 1997/1998 and the La Ni˜na season of 1998/1999, whereas the wet conditions during 1995/1996 are underestimated. This simulation results for 1997/1998 and 1998/1999 are not too surprising as GCM performance over South Africa is strongly linked with the equatorial Pacific Ocean. The dominant forcing from this ocean area is further demonstrated in Figure 14, which shows that area-averaged UKMO-MOS forecasts (from the DEMETER set) and ECHAM4.5-MOS used here also produce strong negative (positive) rainfall anomalies during the 1997/1998 (1998/1999) El Ni˜no (La Ni˜na) season, as well as strong positive anomalies during the 1999/2000 La Ni˜na season. Notwithstanding the poor temporal skill of the GCMRCM system (correlation = 0.1), simulation produced with the nested approach was skillful during the wet year Copyright  2008 Royal Meteorological Society

of 1995/1996 when the other systems failed to capture the wet anomaly. During this season, tropical cyclone Bonita made landfall over southern Africa, inundating the eastern parts of South Africa with rain. RCMs are able to capture the evolution and tracks of tropical-cyclone like vortices over the south-western Indian Ocean, and this was also found to be the case with an older version of the RegCM3, the RegCM2 (Landman et al., 2005b). The rainfall during the 1999/2000 La Ni˜na season is skillfully simulated, especially by the ECHAM4.5-MOS system. During February of 2000, tropical cyclone Eline caused devastating floods over southern Africa (Reason and Keibel, 2004), including South Africa, and although the RCM produced a positive rainfall anomaly, the anomaly was underestimated. Statistical correction should be able to improve on nested simulations, and this notion is addressed hence. A 1-year-out cross-validation procedure using CCA is applied over the 10-year test period by relating the simulated 850 hPa geopotential height fields of the ECHAM4.5-RegCM3 system to station rainfall. Area-averaged indices are again calculated and this MOS Int. J. Climatol. 29: 1535–1549 (2009) DOI: 10.1002/joc

1546

W. A. LANDMAN ET AL. Area−Averaged Rainfall Indices for STATIONS OBS SST Raw ECHAM (24) ECHAM−MOS (24) ECHAM−MOS (4) ECHAM−RCM (4)

3

2

1

1

0

0

−1

−1

−2

−2 1994

1996

OBS ECHAM4.5 UKMO

3

2

1992

1998

2000

Figure 13. Simulated area-averaged (over 963 stations) and then standardized DJF rainfall anomalies over the 10-year test period. The thick black line shows the observed normalized DJF rainfall.

procedure improves the correlation value from 0.1 to 0.5. Notwithstanding this evidence that statistically correcting nested simulations also improve skill, a period longer than 10 years of model output is required to test for the robustness of these prediction equations. In addition, a 1-year-out cross-validation approach could inflate skill levels, but owing to the limited period of 10 years used here, a multi-year-out cross-validation approach was not feasible. It should also be tested if statistically correcting nested simulations outperform statistically corrected GCM simulations over an appropriate testing period spanning several decades.

4.

Area−Averaged Rainfall Indices for STATIONS

Discussion and conclusions

The study has investigated the performance of a number of simulation systems (no forecast lead-time) of varying complexity. These systems’ performance is tested for the December-January-February (DJF) rainfall for both homogeneous rainfall regions and for 963 rainfall stations, and compared with each other over a 10-year test period from 1991/1992 to 2000/2001. Although this test period could be too short to unequivocally demonstrate which simulation method is the best, skill maps of the GCM-MOS system are found to be similar to skill maps obtained over much longer periods (>40 years). Therefore, the conclusions drawn could be applicable over periods longer than the 10 years presented here. The reader should also take note that seasonal climate prediction is Copyright  2008 Royal Meteorological Society

1992

1994

1996

1998

2000

Figure 14. Simulated area-averaged (over 963 stations) and then standardized DJF rainfall anomalies over the 10-year test period for the ECHAM4.5 and UKMO systems only. The thick black line shows the observed normalized DJF rainfall.

at the farthest extent of current modelling capabilities. Better skill than random chance is often considered an accomplishment in this type of modelling system. For the most part the physical model simulations method outscores the empirical method that uses SST anomalies to simulate summer rainfall. Such an empirical forecasting system was the first of its kind in operational seasonal forecasting in the South Africa Weather Service (Landman and Mason, 1999). The result that more sophisticated methods outscore the earlier simple one shows the progress that has been made in seasonal forecasting in the region. Notwithstanding this progress, the north-eastern interior remains an area where SSTs are producing high forecast skill, although the GCM and GCM-MOS systems also produce high skill there. It should be worthwhile to investigate whether or not a combination of physical model output and statistical forecast models that use SSTs as predictors can further improve skill (Coelho et al., 2006) over South Africa. Except for a few isolated geographical areas, there are general improvements in skill when output from GCMs is statistically post-processed. Moreover, statistically corrected simulations generally outscore simulations from a GCM-RCM system. However, applying MOS to the output from RCMs improves these simulations slightly. The superiority of the MOS simulations over that of the RCM presented here suggests that operational centres Int. J. Climatol. 29: 1535–1549 (2009) DOI: 10.1002/joc

PERFORMANCE COMPARISON OF SOME DOWNSCALING METHODS FOR SOUTH AFRICA

such as National Meteorological and Hydrological Services, which do not have the necessary capacity to run nested models, especially in operational forecast mode, should expend available resources to investigate the feasibility of building MOS models to produce operational seasonal forecasts for their regions of interest. Moreover, MOS models can also be used to build applications models such as those used to predict streamflow (Landman et al., 2001b). Data and software to develop MOS models can, for example, be downloaded from the website of the IRI (http://iri.columbia.edu). In addition to statistical post-processing improving skill, this paper demonstrates the advantages of increasing the number of ensemble members, manifesting what has been found before (e.g. Krishnamurti et al., 2000). As raw GCM and GCM-MOS skill improves with an increased number of ensemble members, it can also be expected that the GCM-RCM skill will improve. Moreover, as multiple realizations are not required to describe the internal variability of the RegCM3 (Kgatuke et al., 2008) the increase in the ensemble size of the nested system would mainly contribute to skill improvement. Evidence was also presented that nesting the RegCM3 in NCEP fields improved the skill. Therefore, improvement in GCMs will lead to further improvement in the skill of the nested system. Simulation performance of the systems discussed here is to a large extent a function of the equatorial Pacific SSTs. As mentioned earlier, dry anomalies were simulated by the GCMs and GCM-MOS systems for the 1997/1998 DJF season, and wet anomalies for the 1998/1999 and 1999/2000 seasons. In addition to these years, dry anomalies were simulated over South Africa during the 1991/1992 and 1994/1995 El Ni˜no seasons and wet anomalies during the 1995/1996 La Ni˜na season. Notwithstanding this apparent ENSO signal in the model simulations, the models were successful in simulating the anomalously dry anomalies during the weak La Ni˜na season of 2000/2001. Ocean areas near South Africa have been shown to affect that country’s seasonal rainfall variability during summer (e.g. Mason, 1995), and atmospheric GCM experiments have shown that Indian Ocean SST forcing could influence the circulation across southern Africa (e.g. Washington and Preston, 2006). However, experiments where atmospheric GCMs are being forced with prescribed Indian Ocean SST may be misleading as atmospheric variability over the Indian Ocean is likely to be remotely forced by the equatorial Pacific and not by local SSTs, and that physical models perform better when air–sea coupling over the Indian Ocean is considered (Copsey et al., 2006). In fact, air–sea coupling in the Indian Ocean is necessary for simulating, for example, the Indian monsoon–ENSO relationship and for studying the influence of the Indian Ocean on the ENSO variability (Yeh et al., 2007). Even if circulations are being properly simulated through prescribed SSTs, which force atmospheric GCMs, very little predictability or persistence exists over off-equatorial Indian Ocean (e.g. Landman and Mason, 2001) areas, which have been demonstrated Copyright  2008 Royal Meteorological Society

1547

with atmospheric GCMs to drive atmospheric circulation over southern Africa (e.g. Reason, 2001). Therefore, operational forecasts may only start to benefit fully from signals other than ENSO through fully coupled models, such as the Climate Forecasting system at NCEP (Saha et al., 2006). However, even fully coupled models have low skill in predicting SST in the mid-latitudes (Saha et al., 2006). Poor skill in simulating these SSTs could be a reason even the fully coupled UKMO forecasts of Figure 14 are similar to the rainfall simulations produced by the atmosphere models during the strong El Ni˜na and La Ni˜na events of 1997/1998 and 1998/1999, respectively. Proper investigation into the added benefit of using fully coupled models over two-tiered systems for South Africa needs to be conducted. The potential for using dynamical and statistical downscaling methods and their combination for forecasting South African seasonal regional rainfall variability in an operational environment has been demonstrated. In addition to expanding on the number of ensemble members, the test period of 10 years should be increased in order to test the robustness of the results presented here. An increased ensemble size can also be used to test the probabilistic skill levels of these systems and how a large ensemble can be used in an operational seasonal forecasting environment that demands a description of forecast uncertainties. Acknowledgements The ECHAM4.5 6-hourly data made accessible by Dr L. Sun of the IRI is acknowledged with gratitude. Suggestions to address the objectives of the study and for general discussions by Drs S. Mason and L. Goddard of the IRI are appreciated. This work was sponsored by the Water Research Commission of South Africa (project K1334/5). References Anthes RA, Kuo Y-H, Hsie E-Y, Low-Nam S, Bettge TW. 1989. Estimation of skill and uncertainty in regional numerical models. Quarterly Journal of the Royal Meteorological Society 115: 763–806. Barnston AG. 1994. Linear statistical short-term climate predictive skill in the Northern hemisphere. Journal of Climate 7: 1513–1564. Barnett TP, Preisendorfer RW. 1987. Origins and levels of monthly and seasonal forecast skill for United States air temperature determined by canonical correlation analysis. Monthly Weather Review 115: 1825–1850. Bromwich DH, Bai L, Bjarnason GG. 2005. High resolution regional climate simulations over Iceland using polar MM5. Monthly Weather Review 133: 3527–3547. Chouinard C, Mailhot J, Mitchell HL, Staniford A, Hogue T. 1994. The Canadian regional data assimilation system: operational and research applications. Monthly Weather Review 122: 1306–1325. Coelho CAS, Stephenson DB, Balmaseda M, Doblas-Reyes FJ, van Oldenborgh GJ. 2006. Toward and integrated seasonal forecasting system for South America. Journal of Climate 19: 3704–3721. Cook C, Reason CJC, Hewiston BC. 2004. Wet and dry spells within particularly wet and dry summers in the South African summer rainfall region. Climate Research 26: 17–31. Int. J. Climatol. 29: 1535–1549 (2009) DOI: 10.1002/joc

1548

W. A. LANDMAN ET AL.

Copsey D, Sutton R, Knight JR. 2006. Recent trends in sea level pressure in the Indian Ocean region. Geophysical Research Letters 33: L19712, DOI: 10.1029/2006GL027175. D’Abreton PC, Lindesay JA. 1992. Water vapour transport over southern Africa during wet and dry early and late summer months. International Journal of Climatology 13: 151–170. Fennessy MJ, Shukla J. 2000. Seasonal prediction over North America with a regional model nested in a global model. Journal of Climate 13: 2605–2627. Giorgi F. 1990. Simulation of regional climate using a limited area model nested in a general circulation model. Journal of Climate 3: 941–964. Giorgi F, Bi X. 2000. A study of the internal variability of a regional climate model. Journal of Geophysical Research 105: 29503–29521. Giorgi F, Mearns LO. 1999. Introduction to special section: regional climate modelling revisited. Journal of Geophysical Research 104: 6335–6351. Giorgi F, Marinucci MR, Bates GT. 1993a. Development of a secondgeneration regional climate model (RegCM2). Part I: Boundarylayer and radiative transfer processes. Monthly Weather Review 121: 2794–2813. Giorgi F, Marinucci MR, Bates GT, De Canio G. 1993b. Development of a second-generation regional climate model (RegCM2). Part II: Convective processes and assimilation of lateral boundary conditions. Monthly Weather Review 121: 2814–2832. Giorgi F, Mearns LO, Shields C, Mayer L. 1996. A regional model study of the importance of local versus remote controls of the 1988 drought and the 1993 flood over the central United States. Journal of Climate 9: 1150–1162. Glahn HR, Lowry DA. 1972. The use of model output statistics (MOS) in objective weather forecasting. Journal of Applied Meteorology 11: 1203–1211. Goddard LM, Graham NE. 1999. The importance of the Indian Ocean for simulating rainfall anomalies over eastern and southern Africa. Journal of Geophysical Research 104: 19099–19116. Goddard L, Mason SJ, Zebiak SE, Ropelewski CF, Basher R, Cane MA. 2001. Current approaches to seasonal-to-interannual climate predictions. International Journal of Climatology 21: 1111–1152. Gong X, Barnston AG, Ward MN. 2003. The effect of spatial aggregation on the skill of seasonal precipitation forecasts. Journal of Climate 16: 3059–3071. Grell G. 1993. Prognostic evaluation of assumptions used by cumulus parameterization. Monthly Weather Review 121: 764–787. Grell GA, Dudhia J, Stauffer DR. 1994. A description of the fifthgeneration penn state/NCAR mesoscale model (MM5). Technical report: TN-398+STR. National Center for Atmospheric Research: Boulder, CO, USA; 121 pp. Hoffman RN, Kalnay E. 1983. Lagged average forecasting, an alternative to Monte Carlo forecasting. Tellus 35A: 100–118. Jackson JE. 1991. A User’s Guide to Principal Components. Wiley: New York. Joubert AM, Hewitson BC. 1997. Simulating present and future climate changes of southern Africa using general circulation models. Progress in Physical Geography 21: 51–78. Joubert AM, Katzfey JJ, McGregor JL, Nguyen KC. 1999. Simulating midsummer climate over southern Africa using a nested regional climate model. Journal of Geophysical Research 104: 19015–19025. Kalnay E, Kanamitsu M, Kistler R, Collins W, Deaven D, Gandin L, Iredell M, Saha S, White G, Woollen J, Zhu Y, Chelliah M, Ebisuzaki W, Higgins W, Janowiak J, Mo KC, Ropelewski C, Wang J, Leetmaa A, Reynolds R, Jenne R, Joseph D. 1996. The NCEP/NCAR 40-year reanalysis project. Bulletin of the American Meteorological Society 77: 437–471. Kgatuke MM, Landman WA, Beraki A, Mbedzi MP. 2008. The internal variability of the RegCM3 over South Africa. International Journal of Climatology 28: 505–520. Kidson JW, Thompson CS. 1998. A comparison of statistical and model-based downscaling techniques for estimating local climate variations. Journal of Climate 11: 735–753. Kiehl J, Hack J, Bonan G, Boville B, Breigleb B, Williamson D, Rasch P. 1996. Description of the NCAR community climate model (CCM3). (NCAR) Technical Report: TN-420 + STR. National Centre for Atmospheric Research: Boulder, CO, USA; 152 pp. Krishnamurti TN, Kishtawal CM, Zang Z, LaRow T, Bachiochi D, Williford E, Gadgil S, Surendran S. 2000. Multimodel ensemble Copyright  2008 Royal Meteorological Society

forecasts for weather and seasonal climate. Journal of Climate 13: 4196–4216. Landman WA, Botes S, Goddard L, Shongwe M. 2005a. Assessing the predictability of extreme rainfall seasons over southern Africa. Geophysical Research Letters 32: L15809, DOI: 10.1029/2005GL023965. Landman WA, Seth A, Camargo SJ. 2005b. The effect of regional climate model domain choice on the simulation of tropical cyclonelike vortices in the southwestern Indian Ocean. Journal of Climate 10: 1263–1274. Landman WA, Goddard L. 2002. Statistical recalibration of GCM forecasts over southern Africa using model output statistics. Journal of Climate 15: 2038–2055. Landman WA, Goddard L. 2005. Predicting southern African summer rainfall using a combination of MOS and perfect prognosis. Geophysical Research Letters 32: L15809, DOI: 10.1029/2005GL022910. Landman WA, Mason SJ. 1999. Operational long-lead prediction of South African rainfall using canonical correlation analysis. International Journal of Climatology 19: 1073–1090. Landman WA, Mason SJ. 2001. Forecasts of near-global sea surface temperatures using canonical correlation analysis. Journal of Climate 14: 3819–3833. Landman WA, Mason SJ, Tyson PD, Tennant WJ. 2001a. Retro-active skill of multi-tiered forecasts of summer rainfall over southern Africa. International Journal of Climatology 21: 1–19. Landman WA, Mason SJ, Tyson PD, Tennant WJ. 2001b. Statistical downscaling of GCM simulations to streamflow. Journal of Hydrology 252: 221–236. Landman WA, Tennant WJ. 2000. Statistical downscaling of monthly forecasts. International Journal of Climatology 20: 1521–1532. Leung LR, Ghan SJ. 1999. Pacific northwest climate sensitivity simulated by a regional climate model driven by a GCM. Part I: Control simulations. Journal of Climate 12: 2010–2030. Mason SJ. 1995. Sea-surface temperature – South African rainfall associations, 1910–1989. International Journal of Climatology 15: 119–135. Mason SJ. 1998. Seasonal forecasting of South African rainfall using a non-linear discriminant analysis model. International Journal of Climatology 18: 147–164. Mason SJ, Graham NE. 2002. Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation. Quarterly Journal of the Royal Meteorological Society 128: 2145–2166. Mason SJ, Joubert AM. 1997. Simulating changes in extreme rainfall over southern Africa. International Journal of Climatology 17: 291–301. Mason SJ, Jury MR. 1997. Climate variability and change over southern Africa: a reflection on underlying processes. Progress in Physical Geography 21: 23–50. Murphy J. 1999. An evaluation of statistical and dynamical techniques for downscaling local climate. Journal of Climate 12: 2256–2284. Palmer TN, Alessandri A, Anderson U, Cantelaube P, Davey M, D´el´ecluse P, D´equ´e M, D´ıez E, Doblas-Reyes FJ, Feddersen H, Graham R, Gualdi S, Gu´er´emy J-F, Hagedorn R, Hoshen M, Keenlyside N, Latif M, Lazar A, Maisonnave E, Marletto V, Morse AP, Orfila B, Rogel P, Terres J-M, Thomson MC. 2004. Development of a European multimodel ensemble system for seasonal-to-interannual prediction (DEMETER). Bulletin of the American Meteorological Society 85: 853–872, DOI: 10.1175/ BAMS-85-6-853. Potts JM, Folland CK, Jolliffe IT, Sexton D. 1996. Revised ‘LEPS’ scores for assessing climate model simulations and long-range forecasts. Journal of Climate 9: 34–53. Qian Jendash H, Seth A, Zebiak S. 2003. Reinitialised versus continuous simulations for regional climate downscaling. Monthly Weather Review 131: 2857–2873. Reason CJC. 2001. Subtropical Indian Ocean SST dipole events and southern African rainfall. Geophysical Research Letters 28: 2225–2227. Reason CJC, Keibel A. 2004. Tropical cyclone Eline and its unusual penetration and impacts over the southern African mainland. Weather and Forecasting 19: 789–805. Reynolds RW, Smith TM. 1994. Improved global sea surface temperature analyses using optimum interpolation. Journal of Climate 7: 929–948. Roeckner E, Arpe K, Bengtsson L, Christoph M, Claussen M, D¨umenil L, Esch M, Giorgetta M, Schlese U, Schulzweida U. 1996. The Int. J. Climatol. 29: 1535–1549 (2009) DOI: 10.1002/joc

PERFORMANCE COMPARISON OF SOME DOWNSCALING METHODS FOR SOUTH AFRICA atmospheric general circulation model ECHAM4: Model description and simulation of present-day climate. Report No. 218. Max-PlanckInstit¨ut f¨ur Meteorologie: Hamburg, 90. Saha S, Nadiga S, Thiaw C, Wang J, Wang W, Zhang Q, Van den Dool HM, Pan H-L, Moorthi S, Behringer D, Stokes D, Pe˜na M, Lord S, White G, Ebisuzaki W, Peng P, Xie P. 2006. The NCEP climate forecast system. Journal of Climate 19: 3483–3517. Seth A, Giorgi F. 1998. The effects of domain choice on summer precipitation simulation and sensitivity in a regional climate model. Journal of Climate 11: 2698–2712. Shepard D. 1968. A two-dimensional interpolation function for irregularly-spaced data. Proceedings of the 23rd ACM National Conference: New York, USA; 517–524. Shongwe ME, Landman WA, Mason SJ. 2006. Performance of recalibration systems of GCM forecasts for southern Africa. International Journal of Climatology 26: 1567–1585. Smith TM, Reynolds RW, Livezey RE, Stokes DC. 1996. Reconstruction of historical sea surface temperatures using empirical orthogonal functions. Journal of Climate 9: 1403–1420.

Copyright  2008 Royal Meteorological Society

1549

Taylor AA, Leslie LM. 2005. A single-station approach to model output statistics temperature forecast error assessment. Weather and Forecasting 20: 1006–1020. Tyson PD, Preston-Whyte RA. 2000. The Weather and Climate of Southern Africa. Oxford University Press: Cape Town. Washington R, Preston A. 2006. Extreme wet years over southern Africa: Role of Indian Ocean sea surface temperatures. Journal of Geophysical Research 111: D15104, DOI: 10.1029/2005JD006724. Washington R, Todd M. 1999. Tropical-temperate links in southern African and southwest Indian Ocean satellite-derived daily rainfall. International Journal of Climatology 19: 1601–1616. Wilby RL, Wigley TML. 1997. Downscaling general circulation model output: a review of methods and limitations. Progress in Physical Geography 21: 530–548. Wilks DS. 2006. Statistical Methods in the Atmospheric Sciences, 2nd edn. Academic Press: San Diego, CA. Yeh S-W, Wu R, Kirtman BP. 2007. Impact of the Indian Ocean on ENSO variability in a hybrid coupled model. Quarterly Journal of the Royal Meteorological Society 133: 445–457, DOI: 10.1002/qj.18.

Int. J. Climatol. 29: 1535–1549 (2009) DOI: 10.1002/joc