Asia-Pacific J. Atmos. Sci., 47(2), 199-211, 2011 DOI:10.1007/s13143-011-0009-8
Development of Updateable Model Output Statistics (UMOS) System for Air Temperature over South Korea Jeon-Ho Kang1, Myoung-Seok Suh1, Ki-Ok Hong1 and Chansoo Kim2 1
Department of Atmospheric Science, Kongju National University, Kongju, Korea Department of Applied Mathematics, Kongju National University, Kongju, Korea
2
(Manuscript received 7 August 2009; revised 12 June 2010; accepted 12 December 2010) © The Korean Meteorological Society and Springer 2011
Abstract: In this study, a Updateable Model Output Statistics (UMOS) system has been developed for the forecast of 3-h temperature over South Korea using two significantly different models’ (Regional Data Assimilation and Prediction System (RDAPS) and Korea Meteorological Administration (KMA) Weather Research and Forecasting (WRF) model (KWRF)) outputs based on the Canadian UMOS system (Wilson and Vallee, 2002; 2003). The UMOS system is designed to consider the local climatology and the model's forecasting skills. The 20 most frequently selected potential predictors for each season, station, and forecast projection time from the 67 potential predictors of the Model Output Statistics (MOS) system, were used as potential predictors of the UMOS system. The UMOS equations are developed by a weighted blending of the new and old model data, with weights chosen to emphasize the new model data while including enough old model data in the development to ensure stable equations and a smooth transition to dependency on the new model. The UMOS equations were updated regularly at a predefined time interval to consider the changes of covariance structure between the new model output and observations as the new model data increase. The validation results showed that seasonal mean bias, Root Mean Square Error (RMSE), and correlation coefficients for the total forecast projection times are −0.379~0.055oC, 1.951~2.078oC, and 0.741~0.965, respectively. Although, the forecasting skills of UMOS system are very consistent without regard to the season and geographic location, the performance is slightly better in autumn and winter than in spring and summer, and better in coastal regions than in inland region. When we take into account the significant differences of the RDAPS and KWRF, the UMOS system can be used as a supplementary forecasting tool of the MOS system for 3-h temperature over South Korea. However, the UMOS system is very sensitive to the selected number and/or types of predictors. Therefore, more work is needed to enable the use of the UMOS system in operation, including tuning of the number and types of potential predictors and automation of the updating processes of the UMOS equations. Key words: Model output statistics, temperature, UMOS, RDAPS, KWRF
1. Introduction As physics, dynamics, and numerical techniques steadily improve, various scales and types of numerical weather prediction Corresponding Author: Myoung-Seok Suh, 182 Shinkwan-dong, Kongjucity 314-701, ChungCheongnam-do, Korea. E-mail:
[email protected]
(NWP) models are becoming the most powerful tools in the quantitative forecasting of weather elements. However, most of the NWP models cannot predict surface weather elements, such as air temperature, maximum and minimum temperature, wind, and visibility, which are most relevant to our human lives. The quality of precipitation, especially the heavy precipitation forecast by the NWP model, is far from satisfactory. Many countries have developed various types of statistical methods to produce weather elements that are not generated directly by NWP model or are generated with a poor quality. Model output statistics (MOS) and perfect prog method (PPM) using Kalman filtering (KF), multiple regression technique, and neural network have been commonly used (Lee et al., 1999; Glahn and Ruth, 2003; Zwiers and Von Storch, 2004; Cha and Ahn, 2005; Marzban et al., 2007; William et al., 2007). Among them, MOS can account for the systematic biases and reproduce the weather elements, such as the air temperature and visibility over the observation point by using model output variables as predictors (Glahn and Lowry, 1972; Jacks et al., 1990; Parvinder et al., 2003; Kenneth et al., 2004). The inclusions of local climate values and recent observations as predictors greatly improve the quality of MOS forecasting, especially for short-term forecasting. The Korea Meteorological Administration (KMA) has been using statistical methods to minimize the drawbacks of the NWP model and to produce surface weather element forecasts, such as air temperature, daily maximum and minimum temperature, and precipitation (Seo et al., 2006). In general, MOS is based on the idea of relating model forecasts to observations through linear regression or other statistical methods. Therefore, it is necessary to use the model output and observation data for at least 2~3 years to obtain statistically stable MOS equations. In this case, the model output should be generated by the same conditions (resolution, physics, integration time, and so on) of the model. However, it is well known that model changes, such as the spatial resolution, model physics, and data assimilation methods, have been frequent since the mid1980s (Wilson and Vallee, 2002). This means that the statistical characteristics of model output variables have been changed frequently because any changes in the driving model configuration imply that the model's forecasting performance should also be changed. Such changes inevitably alter the bias and variance of model output, their correlation structure, and, most critically,
200
ASIA-PACIFIC JOURNAL OF ATMOSPHERIC SCIENCES
the covariance structure with respect to observations (Wilson and Vallee, 2002). As a result, MOS equations should be rederived using the new model's output. To derive statistically stable MOS equations, it is necessary to run the model for at least 2-3 years whenever significant changes occur in the deriving model. This is very inefficient or almost impossible not only from an economic aspect but also from a computational aspect. To address the frequent changes of model environment, such as the spatial resolution, model physics, and data assimilation systems, Ross (1987, 1989) proposed a UMOS (Updateable MOS) system. Following Ross's idea, Wilson and Vallee (2002, 2003) developed a UMOS system over Canada and used it for operational forecasting of surface variables, such as air temperature, daily minimum and maximum temperature, and precipitation. In this study, we developed and validated a UMOS system for 3-h temperature over South Korea according to the Canadian UMOS system because there have been many changes including model replacement (e.g., the fifth-generation Penn State/National Center for Atmospheric Research Mesoscale Model (MM5) KMA Weather Research and Forecasting model (KWRF)) and modifications (e.g., resolution, physical parameterization) in the KMA operational model. The KMA plans to replace the Global Data Assimilation and Prediction System (GDAPS), Regional DAPS (RDAPS), and KWRF with the unified model (UM) in 2010. The remainder of this paper is organized as follows. The data used to generate and verify the UMOS forecasts are presented in the next section. Concept and development processes of the UMOS system are described in section 3. An evaluation of UMOS performance, including a comparison of UMOS with the high-resolution numerical model and MOS forecasts, is described in section 4. The results and conclusions are given in Section 5.
2. Data
Table 1. Description of the data produced by the two numerical weather prediction models, RDAPS and KWRF. Contents
RDAPS
KWRF
X
41
178
Y
55
160
Z
25
25
Resolution
30 × 30 km
10 × 10 km
Forecast_time
+ 66 hr (intv. 3 hr)
+ 60 hr (intv. 3 hr)
Operation period
2005. 4~2008. 5
2007. 6~2008. 4
Dimension
from the new model is large enough to produce stable equations, so two model outputs are needed when a change is occurred in the old model. In this study, two model outputs from the RDAPS and KWRF are used. Detailed descriptions of the model outputs and model domains are shown in Table 1 and Fig. 1, respectively. Simulations of the RDAPS are performed two times at 0000 UTC and 1200 UTC every day; each simulation result from 0 to 66 hours is saved every 3-hour period. Simulations of the KWRF are also performed and saved the same way as for the RDAPS except for the simulation time (+ 60 hours). b. Observation data Ground observation data along with model outputs are the basic input for the development of the MOS and UMOS systems. The 3-hourly air temperature observed at the 76 KMA observation stations from 2005 to 2008 were used in this study. We have constructed a climatological data base of 3-hourly temperature using ten years data from 1995 to 2004. In this process, we calculate the climatological value according to the date and hour of the day to take into consideration the daily and hourly variations of air temperature. 10
a. Model data
1 TC3h ( s, d, h ) = ------ ∑ T3h ( s, y, d, h ) 10
(1)
y=1
Unlike the MOS system, UMOS system has to update the forecast equations with recent model output until the sample
Fig. 1. Forecast area of the RDAPS (left) and KWRF (right).
where s, y, d, and h represent the stations (76), the year (19952004), day of year (1-365), and hour of the day (0, 3, 6, ... 21),
28 February 2011
Jeon-Ho Kang et al.
respectively. TC3h represents a 10-year mean climate value of 3hourly air temperature (T3h). These climate values are used as potential predictors in the MOS and UMOS equations. c. Preprocessing Two types of preprocessing are performed for the development of the UMOS system: one is an interpolation of model output into the observation point and the other is to derive the variables that are supposed to affect the predictands. As the spatial resolution of RDAPS is 30 km, we used the inverse distance-area weighted mean of four grid point values that are located around the observation point (Fig. 2a). Whereas the spatial resolution of KWRF is 10 km, we used a simple inverse distance weighted mean of 16 grids around the observation point (Fig. 2b). P = ((x2 × y2) × a) + ((x1 × y2) × b) + ((x1 × y1) × c) + ((x2 × y1) × d)
(2)
P = (((i1 + i2 + i3 + i4) × 2) + ((o1 + o2 + ... + o11 + o12) × 1))/20
(3)
It is natural that a different set of predictors is used in the MOS equations according to the forecast variable, observation location, and forecast time. Therefore, we selected 65 potential predictors including direct model outputs (e.g., geopotential height, 3-D wind, temperature, etc.) and derived variables (e.g., SSI: Showalter Stability Index, thickness, dew point temperature, etc.) that can have an impact on the temperature predictands.
3. Method a. System configuration The total UMOS system mainly consisted of three parts: the first part is a MOS system, the second part is a UMOS system,
201
and the last part is a validation device for the UMOS system. For the development and evaluation of the UMOS system of air temperature over South Korea, the MOS system was developed following Seo et al. (2006). The MOS system was designed and developed to provide the potential predictors for the UMOS system and to compare the relative accuracy of the UMOS system. Using the potential predictors and outputs from the MOS system, the UMOS system has been developed and validated with respect to ground observations. b. Design and development of MOS system In general, it takes 2~3 years of model output and observation archives for the development of a statistically stable MOS system (Glahn and Lowry, 1972). The MOS equation was developed only once using the archived model and observation data before operational use. In this study, we developed the MOS system for the preparation of downscaled potential predictors and evaluation of the relative accuracy of the UMOS system using the two years (April 2005 ~ March 2007) RDAPS and observation data (Table 2). As shown in Table 2, we added one month to the beginning and the end of each season for the development of a stable MOS equation because only two years of output from RDAPS are available. The remaining one year output of RDAPS is used for the validation of MOS equations. The whole process of MOS system design and development including the definition of season and potential predictors, and the predictor selection method is similar to that of Seo et al. (2006). We used 67 potential predictors (65: from model output, 1: climate value, 1: recent observation) for the development of the MOS system (Appendix I). The stepwise selection method (e.g. Draper and Smith, 1998) was used for the selection of predictors from 67 potential predictors. The MOS equation was designed and developed for each season (4), station (76), forecast hour (21: 0, 3, 6, 9, ..., 60 hours), and forecast issue time (2: 0000 and 1200 UTC) to account for the temporal and spatial
Fig. 2. Interpolation method of RDAPS (a) and KWRF (b) to the observation point (circle).
202
ASIA-PACIFIC JOURNAL OF ATMOSPHERIC SCIENCES
Table 2. Definition of season and sampling number of data used in the development of MOS and UMOS system using RDAPS and KWRF data.
RDAPS
KWRF
Spring
Summer
Autumn
Winter
Year / month
2005. 4~6 2006. 2~6
2005. 5~9 2006. 5~9
2005. 8~12 2006. 8~12
2005. 11~2006. 3 2006. 11~2007. 3
Run date
241
306
306
302
Year / month
2008. 2. 1~4.15
2007. 6~9
2007. 8~12
2007. 11~12 2008. 1~3
Run date
75
122
153
151
covariance structures between local climate and model output. Therefore, the total number of MOS equations is 12758 (4 × 76 × 21 × 2 = 12768). c. Design and development of UMOS system The concept of a UMOS system differs from that of the MOS system in two main aspects, as shown in Wilson and Vallee (2002). First, when a change is occurred in the old model, two model (old and new models) outputs are needed along with the observation data for the development of the UMOS system (Fig. 3). Second, the UMOS equations are updated regularly at a predefined interval (e.g., seven days, ten days) to consider the covariance structure between the new model output and the observations in the UMOS equation. As a result, several very important factors must be decided before design and development of the UMOS system, especially for an operational UMOS system. One is that the number and type of the potential predictors must be decided before archiving of model output; the other is the blending and updating method for the two model outputs. The blending and the minimum number of old and new model outputs should be decided. In this study, we used the 20 most frequently selected predictors among the 67 potential predictors in the MOS development to reduce the computer memory and computational time in the development processes of the UMOS system. The prescreened predictors are stored for each station (76), each season (4), each model run (2: 0000 and 1200 UTC), and each forecast
Fig. 3. Conceptual diagram of UMOS system.
projection (21). The most frequently selected predictors are model generated air temperature (T2M), observed temperature (OBST), climate mean temperature (CLMT), mixing ratio (SQ2M), northwesterly wind at 500 hPa (NW500), temperature at 850 hPa (T85), wind speed at 10 m (WS10M), height of planetary boundary layer (PBL), thickness between 1000 hPa and 850 hPa (THCK10_8), total rain (TRAIN), and meridional wind at 10 m (V10M). The predictors that are most similar to the 3 h temperature, T2M, OBST, and CLMT, are ranked as the top three without regard to the season and station. Wilson and Vallee (2002)'s blending and updating methods were used for the blending and updating of the RDAPS and KWRF model output. The key points of the blending of the two model outputs are the maximum weights for the two model outputs and their regular updating method as the sample of new model output increases. This is done in order to force the system to “pay attention” to the new model characteristics while the new model sample is still small. The detailed design and development processes are in Wilson and Vallee (2002). A UMOS system is intended to facilitate the rapid and frequent updating of a large number of MOS equations from a linear statistical model, either multiple linear regressions (MLR) or multiple discrimination analysis (MDA) (Wilson and Vallee, 2002). Both of these techniques use the sums-of-squares and cross-products matrix (SSCP), or components of it. The idea of the updating is to do part of the necessary recalculation of coefficients in nearly real or real time by updating the SSCP matrix and storing the data in that form rather than as raw
28 February 2011
Jeon-Ho Kang et al.
203
three forecast hours, and forecast issue time (0000, 1200 UTC) according to each season. As in the current MOS system (Seo et al., 2006) of the KMA, the UMOS system is also designed and developed according to the season by considering the seasonal variations of local climate. We added one month to the beginning and the end of each season for the development of a stable UMOS equation because only one year output from KWRF is available (Fig. 5). As a result, each season has five months (e.g., summer - May, June, July, August, and September) and the boundary months are used two times at the neighboring season. Therefore, the total array of SSCPold per season and per predictand is 2 × 76 × 21 (00, 03, 06, ..., 60 hours) × 20. This precalculated SSCPold is not changed for the whole period of the UMOS system. So, the selection of potential predictors is very important for the performance of the UMOS system. The SSCP for the old model is calculated as Fig. 4. Weighting value of new model according to sample size of new model when Nmax = 150, and ω max = 1.667.
observations. The multivariate regression model is as follows: y j = β 0 + β 1x1j + β 2x2j + ... + β kxkj + ε j, j = 1, 2, ..., n ε j~N(0,σ2), Cov(εi, ε j) = 0, i ≠ j .
(4)
Equation (4) is rewritten in matrix from as ⎛ y1⎞ ⎛ 1 x11 x21 ... xk1⎞ ⎛ β0⎞ ⎛ ε1⎞ ⎜ y2⎟ ⎜ 1 x12 x22 ... xk2⎟ ⎜ β1⎟ ⎜ ε2⎟ ⎜. ⎟ = ⎜ . .. ⎟⎟ ⎜⎜ .. ⎟⎟ + ⎜⎜ .. ⎟⎟ ⎜. ⎟ ⎜ . ⎝ y ⎠ ⎝ 1 x x ... x ⎠ ⎝ β ⎠ ⎝ ε ⎠ n 1n 2n kn k n
(5)
where y, x, k, n, β and ε represent the predictands, predictors, the number of predictors, sample size, regression coefficients and random errors, respectively. By using the least square method, β is obtained in the following forms: ⎛ β0⎞ ⎜ β1⎟ –1 B = ⎜ .. ⎟ = ( X'X ) ( X'Y ) ⎜.⎟ ⎝β ⎠
(6)
where x11 x12
x21 x22
… …
x1n
x2n
…
(8)
As the samples of new model output increases enough to the threshold (35), we selected the predictors from the combined 20 potential predictors (SSCP) of the old and new model outputs using a stepwise selection method. The reduced set of predictors was screened using stepwise regression at each step, with 0.1% additional reduction of variance as a criterion for stopping selection. SSCPnew of the [(n + 1) × (n + 1)] array is made using the selected predictors, and the same predictors in the SSCPold are selected and used for the reconstruction of the SSCP'old [(n + 1) × (n + 1)], which has the same array. As mentioned in the Wilson and Vallee (2002), the weighted blending of SSCP matrices should be designed to ensure smooth transition both between seasons and following a major model change. Also, the parameters of weighting scheme are somewhat arbitrary, so they have to be adjusted experimentally. In this study, the final SSCP is a weighted blending of the two models’ SSCP matrices based on the Wilson and Vallee (2002) empirical method. ( N1 – Nmax )2 ω 1 = ω max + ( 1 – ω max ) 1 – -----------------------------( Nmin – Nmax )2
(9)
Nmax – ω 1 N1 -, for N1 ≤ Nmax; ω 2 = -------------------------N2 ω 1 = 1, ω 2 = 0, for N1 > Nmax;
k
⎛1 ⎜1 X=⎜ . ⎜ . ⎝1
SSCPold = Xold’Xold
xk1 ⎞ xk2 ⎟ .. ⎟ ⎟ xkn ⎠
(7)
where B is a vector of the estimated coefficients, of length k + 1. X'X is sums of squares and cross products matrix (SSCP). X is the potential predictor used for the development of the UMOS equation and X' is a transpose of matrix X. For computational efficiency, only k = 20 potential predictors are used in this study. For calculation efficiency, we calculated the SSCP matrix of the old model output for the 76 observation points and every
SSCP = ω 1 SSCPnew + ω 2 SSCP'old
(10)
X'Y = ω 1 X'Ynew + ω 2X'Yold
(11)
where ω1 and ω2 are the weights for the new and old model, N1 and N2 are the sample sizes from the new and old models, respectively; Nmin is the minimum permitted sample size from the new model for updating; Nmax is the smallest sample from the new model for which blending with old model data is considered unnecessary; and ωmax is the largest weight applied to new model data. In this study, Nmin and Nmax are set to 30 and 150, respectively. We reduced Nmax from 300 to 150 because the archived
204
ASIA-PACIFIC JOURNAL OF ATMOSPHERIC SCIENCES
Fig. 5. Development and validation period of MOS and UMOS systems using RDAPS and KWRF (a sample for Fall).
data from KWRF is only 150 days. The parameter ωmax must be set high enough to cause a noticeable response to the new model data, but not so high that the equations are misled by the possible over-representation of a small sample from the new model. At present, ωmax is set to 1.667, as used in Wilson and Vallee (2002). The variation of ω1 according to the sample size of the new model data is shown in Fig. 4. As the Nmax, Nmin, and ωmax are adjusted values suited for the Canadian climate by Wilson and Vallee (2002), the parameters may need to be readjusted for operational use in Korea. d. Validation of UMOS system The periods for the development and validation of MOS and UMOS system for fall are shown in Fig. 5. As shown in Fig. 5, the MOS system was developed only once using the two years output from RDAPS, but the UMOS system should be consistently updated every week using the RDAPS output and increased KWRF output.
The absolute performance of the UMOS system was validated using the observed temperature data, whereas the relative performance of the UMOS system was evaluated by comparing it with the MOS output and model output. RDAPS and KWRF represent the 3-h air temperatures forecasted by RDAPS and KWRF. MOS_R and MOS_K represent the 3-h air temperature predicted by the same MOS equations developed with RDAPS using the potential predictors forecasted by RDAPS and KWRF, respectively. Forecast and validation of the UMOS system were performed only after one month of each season because it is not feasible to begin development of new model equations until at least 30 days of data are available. Accuracy measurements used were the bias error (Bias), root mean square error (RMSE), and correlation coefficients (Corr.).
4. Results Figure 6 shows a sample of the UMOS prediction equations derived in this study. The meaning of the numbers and variable
28 February 2011
Jeon-Ho Kang et al.
205
Fig. 6. A sample of UMOS prediction equations. The meaning of numbers and variable names are as follows: station ID, number of new model days, forecast projection time, number of predictors, selected predictors, and corresponding coefficients, respectively.
names are as follows: station ID, number of new model days, forecast projection time, number of predictors, selected predictors, and corresponding coefficients, respectively. These UMOS equations are developed for each season (4), for each station (76), for each forecast projection time (21), and for each model run time (2); all the UMOS equations are updated every seven days. The number of MOS prediction equations is the same as in the UMOS equations but they are derived only one time. To forecast the 3-h temperature using the derived MOS and UMOS equations, the selected predictors of RDAPS and KWRF are used. However, the quality of the numerical weather prediction model is not satisfactory and consistent, especially for geographic location, intensity, and moving velocity of weather
events. Therefore, instead of the one step predictors at the forecast projection time, 3-time step weighted averages of predictors were used for stable predictions (11). Xˆ t – 6h + 2Xˆ t – 3h + 3Xˆ-t⎞ Yˆ t = Ft⎛⎝ -----------------------------------------⎠ + Fcli( Tt ) + Fobs ( Tobs ) 6
(12)
where Yˆ t and Xˆ represent the estimated value at the forecast time t and the selected predictors from the 20 potential predictors by every 3 hours, respectively. The detailed lists of selected potential predictors are given in Table 3. Tobs and Fcli (Tt) are recently observed temperature and 10-year (1995-2004) mean temperature for the prediction time, respectively.
206
ASIA-PACIFIC JOURNAL OF ATMOSPHERIC SCIENCES
Table 3. Sample of the most frequently selected 20 predictors in the RDAPS MOS equations for 3-hourly air temperatures for each season. See the Appendix I for definitions of the acronyms. Rank 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Spring
Summer
Autumn
Winter
0000 UTC
0000 UTC
0000 UTC
0000 UTC
T2M T2M OBST T2M CLMT OBST T2M OBST OBST CLMT THCK10_8 CLMT WS10M WS10M WS10M THCK10_8 NW500 TRAIN CLMT WS10M V10M T85 PCWT TRAIN TRAIN NW500 SQ2M V10M THCK10_8 SQ2M TRAIN WD10M SQ2M CTOP T85 PBL RHM10_7 V10M QHDW NE10M PBL WS500 V10M RHM10_7 WD10M NW10M NW500 NW500 QDWL THCK10_7 PBL SQ2M V5 PBL WS500 QHDW NW10M CBASE QDWL NW700 THCK10_7 PCWT THCK10_7 V5 PCWT THCK10_8 RHM10_5 SWEATI T85 V5 V5 T85 SWEATI THCK8_5 NE10M NW10M NW850 WS850 WD10M V7
a. Absolute validation Table 4 shows the validation results of the UMOS system using ground observation. Seasonal mean bias, RMSE, and correlation coefficients for the total forecast projection time varied −0.379~0.055oC, 1.951~2.078oC, and 0.741~0.965, respectively. Although seasonal variations in the correlation are large, those of bias and RMSE are quite small at both 0000 and 1200 UTC. This means that the UMOS system estimated the 3-h temperature very consistently without regard to the season and model run time. In general, the performance of the UMOS system is better in autumn and winter than in spring and summer, in terms of the correlation. The relatively low performances in spring and summer seemed to be related to the small sample size and characteristics of the weather system (Table 2). Figures 7 and 8 show the spatial distribution of the seasonal mean RMSE of the 3-hourly temperature estimated by the UMOS system (0000 and 1200 UTC). The seasonal mean RMSE of the UMOS are very homogeneous in space and season, within 1.5~2.5oC. However, stations with relatively high RMSE (> 2.5oC) are found, especially in autumn. The largest RMSE of autumn in Table 2 is caused by these stations that have large RMSE. b. Comparison of prediction skills To investigate the objective performance of the UMOS system, the prediction skills of the UMOS system were compared with directly estimated 2 m temperature by RDAPS and KWRF, and estimated 2 m temperature from MOS equations using the
Table 4. Validation results of UMOS system according to season for 3hourly temperature at 0000 and 1200 UTC. Model run time Corr. 0000 UTC
1200 UTC
o
Spring
Summer
Autumn
Winter
0.789
0.741
0.963
0.894
Bias( C)
−0.379
−0.091
0.040
−0.200
o RMSE( C)
2.022
2.068
2.029
2.078
Corr.
0.787
0.758
0.965
0.890
−0.262
−0.101
0.055
0.034
2.012
1.951
1.994
2.062
o
Bias( C) o
RMSE( C)
RDAPS and KWRF predictors (MOS_R and MOS_K respectively). The differences in the prediction skills between MOS_R and UMOS are caused by the combined effects of the different prediction skills of the two models (RDAPS and KWRF) and differences in the statistical regression method, whereas those of MOS_K and UMOS are mainly caused by differences in the statistical regression method. Figure 9 shows the intercomparison results of seasonal prediction accuracy of the five different prediction methods including UMOS system for 3-hourly temperature (0000 UTC). Although, the prediction skills clearly depend on the season, the prediction skills of MOS_R, MOS_K, and UMOS are greater than those of RDAPS and KWRF without regard to season. The prediction accuracy of the UMOS system are superior to those of the MOS_K but inferior to those of the MOS_R. The low correlation of the UMOS system in spring is related to the small sample of the KWRF data (Table 2). And, probably unreliable predictor selection since only the small sample was used for this purpose. Figure 10 shows the correlation coefficients between the observed 3-hourly temperatures and those from the five different prediction methods according to the forecasting times (0000 UTC). The prediction skills of all the five forecasting methods are significantly affected by season: best and worst during autumn and summer, respectively. As in other forecasting methods, all the prediction skills decreased as the forecast projection time increased, especially for summer. However, that of autumn is nearly constant with respect to forecast projection time. The prediction skills of MOS_R are better and more consistent than those of the other methods in all four seasons and the whole forecast projection time. As in Fig. 9, the low correlation of the UMOS system in spring is related with the small sampling of the KWRF data (Table 2). And, probably unreliable predictor selection since only the small sample was used for this purpose. The temporal variation of the RMSE according to the forecast projection time shows that RDAPS and KWRF fail to capture the diurnal variation of air temperature (Fig. 11), whereas MOS_R, MOS_K, and UMOS greatly reduced the two models’ systematic problems. As a result, the magnitude and diurnal variations of RMSE are significantly reduced for the four seasons and for the whole forecast projection time. The performance of the UMOS is comparable to that of the MOS_R and is much better than that of the MOS_K for the four seasons and the whole forecast pro-
28 February 2011
Jeon-Ho Kang et al.
207
Fig. 7. Distribution of seasonal mean RMSE of the 3-hourly temperatures as forecast by the UMOS system (0000 UTC).
jection time. As in the Seo et al. (2006), the RMSE of MOS_R during spring is relatively large compared to that of UMOS and other season. It means that the large RMSE in spring is related with the forecast skill of RDAPS.
5. Discussion and summary It is well known that model changes, such as changes in spatial resolution, model physics, and data assimilation methods, have been frequent since the mid-1980s (Wilson and Vallee, 2002). Since it is necessary to run a new model for at least 2~3 years whenever there are significant changes in the deriving model for the development of statistically stable MOS equations, frequent model changes make it difficult to obtain MOS equations by standard methods. This is very inefficient or almost impossible from not only an economic aspect but also a computational aspect. To address this problem, Ross (1987, 1989) proposed a
UMOS (Updateable MOS) system and it was successfully implemented by Wilson and Vallee (2002, 2003). In this study, we designed and developed UMOS system for the 3-h temperature over South Korea using two significantly different models’ (RDAPS and KWRF) outputs based on the Canadian UMOS system. Like the current KMA MOS system, the UMOS system is designed to consider the climatological characteristics and model forecasting skills. We added one month to the beginning and end of each season for the development of a stable UMOS equation because only two years’ output from RDAPS and one year’s output from KWRF were available. We carried out a comprehensive evaluation of the forecasting skills of the UMOS system, comparing its performance with that of MOS forecasts using RDAPS and KWRF outputs as predictors, and 76 KMA observations data. The most frequently selected 20 potential predictors were used for each season, station, and forecast projection time through the prescreening processes of 67
208
ASIA-PACIFIC JOURNAL OF ATMOSPHERIC SCIENCES
Fig. 8. Same as in Fig. 7 except for 1200 UTC.
Fig. 9. Intercomparison of prediction accuracy of the five different prediction methods for 3-hourly temperature (0000UTC).
28 February 2011
Jeon-Ho Kang et al.
209
Fig. 10. Intercomparison of correlation coefficients between the observed 3-hourly temperatures and those predicted by the five different prediction methods according to the forecasting times (0000UTC).
Fig. 11. Same as in Fig. 10 except for RMSE.
210
ASIA-PACIFIC JOURNAL OF ATMOSPHERIC SCIENCES
potential predictors of the MOS system in order to increase the computational efficiency. The blending method of the two model outputs and their relative weighting values are the same as those used in Wilson and Vallee (2002). The validation results of the UMOS system with ground observation data showed that seasonal mean bias, RMSE, and correlation coefficients for the total forecast projection time are −0.379~0.055oC, 1.951~2.078oC, and 0.741~0.965, respectively. The seasonal mean RMSE of the UMOS system is slightly larger than that (1.5~2.0oC) of the MOS system (Seo et al., 2006). They showed that the forecast skills of the UMOS system for 3-h temperature are significantly affected by season and model run time. In general, the performance of the UMOS system is better in autumn and winter than in spring and summer, especially in the correlation. The seasonal mean forecasting skills of the UMOS system are better in coastal regions than in inland regions. The prediction skills of the UMOS system are superior to those of the MOS_K but inferior to those of the MOS_R. The prediction skills of the UMOS system are significantly affected by season: best and worst during autumn and summer, respectively. The low correlation of the UMOS system in spring is related with the small sampling of the KWRF data. The prediction skills of the UMOS system decreased as the forecast projection time increased, especially for summer. However, that of autumn is nearly constant with the forecast projection time. In general, the performance of the UMOS is comparable to that of the MOS_R and is much better than that of the MOS_K for all seasons and forecast projection times. When we take into account the significant differences of the two models (RDAPS and KWRF) used in this study and the comprehensive validations results, the UMOS system can be used as a supplementary forecasting tool for 3-h temperature over South Korea. Furthermore, we added one month to the beginning and end of each season to increase the sampling and attempted no tuning of the UMOS system, so the validation results are considered to represent the minimum prediction accuracy of the UMOS system. However, the UMOS system is very sensitive to sampling number, as indicated by the spring results and the selected number and/or types of predictors. Although we smoothed the final predictors through the weighted average of the 3-step data, extremely large RMSE occurred and this is mainly caused by the relatively small number of final predictors (1~2) and/or the large spatial/ temporal variation of the final selected predictors, such as thickness, relative humidity, wind, and precipitation. Therefore, more work is needed before implementing the UMOS system in operation, including tuning of the number and types of potential predictors and automation of updating processes of the UMOS equations.
Acknowledgement. The authors thank Dr. S. W. Joo, J. T. Choi and Y. K. Seo for helpful suggestions. The authors also thank Korean Meteorological Administration for their large amount of numerical model outputs. This work was supported by the Korean Meteorological Administration Research and Development Program under Grant RACS 2010-2014.
REFERENCES Cha, Y. M., and J. B. Ahn, 2005: Evaluation of artificial neural network correction skill on dynamically downscaled summer rainfall over South Korea. J. Korean Meteor. Soc., 41, 1125-1135. Draper, N. R., and H. Smith, 1998: Applied regression analysis: 3rd Edition. Wiley-interscience, 736 pp. Glahn, H. R., and D. A. Lowry, 1972: The use of model output statistics (MOS) in objective forecasting. J. Appl. Meteorol., 11, 1203-1211. ______, and D. P. Ruth, 2003: The new digital forecast database of the National Weather Service. Bull. Amer. Meteor. Soc., 84, 195-201. Jacks, E., B. Bower, V. J. Dagostaro, J. P. Dallavalle, M. C. Erickson, and J. C. Su, 1990: New NGM-based MOS guidance for maximum/ minimum temperature, probability of precipitation, cloud amount, and surface wind. Wea. Forecasting, 5, 128-138. Kenneth, A. H., W. J. Steenburgh, D. J. Onton, and A. J. Siffert , 2004: An evaluation of mesoscale-model-based model output statistics (MOS) during the 2002 Olympic and Paralympic winter games. Wea. Forecasting, 19, 200-218. Lee, M. Y., D. I. Lee, and W. J. Lee, 1999: The forecasting the maximum/ minimum temperature using the Kalman filter. J. Korean Meteor. Soc., 35, 283-289. (in Korean with English abstract) Marzban, C., S. Leyton, and B. Colman, 2007: Ceiling and visibility forecasts via neural networks. Wea. Forecasting, 22, 466-479. Parvinder, M., A. Kumar, L. S. Rathore, and S. V. Singh, 2003: Forecasting maximum and minimum temperatures by statistical interpretation of numerical weather prediction model output. Wea. Forecasting, 18, 938952. Ross, G. H., 1987: An updateable model output statistics scheme. World Meteorological Organization, Programme on short- and medium range weather prediction. PSMP Report Series No. 25, 45-48. ______, 1989: Model output statistics - an updateable scheme. Preprints, 11th Conference on Probability and Statistics in Atmospheric Sciences, Boston, MA, Amer. Meteor. Soc., 93-97. Seo, Y. K., J. T. Choi, and J. K. Yang, 2006: Development of air temperature forecast model and operational system using MOS. Korea Meteorological Administration, DFS-TN-2006-5, 189 pp. (in Korean). William, Y., Y. Cheng, and W. J. Steenburgh, 2007: Strengths and weaknesses of MOS, running-mean bias removal, and Kalman filter techniques for improving model forecasts over the Western United States. Wea. Forecasting, 22, 1304-1318. Wilson, L. J., and M. Vallee, 2002: The Canadian updateable model output statistics (UMOS) system: Design and development tests. Wea. Forecasting, 17, 206-222. ______, and ______, 2003: The Canadian updateable model output statistics (UMOS) system: Validation against perfect prog. Wea. Forecasting, 18, 288-302. Zwiers, F. W., and H. V. Storch, 2004: On the role of statistics in climate research, Int. J. Climatol., 24, 665-680.
28 February 2011
211
Jeon-Ho Kang et al.
Appendix Appendix I. List of the 67 potential predictors for the MOS system. Code
Level
Potential Predictors
Number of predictors
T
2 m, 850 hPa, 700 hPa, 500 hPa
Temperature
4
EPT
850 hPa, 700 hPa, 500 hPa
Equivalent potential temperature
3
THCK
1000-850 hPa, 1000-700 hPa, 850-500 hPa
Thickness
3
TDD
850 hPa, 700 hPa, 500 hPa
Dew point depression
3
SQ
2 m, 850 hPa, 700 hPa, 500 hPa
Specific humidity
4
DQ
850 hPa, 700 hPa
Difference between specific humidity and saturated specific humidity at 500 hPa
2
RH
850 hPa, 700 hPa, 500 hPa
Relative humidity
3
RHM
1000-700 hPa, 1000-500 hPa, 850-500 hPa
Layer averaged RH
3
U
10 m, 850 hPa, 700 hPa, 500 hPa
Zonal wind
4
V
10 m, 850 hPa, 700 hPa, 500 hPa
Meridional Wind
4
NE
10 m, 850 hPa, 700 hPa, 500 hPa
North-easterly wind
4
NW
10 m, 850 hPa, 700 hPa, 500 hPa
North-westerly wind
4
WS
10 m, 850 hPa, 700 hPa, 500 hPa
Wind speed
4
WD
10 m, 850 hPa, 700 hPa, 500 hPa
Wind direction
4
TRAIN
-
Total rain
1
PCWT
-
Precipitable water between 1000 and 400 hPa
1
LR
850-700 hPa, 850-500 hPa
Lapse rate
2
SSI
-
Showalter Stability Index
1
KIDX
-
K-Index
1
LINDEX
-
Lifted Index
1
SWEATI
-
Severe Weather Threatening Index
1
CCL
-
CCL (hPa)
1
CTOP
-
Height of cloud top
1
CBASE
-
Height of cloud base
1
DWL HDWL QDWL QHDW
-
Thickness of wet layer Height of wet layer Specific humidity of WL Specific humidity of HWL
4
PBL
-
PBL height
1
CLMT
-
Climatological Temperature
1
OBST
-
Recently observed T.
1
Total
67
