Quantitative Precipitation Forecasting of Wintertime ... - AMS Journals

2834

MONTHLY WEATHER REVIEW

VOLUME 133

Quantitative Precipitation Forecasting of Wintertime Storms in the Sierra Nevada: Sensitivity to the Microphysical Parameterization and Horizontal Resolution VANDA GRUBISˇ IC´ , RAMESH K. VELLORE,

AND

ARLEN W. HUGGINS

Desert Research Institute, Reno, Nevada (Manuscript received 8 July 2004, in final form 21 March 2005) ABSTRACT The skill of a mesoscale model in predicting orographic precipitation during high-impact precipitation events in the Sierra Nevada, and the sensitivity of that skill to the choice of the microphysical parameterization and horizontal resolution, are examined. The fifth-generation Pennsylvania State University– National Center for Atmospheric Research (PSU–NCAR) Mesoscale Model (MM5) and four bulk microphysical parameterization schemes examined are the Dudhia ice scheme, and the Schultz, GSFC, and Reisner2 mixed-phase schemes. The verification dataset consists of ground precipitation measurements from a selected number of wintertime heavy precipitation events documented during the Sierra Cooperative Pilot Project in the 1980s. At high horizontal resolutions, the predicted spatial precipitation patterns on the upwind Sierra Nevada slopes were found to have filamentary structure, with precipitation amounts over the transverse upwind ridges exceeding severalfold those over the nearby deep river valleys. The verification results show that all four tested bulk microphysical schemes in MM5 produce overprediction of precipitation on both the windward and lee slopes of the Sierra Nevada. The examined accuracy measures indicate that the Reisner2 scheme displays the best overall performance on both sides of the mountain range. The examined statistical skill scores on the other hand reveal that, regardless of the microphysical scheme used, the skill of the MM5 model in predicting the observed spatial distribution of the Sierra Nevada orographic precipitation is fairly low, that this skill is not improved by increasing the horizontal resolution of the model simulations, and that on average the quantitative precipitation forecasting (QPF) skill is better on the windward than on the lee side. Furthermore, a significance test shows that differences in skill scores obtained with the four microphysical schemes are not statistically significant.

1. Introduction Despite the advances in computing power and the degree of sophistication of mesoscale models, achieving accurate short-term quantitative precipitation forecasting (QPF) has been an elusive task. The unpredictability of precipitation is accentuated by various sources of error in QPF including the representation of local properties such as orography and land use, representation of physical processes within clouds, model initialization errors of various fields, particularly water substance quantities, as well as spatial resolution of model simulations (Bruintjes et al. 1994; Emanuel et al. 1995; Gaudet and Cotton 1998; Montani et al. 1999; Milbrandt and Yau 2001). Among these sources of forecast error,

Corresponding author address: Dr. Vanda Grubišic´, Division of Atmospheric Sciences, Desert Research Institute, Reno, NV 89512. E-mail: [email protected]

© 2005 American Meteorological Society

MWR3004

the representation of cloud microphysical processes has a profound influence on the amount, spatial distribution, and occurrence of precipitation. To achieve accurate short-term prediction of precipitation in complex terrain, mesoscale models must not only include an accurate treatment of clouds and associated precipitation processes, but also correctly account for all dynamical interactions driven by the finescale variability of topography and atmospheric conditions. This is a grand challenge given that microphysical processes alone, such as ice initiation, are not completely understood, and are represented with simplified descriptions contained in the bulk microphysical schemes. While the QPF skill of operational forecasting models has been traditionally low, and the improvements in the accuracy of precipitation forecasts slow compared to other forecasted variables (Ebert et al. 2003), the importance of achieving accurate precipitation forecasts has always been high. This combination of low forecasting skill and high socioeconomic impact has led the scientific community of

OCTOBER 2005

G R U B I Sˇ I C´ E T A L .

the U.S. Weather Research Program (USWRP) to make improvements in QPF their highest priority (Fritsch et al. 1998). Major mountain ranges in the western United States, such as the Sierra Nevada and Washington Cascades, receive large amounts of precipitation from the frequent passage of Pacific storms during the cool season (from November to March). As a major source of summertime water supply, this wintertime precipitation has a large impact on the local hydrologic cycle (Cayan and Peterson 1989; Dettinger et al. 1995). The Sierra Nevada, the north-northwest–south-southeast-oriented mountain range, with an approximate length of 600 km and a half-width of 100 km, lies nearly perpendicular to the path of the Pacific storms. This mountain range is an ideal environment for studies of orographic precipitation processes due to its linear geometry, absence of larger mountains upstream (aside from the modest coastal mountain ranges of California that extend upward to only about 500 m ASL), and its gentle upwind slope (⬃2%; from 0.1 km ASL in the Central Valley of California to approximately 2.2 km ASL over a horizontal distance of 100 km). There is a very small number of passes that interrupt the compact ridgeline of the Sierra Nevada, and a fairly large number of deep river valleys on its western slopes. The latter give rise to a number of transverse ridges oriented perpendicular to the Sierra Nevada crest line (Fig. 1). Previous theoretical and observational studies have shown that the amount of orographic lifting and precipitation is dependent on the size and shape of the mountain barrier as well as the distribution of moisture and static stability in the lower atmosphere (Smith 1979). The heavy precipitation on the western slopes of the Sierra Nevada is caused mainly by orographic lifting of the oceanic air inflow within the warm sectors of the Pacific storms. Consequently, cloud systems evolve slowly and often persist many hours within a single storm event producing widespread, predominantly stratiform precipitation. Weak instability of the incoming oceanic air in winter leads to embedded shallow convection cells that might enhance the rainfall locally, both over the coastal ranges (James and Houze 2005) and over the Sierra Nevada (Reynolds and Kuciauskas 1988), in particular behind the passage of an upperlevel cold front. The precipitation distribution over the Sierra Nevada varies significantly in the west–east direction with a typical maximum of precipitation 10–20 km upstream of the crest, accompanied by a strong shadowing effect in the lee (Rauber 1992). Recent numerical modeling studies of wintertime precipitation in complex terrain (Colle et al. 1999; Colle and Mass 2000) have suggested that the modeled pre-

2835

FIG. 1. (a) Outline of the MM5 model simulation domains. Topographic contours are shown every 250 m. (b) Topography within the innermost MM5 domain with the horizontal resolution of 1.5 km. The solid line marks the boundary between the windward and leeward side verification subdomains. Black squares and white circles indicate the windward and leeward side verification points described in section 3a. The dashed line marks the baseline AB of the vertical cross sections shown in Figs. 16 and 17, and the letter K marks the location of Kingvale.

cipitation structures are highly sensitive to the spatial resolution of mesoscale model simulations. In a series of studies, these authors evaluated the QPF skill of the fifth-generation Pennsylvania State University– National Center for Atmospheric Research (PSU– NCAR) Mesoscale Model (MM5) using winter months’

2836


VOLUME 133

TABLE 1. SCPP storms selected for QPF validation. The second and third columns show the start and end date and time of significant precipitation periods in the SCPP project area. The precipitation amount in the fourth column reflects the mean precipitation amount of all the SCPP gauges divided by the storm length (in days).

Storm period

Precipitation start MMDD/time (UTC)

Precipitation end MMDD/time (UTC)

Amount (mm day⫺1)

17 and 18 Dec 1982 12 and 13 Feb 1986 14–22 Feb 1986 3 and 4 Jan 1987

1217/0500 0212/0400 0214/0000 0103/1100

1218/0100 0213/1500 0222/0000 0104/0500

78.2 86.6 103.6 103.4

real-time simulations in the Pacific Northwest that were run at 36- and 12-km grid resolutions on nested domains. Their conclusion was that the model performed overall better at 12-km horizontal resolution than at 36 km. However, at 12-km horizontal resolution serious discrepancies were noted between the observed and model-predicted precipitation amounts, with an overestimation of precipitation on the windward side and an underestimation on the lee side. Further modeling studies by these authors, at resolutions down to 4 km, showed that precipitation structures become better resolved at higher spatial resolutions, but the model showed a tendency to produce too much precipitation on the windward slopes for observed light precipitation amounts (Colle et al. 2000). Consequently, the increased resolution alone appeared insufficient to address the challenges of improving QPF, and a need for improved understanding of physical processes in wintertime precipitation events in complex terrain and their improved representation in mesoscale models was emphasized, leading a way to several recent orographic precipitation field experiments (Schultz et al. 2002; Stoelinga et al. 2003). In this paper, we present a detailed investigation of the QPF skill of the MM5 model in predicting the amount and spatial distribution of precipitation in wintertime orographic precipitation events in the Sierra Nevada. The sensitivity of the model QPF skill on the choice of the microphysical parameterization is investigated with a series of high-resolution simulations in order to understand model deficiencies in predicting the precipitation amounts at fine spatial resolutions. Storms considered in this study are a selected number of high-impact long- and short-duration wintertime precipitation events documented in the central Sierra Nevada in the 1980s during the Sierra Cooperative Pilot Project (SCPP; Table 1). The SCPP field project, administered by the U.S. Bureau of Reclamation, was designed to document physical processes involved in orographic precipitation with the goal of verifying the weather modification technology used in supplementing regional water supplies in central California and

western Nevada (Reynolds and Dennis 1986). The focal target area of the SCPP experiment was the American River basin including the economically important Interstate-80 corridor. The paper is organized as follows. The numerical model and design of numerical simulations are described in section 2. Observational datasets and verification methodology are described in section 3. A brief overview of the Sierra Nevada winter storms and predicted precipitation patterns is presented in section 4. Section 5 presents the results of the QPF verification. Differences in precipitation patterns and forecasting skill for different bulk microphysical schemes are discussed in section 6. A summary and conclusions are presented in section 7.

2. Numerical model and experimental setup The numerical model chosen for this study is the MM5, a limited-area, nonhydrostatic model with equations cast in the terrain-following pressure coordinates, a multiple nesting capability, and a full suite of physical parameterizations (Grell et al. 1994). High-resolution numerical simulations in this study were carried out on four stationary nested domains employing one-way nesting. The four domains, each with 103 ⫻ 103 grid points in the horizontal, had horizontal grid increments of 40.5, 13.5, 4.5, and 1.5 km, respectively. Thirty-one unequally spaced vertical levels were chosen between the surface and 100 hPa, with eight levels placed within the lowest 1 km of the domain. The two inner domains with grid increments of 4.5 and 1.5 km define the geographical area of interest in this study. The second nested domain, centered on Lake Tahoe at (39.5°N, 120°W), was designed to cover the northern and central Sierra Nevada, extending to the Coastal Range upwind and including parts of the Great Basin topography downwind. The innermost, third nested domain was focused on the American River basin area in California, with two-thirds of this domain covering the upwind slope and one-third of the domain covering the lee slope of the Sierra Nevada (Fig. 1).

OCTOBER 2005


The model topography and land-use data in the two inner domains were obtained by interpolating the 30-s (0.9 km) U.S. Geological Survey datasets to the model grid. In the two outer domains with a grid resolution of 13.5 and 40.5 km, 5-min (9 km) and 10-min (19 km) resolution topography and land-use datasets were used, respectively. Soil moisture and snow cover were included in the analyses. Initial model atmospheric conditions were obtained by interpolating the National Centers for Environmental Prediction (NCEP)–NCAR global reanalyses (2.5° latitude ⫻ 2.5° longitude resolution) to the model grids. Assimilation of the upper-air observations from radiosondes and surface observations from land stations into the initial analyses was performed by Cressman’s objective analysis (Cressman 1959). These same fields were also used for the update of lateral boundary conditions every 6 h during the model forecast. At the model top, a radiative boundary condition was applied in order to prevent reflection of gravity waves from the upper model boundary (Klemp and Durran 1983). The planetary boundary layer (PBL) was parameterized using the Mellor–Yamada turbulence closure model for vertical mixing of horizontal wind, temperature, and mixing ratio (Mellor and Yamada 1982; Janjic´ 1994). The treatment of longwave and shortwave radiative forcing follows Dudhia (1989). The physical processes for precipitation were represented with the Kain–Fritsch cumulus parameterization (Kain and Fritsch 1993; Kain 2004) for convective precipitation applied in the two outer domains, where convective precipitation could not be resolved explicitly, and a microphysical parameterization active in all domains. The Kain–Fritsch convective parameterization coupled with the chosen PBL parameterization yields the most accurate buoyancy results at the lifting condensation level (Bright and Mullen 2002). To ensure the most accurate simulation of synoptic-scale features, four-dimensional data assimilation of wind, moisture, and temperature fields was applied in the two outer domains to nudge the model fields toward the NCEP– NCAR gridded analyses during the first 12 h of the model simulations. As the focus of this study is the sensitivity of wintertime precipitation forecasts on the choice of the microphysical parameterization, several bulk microphysical parameterization schemes with varying degrees of complexity in the representation of microphysical processes available in MM5 were used. These schemes are as follows: 1) Dudhia’s ice scheme (DUDH; Dudhia 1989), a simple ice scheme modification of the Hsie et al. (1984) warm-rain scheme, in which only ice is present below 0°C (no coexistence of cloud water and cloud ice in the temperature range of ⫺40° to 0°C), and melting of ice

2837

occurs immediately at the melting layer; 2) Schultz’s scheme (SCHUL; Schultz 1995), the mixed-phase scheme which predicts the evolution of five classes of hydrometeors, namely, cloud water, cloud ice, rain, snow, and graupel, with the production following a certain sequence optimized for efficiency required in realtime forecasting; 3) the Goddard Space Flight Center’s mixed-phase scheme (GSFC; Tao and Simpson 1993), which predicts liquid phase categories, such as cloud water and rain, using Kessler-type parameterization (Kessler 1969), and ice phase categories, such as cloud ice, snow, and hail/graupel, following the bulk parameterization of Lin et al. (1983); and 4) Reisner2’s mixedphase scheme (REIS; Reisner et al. 1998; Thompson et al. 2004), which, in addition to all the mixed-phase processes contained in SCHUL and GSFC, includes supercooled water processes and explicit prediction of number concentration of ice. For each selected wintertime storm, different experiments were varied only in the choice of the microphysical parameterization. The selected wintertime storms include three 2-day events, and an 8-day event (Table 1). To minimize effects related to the length of simulations, we have applied the same simulation procedure to all selected events, in which multiday events were broken into individual 24-h periods, each starting at 0000 UTC. Numerical simulations for these individual 24-h periods were initialized 12 h ahead of their start, resulting in 14 individual 36-h model runs. Only the last 24 h of each run were used in model verification, comparing model predictions to observed 24-h precipitation accumulations as described in the next section.

3. Verification dataset and statistics a. Verification data During the 1982–86 SCPP winter seasons, precipitation measurements were carried out at a total of 129 stations. Organized networks of precipitation observations consisted of the SCPP field program network within the focal target area with 84 stations, the Desert Research Institute’s (DRI) network in the Tahoe– Truckee basin that operated 24 stations, and 21 stations in the higher elevations of the Middle and South Fork sections of the American River basin (ARB) area operated by the Sacramento County and the Sacramento Municipal Utility District (SMUD). An additional 15 precipitation measuring sites were in use during the 1987 winter season. The majority of these measuring sites lie on the windward slopes of the Sierra Nevada. To create a denser observational grid for determining statistical skill scores, we have supplemented the SCPP precipitation data with daily precipitation accumula-

2838


tions obtained from the archives of the Western Regional Climate Center (WRCC) at DRI, consisting of daily precipitation totals from all National Weather Service (NWS) cooperative sites in this area including a couple of National Resources Conservation Service (NRCS) snowpack telemetry (SNOTEL) sites. In the SCPP network, the precipitation measurements were carried out with 8- and 12-in. orifice standard Belfort-weighing gauges. The DRI and SMUD networks used 8-in. heated tipping buckets, and 8-in. recording Belfort-weighing gauge-type instruments, respectively. The NWS stations with rain gauges used a standard American 8-in. rain gauge to measure the precipitation totals (Yang et al. 1998). No attempt was made in this study to estimate measurement errors stemming from different instrument design and less than ideal siting of rain gauges. Consequently, the measured precipitation accumulations were treated as ground truth. The selected time period for the QPF statistics in this study is 24 h, except for the mean areal precipitation, which was also computed with 3-h precipitation accumulations. For shorter time periods, the verification results, particularly the QPF skill scores, must be interpreted carefully as the accuracy of scores is sensitive to the length of the verification period, with errors becoming progressively smaller for longer periods (Colle et al. 1999). In the computation of the QPF statistics, the 24-h precipitation accumulations from the storms listed in Table 1 were grouped together since the number of observations was fairly small for each individual storm. This grouping is reasonable since all four of these storms exhibit common features of the typical warm winter storms in the Sierra Nevada. The total number of 24-h forecasts in the verification dataset is 14. The verification domain is the innermost domain of the MM5 model (Fig. 1). Within this area, there are 52 rain gauges (44 from the SCPP networks and 8 from the WRCC archives) that had precipitation data for all the cases from Table 1. Consequently, the total number of individual observations in the entire verification domain for all documented forecast periods is 728 (52 observations for each 24-h period ⫻ 14 days). The same domain and set of observations was used for verification of forecasts from the 13.5-, 4.5-, and 1.5-km domains. To facilitate the analysis of QPF skill for the upwind and lee side separately, we have defined two mutually exclusive verification subdomains within the main verification domain. For all the cases considered in this study the wind direction in the lower to midtroposphere was southwesterly, nearly perpendicular to the Sierra Nevada crest line, so the separation of the two subdomains by the north-northwest–south-

VOLUME 133

FIG. 2. Frequency distribution of the observed 24-h precipitation events for the defined precipitation intervals (mm). The number of events in each interval is shown for the entire verification domain as well as for the upwind and lee sides separately.

southeast-oriented line is justified (Fig. 1). With 33 rain gauge sites within the upwind portion of the verification domain, and 19 rain gauge sites on the lee side (Fig. 1), there are 462 observational data points on the windward side, and 266 on the lee side. Except for a few points in Central Valley, most of the verification sites are clustered close together higher up in the mountains, and occupy a relatively small portion of the verification domain. Since predicted precipitation patterns along the Sierra Nevada do not vary significantly in the alongcrest direction (as shown in the next section), our results are likely representative of high-terrain portions of the Sierra Nevada but might not be representative of the lower upwind slopes. Figure 2 shows the frequency distributions of the 24-h observed precipitation events for the entire verification domain as well as for the upwind and lee sides separately. The precipitation intervals used in this study are ⬍0.254, 0.254–12.7, 12.7–25.4, 25.4–38.1, 38.1–50.8, 50.8–76.2, 76.2–101.6, and ⱖ101.6 mm. The frequency distribution of the entire dataset slopes sharply downward from the maximum (⬎160 events) in the smallest precipitation category (⬍0.254 mm) to the 50–60 events in the largest categories. The distinctly different shapes of the windward and leeside component frequency distributions reflect, respectively, the high precipitation efficiency and the strong rain-shadowing effect of the Sierra Nevada (Smith 1979; Soong and Kim 1996). To obtain the model-predicted and measured data at the same spatial locations, the area-to-point conversion was used, interpolating the MM5 precipitation forecasts from the regular model grids to the precipitation gauge locations using an inverse distance Cressman (1959) method following Colle and Mass (2000). If Pn is the model precipitation at four grid points surrounding a


OCTOBER 2005

2839

TABLE 2. Dichotomous contingency table for categorical forecast verification. In this paper, the “Yes” events represent precipitation, either observed or predicted, falling within a given precipitation interval. Observed—Yes

Observed—No

Total

Hits (A) Misses (C) O⫽A⫹C

False alarms (B) Correct negatives (D) B⫹D

F⫽A⫹B C⫹D A⫹B⫹C⫹D

Predicted—Yes Predicted—No Total

gauge location, then the model precipitation Pm at the gauge location is given by 4

Pm ⫽

兺W P n

n⫽1

n

冒兺

N

4

Wn,

共1兲

DMB ⫽

⌬2 ⫺ dn2 ⌬2 ⫹ dn2

,

⌬ is the model horizontal grid spacing, and dn is the horizontal distance from the model grid point n to the gauge location m. As shown by Tustison et al. (2001), this method introduces a “representativeness error” into the QPF statistics. This error, which is a measure of errors in representing data at a scale other than their own inherent scale, is independent of the model forecast error. In the case of the area-to-point conversion used here, the representativeness error, stemming from different resolution model data being mapped to a single point, is expected to worsen at coarser model resolutions. Thus, improvements in skill that may be noted as model resolution is refined may only reflect the smaller representativeness error while the model performance may not be any better. As we will see later, this does not pose a problem in this study as, for the horizontal resolutions considered here, the skill does not improve as the resolution is increased.

b. Statistics Verification statistics quantify the ability of a forecast to correctly predict the occurrence and location of a precipitation event. Continuous and categorical verification statistics estimate the correspondence between the model forecasts and observations based on quantitative amounts and occurrences, respectively. Continuous statistical measures used in this study are mean areal precipitation (MAP), degree of mass balance (DMB), mean absolute error (MAE), and rootmean-square error (RMSE). MAP is the depth of precipitation averaged over a certain area over a specified time interval. In this study, MAP was calculated using 3- and 24-h precipitation accumulations and a method of weighting factors (appendix A). DMB describes the

兺

k⫽1

n⫽1

where Wn are weights at model grid points, Wn. ⫽

ratio of the predicted to the observed net water mass for a 24-h period, and is given by

冒兺 N

Pkp

共2兲

Pko,

k⫽1

where Pp and Po are predicted and observed precipitation amounts, respectively, and N is the total number of verification records (Charba et al. 2003). MAE and RMSE are accuracy measures based on forecast errors at individual locations N

MAE ⫽

兺 |P

p k

k⫽1

冋兺

⫺ Pko|

冒

k⫽1

冒册

共1Ⲑ2兲

N

RMSE ⫽

共3兲

N,

共Pkp ⫺ Pko兲2

N

共4兲

that give the average magnitude of the forecast error. Neither MAE nor RMSE indicates whether the error was an overestimate or underestimate. The categorical statistics1 measures employed in this study are based on a contingency table (Table 2). In the case of precipitation forecasts, the individual elements of the contingency table represent a number of events or occurrences for which the predicted precipitation amounts fall within or outside of a given observed precipitation interval or exceed or fall below a given threshold. In this study we have adopted the former approach, as it sharpens the specificity of QPF error as a function of the precipitation amount, and is helpful in avoiding the score overlap ambiguities—a known drawback of a more widely used approach with precipitation thresholds. The precipitation interval approach has been employed recently in several QPF verification studies (Schultz and Snook 1996; McDonald 1998; Charba et al. 2003). For the time-integrated precipitation, such as the 24-h precipitation accumulations used here, the frequencies of the observed and predicted events for any chosen precipitation interval are measures of the size of the area within which the given precipitation amounts 1 We refer the reader to Murphy and Daan (1985), Stanski et al. (1989), and Wilks (1995) for reviews of various verification statistics used in meteorology.

2840


were observed and predicted. In terms of the spatial equivalents, the number of hits, A, is equivalent to the correct forecast area that lies on the intersection between the predicted (F ⫽ A ⫹ B) and the observed (O ⫽ A ⫹ C) areas. The correct negatives (D) define the area where a given precipitation amount was neither predicted nor observed. The correct forecast area is nonzero for all precipitation intervals in this study except the smallest precipitation interval on the lee side of the mountain range. Many different skill scores have been formulated over the past decades with a continuing debate as to which skill score is the most appropriate as a categorical measure of the forecast ability. Two main statistical scores that were calculated from the contingency tables and are discussed here are bias (BIAS) BIAS ⬅

F A⫹B ⫽ , O A⫹C

共5兲

and equitable threat score (ETS) ETS ⬅

A ⫺ R共A兲 , A ⫺ R共A兲 ⫹ B ⫹ C

共6兲

where R共A兲 ⫽

共A ⫹ B兲共A ⫹ C兲 A⫹B⫹C⫹D

is a term that accounts for the number of hits that would be realized by random chance. BIAS is a ratio of the frequency of predicted and observed events. The perfect score for BIAS is equal to unity; a BIAS score of less than 1 indicates that the event was predicted fewer times than it was observed (underprediction) and vice versa. As a statistical skill score, ETS measures the improvement that a given forecast offers over a reference forecast. ETS is a modified version of the threat score [TS ⫽ A/(A ⫹ B ⫹ C)] that is rendered equitable by taking away the number of hits expected by random chance (Shaefer 1990). For a perfect forecast, ETS is equal to unity. A forecast that has the same skill as the reference forecast generates a null ETS value, whereas forecasts that perform worse than the reference forecast produce negative ETS values. Both BIAS and ETS were computed for the entire verification dataset as well as for the windward and lee side verification subsets.

4. Synoptic setting and precipitation patterns in the Sierra Nevada heavy-precipitation wintertime storms Winter storm systems that produce heavy precipitation in the ARB area are commonly embedded in a westerly or southwesterly airflow from the Pacific

VOLUME 133

Ocean, which steers the large-scale moisture supply toward the California coast (Heggli and Rauber 1988; Rauber 1992; Soong and Kim 1996). New research findings also show that the ARB area is often the terminus of several hundred kilometers (⬃400 km) wide “atmospheric rivers” of enhanced column water vapor originating in the Tropics (Ralph et al. 2004). In general, wintertime storms that produce the heaviest precipitation in the Sierra Nevada are warm winter storms associated with the southwesterly airflow, in which the precipitation episodes are significantly enhanced by orography (Dettinger et al. 2004). The bases of associated cloud systems that evolve over the mountains are above freezing temperatures while their cloud tops extend well above the freezing level (Rauber 1992). For these warm winter storms temperatures in the lower to midtroposphere are greater than ⫺20°C at 500 hPa, the mean freezing level falls in the range of 2 to 2.8 km, and the temperature at 700 hPa is about ⫺2°C upwind of the Sierra Nevada (Pandey et al. 1999). The synoptic-scale flow and model-predicted precipitation patterns are illustrated here for the 12–13 February 1986 storm. Rauber (1992) presents a comprehensive observational analysis of the microphysical structure and evolution of this storm. This welldocumented case was also subject of two earlier orographic precipitation studies, both employing a twodimensional modeling framework (Meyers and Cotton 1992; Colle and Zeng 2004). The 12–13 February 1986 storm developed over the south-central Pacific and moved across California in the southern branch of the jet stream. Aside from a landfalling Pacific front, an important kinematic ingredient of this event was orographic blocking, resulting in a Sierra-parallel barrier jet with a maximum strength of 25 m s⫺1 at 1 km AGL. Persistent moist air advection in the lower atmosphere produced a widespread precipitation pattern in north and central California and a shallow orographic cloud over the barrier on the first day of this event. As the synoptic system moved to the southeast, the next day the area of significant precipitation shifted directly over the mountain range (Fig. 3). The 1200 UTC 12 February sounding at Kingvale (39.32°N, 120.43°W, 1864.8 m ASL) shows a good agreement between the observed and model-predicted kinematic and thermodynamic profiles (Fig. 4). Both a strong cross-barrier flow of 25 m s⫺1 at 2 km above ground (at or above the Sierra crest height), and a weak barrier jet with the Sierraparallel, south-southwest wind in the lowest 1 km above ground are reproduced by the model. However, the model-predicted cross-barrier flow is too strong, in particular in the lowest 1.5 km above ground, and the barrier jet is more southerly in the model predictions. Even

OCTOBER 2005


2841

FIG. 3. (a) Geopotential height (contouring interval 30 m) and temperature (shaded) at 850 hPa at 1200 UTC 12 Feb 1986. (b) Same as in (a) except at 500 hPa. The geopotential height contouring interval is 60 m. (c) Map of the 24-h accumulated precipitation predicted by the model in the 13.5-km domain for the period between 0000 UTC 12 Feb and 0000 UTC 13 Feb. (d) Same as in (c) but for the period between 0000 UTC 13 Feb and 0000 UTC 14 Feb.

though Kingvale is located high on the upwind slope (cf. Fig. 1), a barrier jet was present there as well. A barrier jet on the upwind side, which appears to be a common ingredient of the Sierra Nevada wintertime storms (Parish 1982; Marwitz 1983, 1987), might be important for understanding the spatial precipitation patterns shown below. Maps of the 24-h accumulated precipitation for 13 February predicted by the model in the 4.5-km domain using the four microphysical parameterizations are shown in Fig. 5. At this resolution, differences in the predicted spatial precipitation distributions by different

schemes are noticeable, but they do not mask a dominant filamentary structure of the precipitation maxima on the upwind slope, with the maximum precipitation filaments oriented perpendicular to and connected to the Sierra crest. The location of these enhanced precipitation filaments appears closely related to the transverse ridges on the western Sierra Nevada slopes. This filamentary structure was a consistent pattern in all four winter storms included in this study. A similar spatial precipitation pattern, with precipitation amounts over ridges exceeding several times those over nearby valleys, was noted also in the analysis of high-resolution

2842


VOLUME 133

FIG. 4. Vertical profiles of (a) wind speed, (b) wind direction, (c) temperature, and (d) water vapor mixing ratio at Kingvale (Fig. 1). The observed and MM5-predicted profiles are for 1200 UTC 12 Feb 1986.

precipitation forecasts for the Olympic Mountains of Washington (Anders et al. 2004). Further verification that the model simulations correctly capture the synoptic structure and evolution of the four storms is given by the MAP (Fig. 6), which was computed using 3-h accumulations from the observations and the model predictions. These diagrams show that the timing of the significant heavy precipitation periods in the model predictions was generally in good

agreement with the observed course of precipitation events in these storms, with the exception perhaps of the 3–5 January 1987 storm, for which the model predicted a somewhat earlier onset of precipitation. It is apparent also that irrespective of the choice of the microphysical scheme, the model predicted amounts of precipitation are larger than those observed. This is also true at 1.5-km resolution (not shown here). The rest of the QPF verification results are presented in the next section.

OCTOBER 2005


2843

FIG. 5. Maps of the 24-h accumulated precipitation predicted by the model in the 4.5-km domain for the period between 0000 UTC 13 Feb and 0000 UTC 14 Feb for the (a) DUDH, (b) SCHUL, (c) GSFC, and (d) REIS schemes.

5. QPF verification results a. Mean areal precipitation The MAP values, computed using both the 3- and 24-h precipitation accumulations, show a consistent model propensity for precipitation overprediction on

both sides of the mountain range regardless of the microphysical scheme employed. Based on the RMSE of the 24-h MAP (Table 3), SCHUL appears to be the poorest performer on the windward side, and DUDH on the lee side. While not necessarily the one with the smallest errors at all resolutions, the REIS

2844


VOLUME 133

FIG. 6. MAP (mm) for the winter storms: (a) 17–19 Dec 1982, (b) 12–14 Feb 1986, (c) 14–22 Feb 1986, and (d) 3–5 Jan 1987, computed from the observations (solid line) and model predictions employing the four different microphysical parameterizations.

scheme seems to perform equally well on both sides of the mountain range.

b. Accuracy measures DMB and MAE values as a function of the microphysical scheme calculated from the model data at the horizontal resolutions of 13.5, 4.5, and 1.5 km are shown in Figs. 7 and 8. In addition to the entire verification domain, the DMB and MAE values were also computed for the windward and leeward subdomains.

As DMB is the ratio of the predicted to the observed net water mass, the perfect forecast should yield the DMB value of unity. From Fig. 7 it is apparent that for all four microphysical parameterizations the model overpredicts the precipitation on both the windward and lee slopes of the Sierra Nevada. It is also clear that increasing the horizontal resolution from 13.5 to 4.5 km amplifies these tendencies, and that the further increase in horizontal resolution to 1.5 km produces additional amount of overprediction, particularly on the lee slope.


OCTOBER 2005

TABLE 3. The RMSE (mm) for the 24-h MAP in the 13.5-, 4.5-, and 1.5-km domains, shown separately for the windward and leeward side. Microphysical scheme

13.5 km

4.5 km

1.5 km

Windward side

DUDH SCHUL GSFC REIS

29.5 45.9 45.4 28.4

31.2 63.0 52.7 44.2

40.3 74.1 62.0 53.0

Leeward side

DUDH SCHUL GSFC REIS

24.8 11.7 25.6 15.9

45.6 23.6 43.4 30.7

58.6 24.7 52.3 40.5

For all the schemes except SCHUL, the overprediction is more pronounced on the lee side. With a larger number of precipitation gauges located on the windward side, the “entire-domain” DMB values are closer to the DMB values for the windward side. Consistent with the MAP results, the largest overprediction on the windward and leeward side at both 4.5 and 1.5 km is obtained, respectively, with the SCHUL and DUDH schemes. It is these two schemes also that display the largest (DUDH) and the smallest (SCHUL) difference between the windward and leeward side overprediction. At all three horizontal resolutions, the lowest DMB values in the entire verification domain are obtained with the REIS scheme. The same conclusions emerge also from the MAE

2845

diagram in Fig. 8, as well as from the RMSE (not shown here), where the latter only more strongly emphasizes large errors. We note that for GSFC and REIS schemes, the differences between the windward and leeward side MAE and DMB values are of the opposite sign (negative for DMB versus positive for MAE). This seeming contradiction is resolved by recalling that the observed precipitation amounts on the lee slope are generally very small (cf. Fig. 2), so that with larger predicted than observed values there one can simultaneously obtain very large DMB and relatively small MAE values. In Fig. 9, the RMSE for the four schemes is shown as a function of the precipitation interval for the two highest model horizontal resolutions. For all the schemes, the RMSE minima lie in the second interval (0.254–12.7 mm). The RMSE maxima, on the other hand, appear either in the very lowest or one of the highest precipitation intervals. For the DUDH scheme, the maximum is in the lowest precipitation interval, the SCHUL and GSFC schemes reach the maximum in one of the three highest precipitation intervals, whereas the REIS scheme shows a tendency of producing large RMSE values at both ends of the precipitation scale. In general, SCHUL appears to be the scheme with the largest RMSE (in particular in the intermediate and large precipitation intervals), and REIS the scheme with the smallest RMSE. The latter is not true at the large end of the scale, where DUDH scheme is the one with the smallest errors.

FIG. 7. DMB for the windward side, leeward side, and both sides of the verification domain (Fig. 1) using the model results from the nested domains with the horizontal resolutions of (a) 13.5, (b) 4.5, and (c) 1.5 km.

2846


VOLUME 133

FIG. 8. Same as in Fig. 7 but for MAE.

c. Skill scores 1) BIAS We start the discussion of skill scores with BIAS, which is a simple ratio of the frequency of predicted and observed events without regard to the forecast accuracy. In terms of the spatial equivalents, this translates into a ratio of the size of the observed and forecasted precipitation areas without regard for the correct forecast area size. Figure 10 shows the BIAS scores for the four microphysical schemes as a function of the precipitation interval for the two highest model horizontal resolutions. At 4.5-km resolution, and irrespective of the choice of the microphysical scheme, we find the precipitation to be underpredicted (BIAS ⬃0.4–0.6) in the three lowest precipitation intervals and overpredicted (BIAS ⬎1.5) in the high intervals (⬎50.8 mm), with the BIAS scores close to unity in between. Increasing horizontal resolution to 1.5 km decreases BIAS slightly in the small to intermediate precipitation intervals, and increases it more significantly in the high intervals, accentuating differences in under- and overprediction between the extreme ends of the precipitation scale. For the horizontal resolution of 13.5 km (not shown here), the BIAS distribution is similar to that at 4.5 km except that there is a wider range of intervals at the small to intermediate end of scale with BIAS scores close to unity. Both the underprediction of small precipitation amounts and overprediction of large amounts is caused by the model tendency to predict large precipitation amounts (ⱖ101.6 mm) where the observed precipita-

tion is generally small (often less than 0.254 mm). Analyzing the windward and leeward side BIAS separately (Fig. 11), we find more extreme BIAS distributions on the lee side, with a severe underprediction in the smallest precipitation interval, and a more pronounced overprediction in the high-precipitation intervals. While the windward BIAS distributions at different horizontal resolutions closely parallel that for the entire domain, the lee side BIAS shows an interesting change from an overprediction in the intervals from 0.254 to 38.1 mm at 13.5 km to a slight underprediction in those intervals at higher resolutions.

2) ETS To gain information on the size of the correct forecast area it is helpful to examine some form of threat score. Here we discuss the ETS scores, which additionally convey information on the quality of the particular forecast related to a random forecast. The ETS scores for the four microphysical schemes as a function of the precipitation interval are shown in Fig. 12 at the two highest model horizontal resolutions. Irrespective of the choice of the microphysical scheme, we find that the skill of the MM5 model in placing the given amount of precipitation in the right location is rather low for a wide range of precipitation intervals (from 12.7 to 101.6 mm). Additionally, this skill appears to be no better than what would be achieved by a random forecast (ETS values close to 0). Some skill, although rather low (0.1 ⬍ ETS ⬍ 0.25), is displayed for the precipitation amounts in the lowest (⬍12.7 mm) and the highest precipitation intervals (ⱖ101.6 mm). A

OCTOBER 2005


2847

FIG. 9. (a) Histogram of RMSE (mm) for the defined precipitation intervals (mm) for the four microphysical schemes for the 4.5-km model domain. (b) Same as in (a) but for the 1.5-km domain.

slight increase of skill in the lowest precipitation interval, and a more pronounced decrease of skill for the largest precipitation interval, was obtained for increasing the horizontal resolution from 4.5 to 1.5 km. In analyzing the windward and leeward sides separately (Fig. 13), in this case we find a distinctly different result for the two regions. The windward ETS scores, again similar to those for the entire domain, reveal an even higher skill (0.2 ⬍ ETS ⬍ 0.3) in the extreme precipitation intervals. The skill on the lee side, however, was found to be uniformly low across all the precipitation intervals with ETS values generally between 0 and 0.1. In spite of varying degrees of complexity of the four microphysical schemes, the differences in the BIAS and ETS skill scores between the individual schemes appear rather small, lacking any clearly recognizable pattern.

In the next section we discuss whether the obtained differences in the BIAS and ETS scores for the four schemes carry any statistical significance.

d. Statistical significance The significance test for the differences in ETS and BIAS applied in this study is based on a resampling method following Hamill (1999). The essence of this method lies in forming a resampled distribution of the statistics of interest (by selecting a large number of special random samples from the original population), and determining whether the true statistics falls within predetermined probability ranges defining confidence intervals of the resampled distribution. The test statistic in our case is the difference in skill scores (BIAS and ETS) obtained for different microphysical schemes. In constructing these differences, we

2848


VOLUME 133

FIG. 10. (a) BIAS scores as a function of the precipitation interval (mm) for the four microphysical schemes based on the model predictions from the 4.5-km domain. (b) Same as in (a) but for the 1.5-km domain.

have selected DUDH to be the reference scheme as it is the simplest of the four schemes in this study. Our conclusions, however, do not depend on this choice as confirmed by selecting another scheme and repeating the procedure. Details of the resampling procedure and hypothesis testing are explained in appendix B. Results of the testing for both BIAS and ETS scores are shown in Fig. 14, in which 95% confidence intervals are shown as error bars. We note a rather wide variation in the width of the obtained confidence intervals for BIAS. Narrow confidence intervals for the low-precipitation amounts point to a narrowly distributed spectrum of differences compared to the higher-precipitation intervals. For the latter, a larger number of data points would be beneficial for obtaining a better estimate of the confidence interval.

Based on the results in Fig. 14, it appears that the skill scores obtained for the REIS, GSFC, and SCHUL schemes are not statistically different from those for the DUDH scheme as all the differences in BIAS and ETS scores fall within the determined confidence intervals. In other words, the more complex microphysics in those schemes does not contribute significantly toward improving the QPF skill scores of the MM5 model over those obtained with the simplest microphysical representation already contained in the DUDH scheme.

6. Discussion The results of our statistical verification study indicate that whereas the skill and accuracy of the MM5

OCTOBER 2005


2849

FIG. 11. (a) BIAS scores as a function of the precipitation interval (mm) for the four microphysical schemes on the windward side of the 4.5-km model domain. (b) Same as in (a) but for the leeward side. For the three highest precipitation intervals, BIAS values exceeding 4.5 are not shown.

model in predicting a correct spatial distribution of wintertime precipitation in the Sierra Nevada do not significantly vary across four different bulk microphysical parameterizations used, the sensitivity of the skill and accuracy to the horizontal resolution does exist. The nature of that sensitivity is largely counterintuitive, with successively higher resolutions leading to larger errors and lower skill. As this dependence of the total model error and skill on the horizontal resolution is opposite to the known dependence of the representativeness error in the area-to-point verification (Tustison et al. 2001), this means that the model error makes up a larger portion of the total error. The statistical results also indicate that higher horizontal resolutions only accentuate differences in specific behavior of the bulk

schemes examined, but do not lead to qualitatively different predictions by these schemes. For example, the noted tendency of the DUDH and GSFC schemes to place more precipitation on the leeward side compared to the other two schemes, and the stronger overprediction on the windward side by the SCHUL scheme, worsen as the horizontal resolution is increased. To illustrate this point, in Fig. 15 we show maps of the 24-h accumulated precipitation in the 1.5-km domain for the same time period as shown in Fig. 5. While the filamentary structure of the precipitation maxima is clearly visible at 1.5 km as well, it is the differences between individual schemes’ prediction patterns that stand out at this resolution. It is immediately noted that the pronounced leeward side overprediction produced

2850


VOLUME 133

FIG. 12. Same as in Fig. 10 but for ETS.

by the DUDH scheme stems from the combined effect of the presence of high terrain on the lee side of the Sierra Nevada (i.e., the Carson Range, the north–southoriented ridge that forms the eastern wall of the Tahoe basin) and the propensity of this scheme to deposit precipitation on top of the highest terrain. To gain further insight into the specific behavior of individual schemes, microphysical variables predicted by these schemes are shown in the vertical cross sections along line AB that passes through many of our upwind and leeward side verification points (cf. Fig. 1). Without supercooled water and graupel, the DUDH scheme (Figs. 16a and 17a) produces less precipitation on the lower windward slopes and significantly more snow aloft than either the SCHUL or REIS schemes (Figs. 16b,d and 17b,d). This

snow is deposited on high peaks that lie above the freezing level, but is also advected to the leeward side. Even though GSFC is a mixed-phase scheme, the snow concentrations aloft and precipitation at the ground it produces are more similar to DUDH than either to the SCHUL and REIS schemes. The larger amount of graupel and less snow in the SCHUL and REIS scheme forecasts (Figs. 16b and 17b), result in more fallout over the windward slope and less in the lee. These two schemes also produce similar precipitation patterns at the ground. These findings are consistent with those of Colle and Zeng (2004), who have conducted a detailed microphysical sensitivity analysis employing these same schemes in an idealized 2D MM5 modeling study of the 12 February 1986 event.

OCTOBER 2005


2851

FIG. 13. Same as in Fig. 11 but for ETS.

7. Summary and conclusions We have examined the skill of the MM5 model in predicting the orographic precipitation in high-impact wintertime precipitation events in the Sierra Nevada, and the sensitivity of that skill to the choice of the microphysical parameterization and horizontal resolution. The performance of four existing microphysical schemes in MM5 (DUDH, SCHUL, GSFC, and REIS) was examined. High-resolution numerical simulations were carried out on three nested domains, with horizontal grid increments of 13.5, 4.5, and 1.5 km. The domain of verification was the innermost domain of the MM5 model. To facilitate the analyses for the windward and leeward sides separately, two verification sub-

domains were defined within the main verification domain. The verification dataset consisted of ground precipitation measurements obtained during a selected number of wintertime storms documented during the SCPP experiment in the 1980s. For the verification purposes, the measured and predicted precipitation amounts within a given 24-h forecast period were divided into the following precipitation intervals: ⬍0.254, 0.254–12.7, 12.7–25.4, 25.4–38.1, 38.1–50.8, 50.8–76.2, 76.2–101.6, and ⱖ101.6 mm. The total number of individual observations in the entire verification domain was 728, with 462 and 266 lying on the windward and leeward sides, respectively. Continuous and categorical statistical measures were used to quantify the model QPF and the forecast errors.

2852


VOLUME 133

FIG. 14. (a) Differences in BIAS scores between DUDH and the other three examined microphysical schemes as a function of the precipitation interval. Shown are the results for the 4.5-km domain. Error bars indicate the 95% confidence interval range (2.5th and 97.5th percentiles of the resampled distribution). (b) Same as in (a) but for ETS.

In the first category, the accuracy measures used were mean areal precipitation (MAP), degree of mass balance (DMB), mean absolute error (MAE), and rootmean-square error (RMSE). Categorical statistics was employed to construct the skill scores for QPF verification. Two scores, BIAS, and equitable threat score (ETS), were computed to evaluate the model QPF skill in the defined precipitation intervals. The continuous verification statistics showed that all the examined microphysical schemes tend to overpredict the 24-h precipitation accumulations on both the windward and the leeward slopes of the Sierra Nevada. Furthermore, the QPF errors of the 24-h precipitation accumulations were found to increase with increasing horizontal reso-

lution. The largest tendency for overprediction is displayed by the SCHUL scheme on the windward slopes and by the DUDH scheme on the leeward slopes, whereas the REIS scheme is the best performer when both sides of the mountain range are considered together. In agreement with the earlier studies, the categorical QPF skill scores were not improved by reducing the numerical grid size below 13.5 km. Irrespective of the choice of the microphysical scheme, we found the precipitation to be underpredicted for the smallest precipitation amounts, and overpredicted for the largest amounts. Both underprediction of small precipitation amounts and overprediction of large amounts was

OCTOBER 2005


2853

FIG. 15. Maps of the 24-h accumulated precipitation predicted by the model in the 4.5-km domain for the period between 0000 UTC 13 Feb and 0000 UTC 14 Feb for the (a) DUDH, (b) SCHUL, (c) GSFC, and (d) REIS schemes.

caused by the model tendency to predict large precipitation amounts where the observed precipitation was small. Even though for these extreme precipitation intervals the size of the predicted area is either too large or too small compared to the observed area as revealed by BIAS, it is in these precipitation classes that the model shows some skill in placing the predicted pre-

cipitation in the right locations as inferred from ETS. In contrast, for the intermediate precipitation intervals, for which the BIAS scores are close to unity, we find that, irrespective of the microphysical scheme, the skill of the MM5 model in placing the given amount of precipitation in the right location is rather low and no better than what would be achieved by a random forecast

2854


VOLUME 133

FIG. 16. Mixing ratio of cloud water (g kg⫺1; thin solid) and cloud ice (g kg⫺1; dashed) predicted by the four microphysical schemes at 1200 UTC 13 Feb 1986, shown in the vertical cross section along line AB (Fig. 1). Shown are also wind vectors, and zero (°C) temperature isoline (freezing level; thick solid). The contour intervals for cloud water and cloud ice are 0.1 and 0.05. The wind vector scale is shown in the lower right corner of each panel.

(ETS values close to 0). On average, for all the precipitation intervals, MM5 QPF skill is better on the windward side compared to the leeward side of the Sierra Nevada. In spite of the varying degree of complexity of the four tested microphysical schemes, the differences in the BIAS and ETS skill scores for the individual schemes appear rather small, lacking any clearly recognizable pattern. Testing the statistical significance of these differences, reveals that the skill scores obtained for the REIS, GSFC, and SCHUL schemes are not statistically different from those obtained for the DUDH

scheme. The conclusion that emerges from this result is that the more sophisticated microphysics contained in those schemes does not on average contribute significantly toward improving the QPF skill of the MM5 model over the skill that is achieved with the simplest microphysical representation already contained in the DUDH scheme. The statistical results also indicate that higher horizontal resolutions only accentuate differences in specific behavior of the bulk schemes examined, but do not lead to qualitatively different predictions by any of these schemes. The overprediction of precipitation on both the up-

OCTOBER 2005


2855

FIG. 17. Same as in Fig. 16 but for the mixing ratio of rain (g kg⫺1; thin solid), snow (g kg⫺1; thin dashed), and graupel (g kg⫺1; thick dashed). The contouring interval for all three mixing ratios is 0.1. All contours in (a) are solid as the same variable is used for rain and snow in the DUDH scheme.

wind and lee sides of the Sierra Nevada documented in this study stands in contrast to the study by Colle and Mass (2000), who have found that, at comparable horizontal resolutions, the MM5 model predicted too much precipitation along the windward slopes and too little in the lee of the Cascade Range during a strong wintertime precipitation event. While the need to better resolve the mountain-induced circulations was noted, the excessive precipitation shadowing produced by the model was attributed to fall speeds of hydrometeors being too large, leaving too little ice aloft to be carried over to the lee side. Our findings, however, agree quite

nicely with new Improvement of Microphysical Parameterization through Observational Verification Experiment (IMPROVE-2) results (Garvert et al. 2005; Colle et al. 2005), which, taking advantage of the extensive IMPROVE-2 datasets, including radar data and in situ microphysical measurements by aircraft, show that the MM5 model (with the Reisner2 scheme) overpredicts precipitation on both the upwind and the lee slopes of the Cascade Range. In these new studies, the leeward side precipitation overprediction is diagnosed to be due to too much snow aloft that is carried over to the leeward side.

2856


The filamentary spatial pattern of precipitation on the upwind Sierra Nevada slopes, with the maximum precipitation filaments oriented perpendicular to and connected to the Sierra crest, was found to be common to all the examined storms. The location of these enhanced precipitation filaments appears closely related to the transverse ridges on the western Sierra Nevada slopes, with the local precipitation maxima lying on top of the ridges. As the horizontal resolution is increased, differences in the spatial precipitation patterns are found to increase, revealing the distinct characteristics resulting from different microphysics contained in the schemes, showing, for example, that the DUDH and GSFC schemes place more precipitation on the lee slopes compared to the other two schemes. The likely reason for this behavior is production of too much ice aloft by these schemes that is then rapidly advected over the mountain crest to the lee side in the strong cross-barrier flow. Too much precipitation on the lee side could result also from unrealistically small hydrometeor fall speeds as well as from overly strong modelpredicted cross-barrier flow, leading to the spurious spillover. A more detailed investigation of the orographic airflow in these precipitation events is needed, and is planned for the future, to document the threedimensional structure of the cross-barrier flow and the barrier jet, and their effect on the resultant precipitation processes and spatial precipitation distribution in the Sierra Nevada, similar to studies recently conducted in the Alps and the Oregon Cascades (Medina et al. 2005). The results of this study confirm that the model horizontal resolution is not the key to obtaining the critically needed improvement in the QPF skill of mesoscale models. These results also show that the differences in sophistication of the existing single- and mixed-phase bulk microphysical schemes do not lead to statistically significant differences in QPF skill of the MM5 model, which for the Sierra Nevada orographic precipitation is fairly low. While improving the shortcomings of the existing bulk microphysical schemes is likely to lead to further incremental improvement in the QPF skill, the gap that needs to be closed between the current and ideal forecasts appears rather large. It is conceivable, however, that other sources of QPF error, such as initialization errors in general, and of water substance in particular, are limiting the performance of the bulk microphysical schemes, and that with better initial conditions one would see a larger spread in the QPF skill obtained with these schemes. Acknowledgments. We thank three anonymous reviewers for detailed and insightful comments. The

VOLUME 133

MM5 runs for this study were performed on the SGI Altix 3700 of the Nevada Environmental Computing Grid (NECG). NECG has been developed under the auspices of the ACES program (www.aces.dri.edu), which is funded by NSF EPSCoR and led by the first author. This research was supported in part by the U.S. Weather Research Program (USWRP) NSF Grant ATM-9908995, and by the DOE ARM Grant DE-FD03-00ER62935.

APPENDIX A Mean Areal Precipitation Mean areal precipitation (MAP) gives the areal average value of precipitation amount for a specific time period, computed from a regular or irregular network of precipitation observations throughout the area of interest. Let us consider that there are n stations reporting precipitation amounts. In this study, the irregular network of observations is triangulated using the Delaunay triangulation procedure (Rourke 1998). This is the most commonly used optimized two-dimensional triangulation procedure in which k triangles are constructed using the spatial positions of n stations, with a criterion that no fourth observing station lies within the circumcircle of each triangle. This procedure yields triangles that are as close as possible to being equilateral. The precipitation amounts are then calculated at the centroid of the triangles and MAP is computed as follows: MAP ⫽

兺 a P 冒兺 a , k

k

k

k

k

where Pk are the precipitation amounts at the centroid of triangles, and ak is the area of triangles.

APPENDIX B Resampling Method The purpose of the resampling method (sometimes referred as the “bootstrap method”) is to estimate confidence intervals (CI) for some statistic of interest. Briefly, new datasets (samples) are generated by randomizing the original dataset (population) in a prescribed way (Hamill 1999). Analogous to Hamill’s procedure, in which differences in the BIAS and ETS scores obtained for two different forecasting systems over a wide range of weather conditions were examined, we evaluated the differences in the skill scores obtained for two different microphysical schemes of the same forecasting system.


OCTOBER 2005

The test statistic of interest is the difference in the BIAS or ETS scores, obtained for two microphysical schemes. It is defined as 共BIAS1 ⫺ BIAS2兲

or 共ETS1 ⫺ ETS2兲,

共B1兲

where subscript 1 stands for DUDH and subscript 2 for any one of the three other schemes (SCHUL/GSFC/ REIS). The statistical test to evaluate the difference in the scores is based on two competing hypotheses; the null hypothesis, denoted by Ho, against the alternative hypothesis, denoted by HA. These hypothesis are HO⬚: BIAS1 ⫺ BIAS2 ⫽ 0,

共B2兲

ETS1 ⫺ ETS2 ⫽ 0,

共B3兲

HA: BIAS1 ⫺ BIAS2 ⫽ 0,

共B4兲

ETS1 ⫺ ETS2 ⫽ 0.

共B5兲

The confidence interval, determined from the resampled distribution of the test statistics (constructed consistent with the null hypothesis) assuming some probability thresholds, provides a range of values for testing the given statistics. If the test statistics from the original population falls within this confidence interval, the null hypothesis is accepted. Otherwise, the alternative hypothesis is true. The resampling procedure following Hamill (1999) is briefly explained here. The vector of contingency table elements A, B, C, and D, as shown in Table 2, is given by 共A, B, C, D兲i,

i ⫽ 1, 2,

where index i determines the choice of the microphysical scheme. The test statistic [Eq. (B1)] is calculated by substituting these elements in Eqs. (5) and (6). To form the resampled distributions of our statistics consistent with the null hypothesis, two sets of samples (with m ⫽ 364 elements) were constructed by randomly selecting forecasts obtained with either scheme 1 or scheme 2 from the original population (n ⫽ 728 elements) of the 24-h precipitation forecasts. This selection was carried out k (⫽ 5000) times. For each of those k random samples of m forecasts, contingency table elements were determined and the skill scores computed for all the defined precipitation intervals. The resampled test statistic from these samples (e.g., ˆS1 ⫺ BIA ˆS2) were computed next, and the reBIA sampled distributions of k elements constructed for each of the precipitation intervals. Finally, the 95% CIs were determined as the 2.5th and 97.5th percentiles of the resampled distributions, which were close to the true Gaussian distributions. The confidence intervals

2857

for the differences in BIAS and ETS scores between the DUDH and the other schemes, determined using the above procedure, are shown in Fig. 14 as error bars. REFERENCES Anders, A. M., G. H. Roe, and D. R. Durran, 2004: Precipitation and the form of mountains. Bull. Amer. Meteor. Soc., 85, 498–499. Bright, D. R., and S. L. Mullen, 2002: The sensitivity of the numerical simulation of the southwest monsoon boundary layer to the choice of PBL turbulence parameterization in MM5. Wea. Forecasting, 17, 99–114. Bruintjes, R. T., T. L. Clark, and W. D. Hall, 1994: Interactions between topographic airflow and cloud/precipitation development during the passage of a winter storm in Arizona. J. Atmos. Sci., 51, 48–67. Cayan, D. R., and D. H. Peterson, 1989: The influence of North Pacific atmospheric circulation on streamflow into the west. Aspects of Climate Variability in the Pacific and Western Americas, Geophys. Monogr., No. 55, Amer. Geophys. Union, 375–396. Charba, J. P., D. W. Reynolds, B. E. McDonald, and G. M. Carter, 2003: Comparative verification of recent quantitative precipitation forecasts in the National Weather Service: A simple approach for scoring forecast accuracy. Wea. Forecasting, 18, 161–183. Colle, B. A., and C. F. Mass, 2000: The 5–9 February 1996 flooding event over the Pacific Northwest: Sensitivity studies and evaluation of the MM5 precipitation forecasts. Mon. Wea. Rev., 128, 593–617. ——, and Y. Zeng, 2004: Bulk microphysical sensitivities with the MM5 for orographic precipitation. Part I: The Sierra 1996 event. Mon. Wea. Rev., 132, 2780–2801. ——, K. J. Westerick, and C. F. Mass, 1999: Evaluation of MM5 and Eta-10 precipitation forecasts over the Pacific Northwest during the cool season. Wea. Forecasting, 14, 137–154. ——, C. F. Mass, and K. J. Westrick, 2000: MM5 precipitation verification over the Pacific Northwest during the 1997–99 cool season. Wea. Forecasting, 15, 730–744. ——, M. Garvert, J. B. Wolfe, and C. F. Mass, 2005: The 13–14 December 2001 IMPROVE-2 event. Part III: Simulated microphysical budgets and sensitivity studies. J. Atmos. Sci., 62, 3535–3558. Cressman, G. P., 1959: An operational objective analysis system. Mon. Wea. Rev., 87, 367–374. Dettinger, M. D., D. R. Cayan, H. F. Diaz, and D. M. Meko, 1995: Large-scale atmospheric forcing of recent trends toward early snowmelt in California. J. Climate, 8, 606–623. ——, K. Redmond, and D. Cayan, 2004: Winter orographic precipitation ratios in the Sierra Nevada—Large-scale atmospheric circulations and hydrologic consequences. J. Hydrometeor., 5, 1102–1116. Dudhia, J., 1989: Numerical study of convection observed during the winter monsoon experiment using a mesoscale twodimensional model. J. Atmos. Sci., 46, 3077–3107. Ebert, E. E., U. Damarath, W. Wergen, and M. E. Baldwin, 2003: The WGNE assessment of short-term quantitative precipitation forecasts. Bull. Amer. Meteor. Soc., 84, 481–492. Emanuel, K. A., and Coauthors, 1995: Report of the first prospectus development team of the U.S. Weather Research Pro-

2858


gram to NOAA and the NSF. Bull. Amer. Meteor. Soc., 76, 1194–1208. Fritsch, J. M., and Coauthors, 1998: Quantitative precipitation forecasting: Report of the eighth prospectus development team, U.S. Weather Research Program. Bull. Amer. Meteor. Soc., 79, 285–299. Garvert, M. F., B. A. Colle, and C. F. Mass, 2005: The 13–14 December 2001 IMPROVE-2 event. Part I: Synoptic and mesoscale evolution and comparison with a mesoscale model simulation. J. Atmos. Sci., 62, 3474–3492. Gaudet, B., and W. R. Cotton, 1998: Statistical characteristics of real-time precipitation forecasting model. Wea. Forecasting, 13, 966–982. Grell, G. A., J. Dudhia, and D. R. Stauffer, 1994: Description of the fifth-generation Penn State/NCAR Mesoscale Model (MM5). NCAR Tech. Note TN-398, 128 pp. Hamill, T. M., 1999: Hypothesis tests for evaluating numerical precipitation forecasts. Wea. Forecasting, 14, 155–167. Heggli, M. F., and R. M. Rauber, 1988: The characteristics and evolution of supercooled water in wintertime storms over the Sierra Nevada: A summary of microwave radiometric measurements taken during the Sierra Cooperative Pilot Project. J. Appl. Meteor., 27, 989–1015. Hsie, E.-Y., R. A. Anthes, and D. Keyser, 1984: Numerical simulation of frontogenesis in a moist atmosphere. J. Atmos. Sci., 41, 2581–2594. James, C. N., and R. A. Houze Jr., 2005: Modification of precipitation by coastal orography in storms crossing northern California. Mon. Wea. Rev., in press. Janjic´ , Z. I., 1994: The step-mountain Eta coordinate model: Further developments of the convection, viscous sublayer, and turbulence closure schemes. Mon. Wea. Rev., 122, 927–945. Kain, J. S., 2004: The Kain–Fritsch convective parameterization: An update. J. Appl. Meteor., 43, 170–181. ——, and J. M. Fritsch, 1993: Convective parameterization for mesoscale models: The Kain–Fritsch scheme. The Representation of Cumulus Convection in Numerical Models, Meteor. Monogr., No. 24, Amer. Meteor. Soc., 165–170. Kessler, E., 1969: On the Distribution and Continuity of Water Substance in Atmospheric Circulations. Meteor. Monogr., No. 10, Amer. Meteor. Soc., 84 pp. Klemp, J. B., and D. R. Durran, 1983: An upper boundary condition permitting internal gravity wave radiation in numerical mesoscale models. Mon. Wea. Rev., 111, 430–444. Lin, Y.-L., R. D. Farley, and H. D. Orville, 1983: Bulk parameterization of the snow field in a cloud model. J. Climate Appl. Meteor., 22, 1065–1092. Marwitz, J. D., 1983: The kinematics of orographic airflow during Sierra storms. J. Atmos. Sci., 40, 1218–1227. ——, 1987: Deep orographic storms over the Sierra Nevada. Part I: Thermodynamic and kinematic structure. J. Atmos. Sci., 44, 159–173. McDonald, B. E., 1998: Sensitivity of precipitation forecast skill to the horizontal resolution. Ph.D. dissertation, University of Utah, 135 pp. Medina, S., B. F. Smull, R. A. Houze Jr., and M. Steiner, 2005: Cross-barrier flow during orographic precipitation events: Results from MAP and IMPROVE. J. Atmos. Sci., 62, 3580– 3598. Mellor, G. L., and T. Yamada, 1982: Development of a turbulence closure model for geophysical fluid problems. Rev. Geophys. Space Phys., 20, 851–875.

VOLUME 133

Meyers, M. P., and W. R. Cotton, 1992: Evaluation of the potential for wintertime quantitative precipitation forecasting over mountainous terrain with an explicit cloud model. Part I: Two-dimensional sensitivity experiments. J. Appl. Meteor., 31, 26–50. Milbrandt, J., and M. K. Yau, 2001: A mesoscale modeling of the 1996 Saguenay Flood. Mon. Wea. Rev., 129, 1419–1440. Montani, A., A. J. Thorpe, R. Buizza, and P. Unden, 1999: Forecast skill of the ECMWF model using targeted observations during FASTEX. Quart. J. Roy. Meteor. Soc., 125, 3219–3240. Murphy, A. H., and H. Daan, 1985: Forecast evaluation. Probability, Statistics, and Decision Making in the Atmospheric Sciences, A. H. Murphy and R. W. Katz, Eds., Westview Press, 379–437. Pandey, G. R., D. R. Cayan, and K. P. Georgakakos, 1999: Precipitation structure in the Sierra Nevada of California during winter. J. Geophys. Res., 104, 12 019–12 030. Parish, T. R., 1982: Barrier winds along the Sierra Nevada mountains. J. Appl. Meteor., 21, 925–930. Ralph, F. M., P. J. Nieman, and G. A. Wick, 2004: Satellite and CALJET aircraft observations of atmospheric rivers over the eastern North Pacific Ocean during the winter of 1997/98. Mon. Wea. Rev., 132, 1721–1745. Rauber, R. M., 1992: Microphysical structure and evolution of a Sierra Nevada shallow orographic cloud system. J. Appl. Meteor., 31, 3–24. Reisner, J., R. M. Rasmussen, and R. T. Bruintjes, 1998: Explicit forecasting of supercooled liquid water in winter storms using MM5 mesoscale model. Quart. J. Roy. Meteor. Soc., 124, 1071–1107. Reynolds, D. W., and A. S. Dennis, 1986: A review of the Sierra Cooperative Pilot Project. Bull. Amer. Meteor. Soc., 67, 513– 523. ——, and A. P. Kuciauskas, 1988: Remote and in situ observations of Sierra Nevada winter mountain clouds: Relationships between mesoscale structure, precipitation and liquid water. J. Appl. Meteor., 27, 140–156. Rourke, J., 1998: Computational Geometry in C. Cambridge University Press, 376 pp. Schultz, D. M., and Coauthors, 2002: Understanding Utah winter storms: The Intermountain Precipitation Experiment. Bull. Amer. Meteor. Soc., 83, 189–210. Schultz, P., 1995: An explicit cloud physics parameterization for operational numerical weather prediction. Mon. Wea. Rev., 123, 3331–3343. ——, and S. Snook, 1996: Objective verification of mesoscale model precipitation forecasts. Preprints, 15th Conf. on Weather Forecasting and Analysis, Norfolk, VA, Amer. Meteor. Soc., 124–126. Shaefer, J. T., 1990: The critical success index as an indicator of warning skill. Wea. Forecasting, 5, 570–575. Smith, R. B., 1979: The influence of mountains on the atmosphere. Advances in Geophysics, Vol. 21, Academic Press, 87–230. Soong, S., and J. Kim, 1996: Simulation of a heavy wintertime precipitation event in California. Climate Change, 32, 55–77. Stanski, H. R., L. J. Wilson, and W. R. Burrows, 1989: Survey of common verification methods in meteorology. Atmospheric Environment Service, Toronto, ON, Canada, Research Rep. 89-5, 114 pp. Stoelinga, M. T., P. V. Hobbs, C. F. Mass, J. D. Locatelli, N. A. Bond, B. A. Colle, R. A. Houze Jr., and A. Rango, 2003:

OCTOBER 2005


Improvement of Microphysical Parameterization through Observational Verification Experiment (IMPROVE). Bull. Amer. Meteor. Soc., 84, 1807–1826. Tao, W.-K., and J. Simpson, 1993: Goddard Cumulus Ensemble Model. Part I: Model description. Terr. Atmos. Oceanic Sci., 4, 35–72. Thompson, G., R. Rasmussen, and K. Manning, 2004: Explicit forecasts of winter precipitation using an improved bulk microphysics scheme. Part I: Description and sensitivity analysis. Mon. Wea. Rev., 132, 519–542.

2859

Tustison, B., D. Harris, and E. Foufoula-Georgiou, 2001: Scale issues in verification of precipitation forecasts. J. Geophys. Res., 106, 11 775–11 784. Wilks, D., 1995: Statistical Methods in Atmospheric Sciences: An Introduction. Academic Press, 467 pp. Yang, D., B. E. Goodison, J. R. Metcalfe, V. S. Golubev, R. Bates, T. Pangburn, and C. L. Hanson, 1998: Accuracy of NWS 8⬙ standard nonrecording precipitation gauge: Results and application of WMO intercomparison. J. Atmos. Oceanic Technol., 15, 54–68.