Shrunken Locally Linear Embedding for Passive ...

Shrunken Locally Linear Embedding for Passive Microwave Retrieval of Precipitation Ardeshir M. Ebtehaj∗

Rafael L. Bras∗

Efi Foufoula-Georgiou†

Abstract This paper introduces a new approach to the inverse problem of passive microwave rainfall retrieval. The proposed methodology relies on modern supervised manifold learning and regularization paradigms, which makes use of two joint dictionaries of coincidental rainfall profiles and their upwelling spectral radiative fluxes. A sequential detection-estimation strategy is adopted which relies on a geometrical perception that similar rainfall intensity values and their spectral radiances lie on or live close to some sufficiently smooth manifolds with analogous geometrical structure. The detection step employs of a nearest neighborhood classification rule, while the estimation scheme is equipped with a constrained shrinkage estimator to ensure sufficiently stable retrieval and some physical consistency. The algorithm is examined using coincidental observations of the active precipitation radar (PR) and passive microwave imager (TMI) on board the Tropical Rainfall Measuring Mission (TRMM) satellite. We present improved instantaneous retrieval results for a wide range of storms over ocean, land, and coastal zones and demonstrate that the algorithm is capable of recovering high-intensity rain-cells and captures sharp gradients in storm morphology. The algorithm is also compared at an annual scale for calendar year 2013 versus the current standard rainfall product 2A12 (version 7) of the TRMM and marked improved retrieval skill is reported.

1

Introduction

The history of remote sensing of rainfall traces back to the World War II, where the radar operators noticed the hydrometeors’ nuisance echoes on military radar screens. In mid 50’s, these observations became the motivation for developing land-based weather radars for active sensing of mesoscale atmospheric activities and precipitation forecasting. The launch of the U.S. Environmental Science Services Administration Satellite (ESSA) program in 1966 initiated a large body of research aiming at estimating surface rainfall from spaceborne passive observations at a global scale. The early attempts were focusing on the use of cloud photography in the visible range of the electromagnetic (EM) spectrum. Upon successful launch of Synchronous Meteorological Satellites (SMS) in mid 70’s, interest shifted to rainfall estimation using infrared (IR) wavelengths. In parallel, research was devoted to understanding the physical relationship of the surface rainfall and upwelling spectral radiance in microwave wavelengths. This line of research gained significant momentum upon the successful launch of the Microwave Sounding Unit (MSU) in 1979; Special Sensor Microwave Imager (SSM/I) in 1987; and the Tropical Rainfall Measuring Mission (TRMM) in 1997. The TRMM satellite was the first of its kind that successfully carried a single polarization Ku-band weather radar along with a multichannel radiometer which has provided a wealth of highly accurate information about the rainfall global patterns and distributions over the tropics. From a mathematical standpoint, rainfall retrieval from remotely sensed observations is an inverse problem in which we aim to estimate the rainfall intensity from its indirect and noisy remotely sensed observations. Active retrieval of surface rainfall from returned echoes of weather radars is relatively straightforward. This retrieval typically requires a proper characterization of the rainfall drop-size distribution and uses the well known Z-R relationships [1, 2]. On the other hand, passive retrieval of rainfall from observed upwelling spectral radiance is typically much more involved, chiefly because the observations are often highly corrupted with noise, downsampled, and non-linearly related to the rainfall vertical profile. In general, depending on the sampling interval of the EM spectrum, rainfall retrieval from passive observations relies on two distinct approaches. Typically, those methods which use the visible and infrared range of the EM spectrum are empirical and seek a functional mapping for linking the observed spectral signatures to the surface rainfall intensity values. On the other hand, in the microwave wavelengths the retrieval methods rely on both empirical and physically-driven approaches. ∗ School of Civil and Environmental Engineering, Georgia Institute of Technology, Atlanta GA (e-mail: [email protected]; [email protected]) † Department of Civil Engineering, University of Minnesota, Minneapolis MN (e-mail: [email protected])

1

In the visible range of the EM spectrum, clouds are more reflective than their surroundings depending on their texture and thickness. This hypothesis is of course more significant over tropics and subtropics where convective activities are the main driver of precipitation events rather than the extratropics where stratiform precipitation dominates due to the baroclinic systems [3]. Motivated by this hypothesis, in one of the earliest attempts, [4] proposed a cloud indexing methodology in combination with a simple polynomial regression to estimate rainfall intensities using cloud photography provided by the ESSA-1 satellite. [5] and [6] also examined the linear correlation between rainfall intensity and cloud reflectance in the visible light wavelengths. Given the high-quality day/night cloud-cover data provided by the Visible Infrared Spin-Scan Radiometer (VISSR) aboard the Synchronous Meteorological Satellite (SMS-1), [7] studied the linear relationship between fractional cloud coverage and accumulated rainfall, using ground-based C-band radar data in the GARP Atlantic Tropical Experiments (GATE) in 1974. [8] focused on cumulus and stratus systems and suggested a two-dimensional pattern recognition approach to detect raining areas of the clouds. In this study, it was concluded that the visible and IR wavelengths chiefly respond to the density of water vapor and cloud particles and not directly to the vertical structure of hydrometeors. A review of rainfall statistical retrieval techniques in visible and IR bands –up to mid 80’s– is documented in [9]. In addition, successful implementation of machine learning approaches, using artificial neural networks, is reported by [10] for cloud classification and rainfall estimation in infrared bands and further extended to operational levels in [11, 12, 13]. In microwave frequencies (∼6-to-200 GHz), the hydrometeor vertical profile is optically active and alters the upwelling radiation through absorption-emission (over ocean) and scattering (over land) processes. In general, over ocean, emission of liquid water is captured in microwave frequencies smaller than 21 GHz, while high intense rainfall and ice crystals alter high frequency features (> 60 GHz) with notable polarization and some scattering effects. On the other hand, over land, rain drops emission is severely masked by strong warm background radiation and rainfall spectral signatures are mainly determined by scattering of the upwelling beams, especially by frozen hydrometeors. Over land, intuitively speaking, the upwelling background radiation is split into an infinite number of incoherent beams due to the interaction with the precipitation profile. These beams propagate in different directions, thus giving rise to lower signal-to-noise ratio and less crisp raining signatures on the observed radiative fluxes, compared to the emission of rain drops over ocean. In addition, we need to emphasize that the background radiation over land depends on the space-time variability of the surface emissivity, which adds to the complexity of raining spectral patterns. Consequently, two families of microwave retrieval approaches have emerged. The land-based retrieval algorithms in the scattering regime are mainly empirical while ocean-based approaches are more physically-driven. By an empirical approach, we refer to those algorithms that are only data-driven [14, 15, 16, 17, 18, 19], while physically-based approaches are those in which we also rely on the underlying physical laws of the rainfall radiative transfer equations [20, 21, 22, 23, 15, 24]. Data driven retrieval approaches extend from thersholding and piece-wise regressions [15] to Bayesian techniques [25, 26, 27] and advanced low-dimensional approximation methods using principal component analysis [18, 19]. Note that a large body of the algorithms in this class rely on supervised learning from a priori collected libraries of coincidental observations of the weather precipitation radar (PR) and microwave imager (TMI) aboard the TRMM satellite, first pursued in [28]. On the other hand, physically-based methods typically follow two distinct strategies. The first family of physically-based algorithms [20, 15] simplify the basic radiative transfer equation for atmospheric constituents under the axially symmetric scattering and Rayleigh-Jeans approximation. Given the observed spectral radiative fluxes with minimal scattering effect, the simplified transfer equations allow us to obtain atmospheric absorptivity, rainfall dropsize distribution, and thus the rainfall intensity profile. In this approach, the atmosphere is divided into sufficiently thin optical layers and simplified assumptions are made about the dropsize distribution, freezing level, relative humidity, cloud water droplets, ice crystals, and temperature lapse rate. The second class of methodologies, suggested by [22], are known as the Bayesian retrieval approaches [29]. A key feature of this class is that the causal relationships between the precipitation profiles and upwelling spectral radiance are modeled using a combination of cloud resolving models and radiative transfer equations. Sophisticated numerical cloud resolving models (e.g., Goddard Cumulus Ensemble Model) are being used to reproduce a large collection of raining and non-raining cloud structures with distinct hydrometeor profiles. Then, for all of these profiles, a radiative transfer model is employed to obtain their spectral radiances at the top of the atmosphere. Finally, this a priori database is utilized to retrieve rainfall profiles of any given spectral radiance observation using a Bayesian inversion scheme. This approach has been the corner stone of the Goddard Profiling algorithm [22, 23, 24] used to produce the TRMM operational products. In this paper, motivated by the continuous quest for robust and accurate rainfall retrieval, especially in the view of the recently launched Global Precipitation Measurement (GPM) mission, we introduce a new algorithm that relies on a sequence of detection-estimation approaches for the microwave rainfall retrieval problem. The detection part makes use of a simple k-nearest neighborhood classification rule, while the estimation part relies on the modern developments in manifold learning and regularized estimation. The core part of the estimation approach relies on a geometrical notion, which assumes that similar raining microwave spectral signatures and their corresponding rainfall intensity values lie on or live near to some joint manifolds with analogous geometrical structure. Thereby, we use axioms of Euclidean

2

space to linearly estimate rainfall from a collection of spectral signatures and their corresponding rainfall profiles in an a priori organized data set, the so-called dictionaries. Specifically, this algorithm makes use of two joint dictionaries, the so-called spectral and rainfall dictionaries. These dictionaries contain basis elements or atoms of spectral responses for different rainfall profiles. In summary, the main theoretical advantages over previously introduced algorithms are: 1) The detection part provides notable accuracy in recovery of raining areas or storm support sets, especially over land and coastal zones; 2) The core estimation part makes use of a modern constrained Bayesian estimator, giving rise to sufficiently stable solutions with reduced error and improved recovery of rainfall extremes; 3) The algorithm can adaptively find potential low-dimensional representations and does not require any global low-dimensionality assumption or subspace projection to handle redundancies in observed rainfall spectral radiances; (4) The algorithm is flexible enough to use an a priori rainfall data set obtained either empirically or via physically-based modeling. Section 2 is devoted to gaining insight into the rainfall spectral separability and patterns in microwave wavelengths by learning from coincidental observations of the PR and TMI sensors. Section 3 explains the algorithmic details. Retrieval results from the proposed algorithm are presented in Section 4 using the TRMM data and compared with the currently operational 2A12 (version 7) retrieval product. Conclusions are drawn and future lines of research are pointed out in Section 5.

2

Rainfall spectral patterns

Before we embark upon the detailed algorithmic explanations, we provide some insight into the data structure of raining and non-raining microwave spectral patterns, essential to the development of the presented retrieval algorithm in this paper. Specifically, using the TRMM data, we study separability of raining spectral signatures from the non-raining background radiation and study their correspondence with the intensity of surface rainfall over ocean and land. The TRMM precipitation radar (PR) is a ku band radar that operates in a single polarization mode at frequency 13.8 GHz. Currently, the PR provides rainfall reflectivity images at grid size 4-to-5 km over a swath width of 247 km. This sensor nominally samples the first 15 km of the troposphere at every 250 m at nadir. On the other hand, TMI is a double frequency radiometer that operates on central frequencies 10.65, 19.35, 21.3, 37.0, and 85.5 GHz. All of the channels are horizontally and vertically polarized except the vertical water vapor 21.3 GHz. Currently, TMI provides a datacube of spectral brightness temperatures over a swath width of 878 km with nominal spatial resolution greater than 5.1 km. A thorough exposition of the TRMM sensor packages can be found in [30]. By design, the TMI and PR sensors provide overlapping observations over the inner swath; however, these observations are not registered at the same grid coordinates and spatial resolutions. TMI is a conical scanner, for which the cross-track resolution is finer than the down-track resolution while its resolution heavily depends on the antenna angle with respect to the nadir axis. On the other hand, PR is a cross-track scanner and provides more regularly spaced observations than the TMI. Here, we confine our data sources to version 7 of the orbital PR-2A25 and TMI-1B11 products. Notice that 2A25 is a level-II product derived from the radar profiling algorithm [31], while 1B11 is a level-I product which uses almost the raw observations of the spectral brightness temperatures. In 2A25, at each orbital scan, the near surface rainfall estimates and their reflectivity values are registered onto an almost regular grid in R49×9250 . In words, each 2A25 orbital product of the near surface rainfall is an image with 49 rows and 9250 columns, registered onto the latitude and longitude geographic coordinate system. On the other hand, the 1B11 product is registered onto a datacube of spectral brightness temperatures, registered onto a graticule in R104×9250×7 for frequency channels ≤ 37 GHz, and in R208×9250×2 for 85.8 GHz frequency channels. As is evident, to obtain coincidental PR and TMI observations, it is desirable to uniformly register all layers of observations onto a single graticule. To this end, we adopt the simplest registration technique and use the nearest neighborhood interpolation to map the TMI datacube of spectra temperatures onto the PR grids. Accordingly, throughout this paper we use a large collection of coincidental TMI and PR data, hereafter called rainfall database, which are collected and processed over the TRMM inner swath for all orbital tracks in the calendar years 2002, 2005, 2008, 2011, and 2012. Given this rainfall database, Fig. 1 shows the grouped scatter plots of raining versus non-raining spectral pairs over ocean (top panel) and land (bottom panel). These plots demonstrate grouped projections of more than 50,000 randomly sampled spectral brightness temperatures onto the Euclidean planes spanned by the TMI channel pairs. Over ocean, for the low-frequency channels (ν ≤ 37 GHz), vertical and horizontal polarizations of spectral brightness temperatures are highly correlated and live close to linear subspaces. Typically, due to the domination of the emissionabsorption regime, it can be seen that the raining spectral pairs are warmer than those of the non-raining pairs and are concentrated at the upper right corner of the scatter plots. Therefore, we can state that the polarization effect only creates a shift between the mean of raining and non-raining pairs in lower-frequency channels. On the contrary, raining signatures in channel 85 GHz are colder than those of non-raining background, mainly due to the dominant ice-scattering effect. In this band, we see that polarization creates nearly distinct linear subspaces with two different principle angles. These two linear subspaces are intersecting over their upper bounds and can be well separated with a linear decision

3

boundary. In other words, polarization in these channels not only creates a minor shift but also significantly alters the angular spread of the raining signatures versus the non-raining background radiation. In general, we can state that more distant channels provide better separability; however, the decision surfaces that separate raining and non-raining pairs become more nonlinear. Over land, it is seen that the spectral patterns of raining and non-raining pairs are more irregular than those over ocean. In particular, we see that in each low-frequency channel, vertical, and horizontal polarizations live far from an ideal linear subspace. For frequencies smaller than the water vapor absorption band (21 GHz), the majority of raining pairs is warmer than the non-raining background radiations; however, those pairs are distributed over locally segregated clusters with pronounced overlapping regions with the non-raining pairs. Due to the domination of the scattering regime over land, it can be seen that the high-frequency channels 85 GHz play a key role in rainfall spectral separability. Specifically, we can see that although raining and non-raining pairs are weakly separated over low-frequency channels, their pairs in the 85 GHz channel provide strong spectral separability. Fig. 2 shows the conditional expectation of randomly sampled spectral brightness temperatures b ∈ Rnc , given different ranges of rainfall intensity values, where nc = 9 denotes the number of spectral channels. In other words, each column shows E [b| rl ≤ r < ru ], where rl and ru determine the lower and upper bounds of the chosen rainfall intensity intervals. In the computation of conditional expectations, we assumed that in each interval the spectral vectors are independent and drawn from a uniform density. Note that, in Fig. 2, the rainfall intervals on the x-axis are logarithmically spaced between 0.2 to 200 mm/hr. Conditioning our knowledge to the accuracy of the TRMM sensors and radar profiling algorithm, three key pieces of information can be extracted from this Figure regarding to: 1) approximate probability distribution of the rainfall intensity values in a global scale over 38◦ N-S; 2) patterns of spectral response to rainfall intensity, and 3) relative sensitivity of each channel in response to the rainfall variability. Histograms confirm that the rainfall distributions lean to the left side of the mean and are positively skewed. The mean detected rainfall intensity over ocean (3.05 mm/hr) is approximately equal to the mean over land (3.1 mm/hr), while the mode over land (0.9 mm/hr) is slightly smaller than that over ocean (1.1 mm/hr), giving rise to a larger kurtosis and heavier tail of rainfall distribution. Over ocean, due to the domination of the absorption-radiation regime, it can be seen that the low-frequency channels (ν = 10 GHz) are getting monotonically warmer for larger rainfall intensities. However, this monotonic response is often limited to some saturation thresholds. For example, we see that beyond rainfall intensity ∼35 mm/hr, spectral response in the channel 19 GHz becomes saturated and flat. This counter intuitive behavior is known to be due to the marked backscattering of dense rain-filled atmospheric medium in heavy precipitation events [20]. On the other hand, highfrequency channels are getting monotonically cooler in response to higher rainfall intensities due to the cold ice scattering regime. This response seems more apparent for high-intense rainfall intensities greater than 5 ∼ 20 mm/hr, while heavy rainfall backscattering also contaminates the decreasing pattern of these channels over very high intense rainfalls roughly greater than 100 mm/hr. Over land, most of the low-frequency channels (ν ≤ 21 GHz) are saturated by the background radiation and a specific temperature pattern is not very apparent. However, high-frequency bands (≥ 37 GHz) are more responsive to the underlying rainfall intensities and get monotonically cooler as rainfall intensity increases. The stem plots of the coefficients of variation in Fig. 2 reveal the relative importance of different frequencies and polarizations in response to the underlying rainfall intensities. Over ocean, this plot shows that almost all frequencies are sufficiently responsive to the underlying surface rainfall variability while the vertical polarization in frequencies of 21 and 37 GHz exhibit minimal variability. Surprisingly, it is seen that in almost all of the low-frequency channels vertical polarizations tend to be slightly less responsive to the rainfall variability than those of horizontal polarizations (Fig. 2a). This phenomenon is suspected to be related to rainfall microphysics and geometry of raindrops. Raindrops are not spherical due to the air drag and get flattened on their base as they fall to the ground. Due to the seminal work by [32], it is well understood that this asymmetric structure is highly influenced by rainfall dropsize and thus the average rain rates. We speculate that the correlation of the surface rainfall intensity with the raindrops’ horizontal dimension is stronger than that of the vertical dimension and thus leads to a more reactive spectral responses in horizontal polarizations. Over land, almost all of the low-frequency channels below the water vapor channel of 21 GHz show very low coefficients of variation. Only frequency channels ≥ 37 GHz exhibit relatively large variations without noticeable polarization effect. We will see later that these coefficients of variations will be used to properly weight the relative importance of each channel in the proposed rainfall retrieval algorithm. To attain deeper knowledge of the correspondence between the raining spectral brightness temperatures and their M surface rainfall intensities, we independently collected two learning sets of the form L = {(bi , ri )}i=1 over ocean and 6 land. Each set contains a large number M u 10 of coincidental TMI-1B11 spectral brightness temperatures b ∈ R9 and their corresponding PR-2A25 surface rainfall r ∈ R observations. A simple Euclidean nearest neighborhood search reveals that the spectral temperatures over ocean and land are not uniquely related to the estimated surface rainfall intensities. Indeed, we found several exactly similar pixelcubes of spectral brightness temperatures with completely different surface rainfall intensities. This non-uniqueness may be related to multiple reasons including: inadequate

4

Figure 1: Matrix scatter plots of spectral brightness temperatures in Kelvin, grouped for raining signatures and non-raining background temperatures over ocean (top panel) and land (bottom panel). In the scatter plots light-blue triangles (4) denote the raining and dark-blue and -brown circles (◦) show the non-raining pairs over ocean and land, respectively. The data are obtained from 50,000 randomly drawn samples from coincidental observations of the TMI-1B11 and PR-2A25 products of the TRMM satellite. The panels on the right display a close up of four arbitrary chosen channel pairs.

5

Figure 2: Mean spectral brightness temperatures for different intervals of rainfall intensity values over ocean (left panel) and land (right panel). The graphs are inferred from coincidental pairs of the TMI-1B11 and PR -2A25 products obtained from 1000 randomly chosen orbits from our rainfall database. Each column of the middle images demonstrates the average values of spectral brightness temperatures for a selected rainfall intensity interval. The top histograms approximate probability distribution of rainfall intensities, while the stem plots demonstrate the coefficients of variation for each spectral band in response to different rainfall intensity values.

spectral bands, observation noise, sensor’s spatial resolutions, and 2A25 algorithmic limitations. Nevertheless, in the lack of uniqueness, a basic question arises: How can we obtain stable and accurate estimates of surface rainfall using neighboring spectral brightness temperatures in a properly collected learning set? Specifically, for a given pixelcube of K brightness temperature y ∈ R9 , if we find the set of its k-nearest neighbors Lk (y) = {(bk , rk )}k=1 ⊂ L in Euclidean K distance; how can we use the corresponding rainfall values {rk (y)}k=1 to obtain an estimate x ˆ ∈ R of the true surface rainfall intensity x ∈ R? The top panels from left to right in Fig. 3 demonstrate two arbitrary vectors of raining spectral brightness temperK=50 atures y (black dashed lines) together with their fifty nearest spectral neighbors {bk (y)}k=1 (gray solid lines) over ocean and land, respectively. On the other hand, the bottom panels show the corresponding surface rainfall values x of K=50 the chosen y’s and the rainfall probability histograms of the neighboring rainfall values {rk (y)}k=1 . Surprisingly, it turns out that all of the fifty nearest neighboring spectral brightness temperatures were raining except for only one of them over land. This observation implies that the supervised nearest neighborhood search, using coincidental PR and TMI data, might be a very powerful approach for the rain/no-rain classification problem. Fig. 3 clearly illustrates that the neighboring spectral temperatures are highly correlated and the retrieval problem maybe subject to more uncertainty over land than ocean due to a larger spread of spectral temperatures and their corresponding surface rainfalls. Furthermore, it can be seen that the first nearest neighbor in spectral space (1-nnT) does not necessary relate to the K=50 nearest neighbor (1-nnR) in the rainfall space. However, in both cases, {rk (y)}k=1 are distributed around x and bound it asymmetrically. As expected, these bounds, both in the spectral and rainfall spaces, are tighter over ocean than over K land. Therefore, for each sampled y, it can be naturally concluded that a properly chosen statistic of {rk (y)}k=1 in the following form: x ˆ=

K X

ck rk (y)

(1)

k=1

may be adopted as an estimator of x, where ck denotes some optimal representation coefficients.

3 3.1

Shrunken locally linear embedding for retrieval of precipitation Precipitation retrieval: As a discrete inverse problem

Physically-based rainfall retrieval in microwave bands is basically a nonlinear inverse problem, where its solution shall be constrained to the underlying laws of atmospheric thermal radiative transfer in a weak or strong sense. In a nonscattering thermal medium at local thermodynamic equilibrium, the classic form of the radiative transfer equation is a first-order ordinary differential equation with respect to the spectral intensities. In microwave bands, using RayleighJeans approximation, transfer of radiative fluxes in the ray path Ω from a surface at s0 to a detector at s1 can be explained in terms of the spectral brightness temperature Tνb at frequency ν in the following reduced form: ˆ s1 0 (2) Tνb (s1 , Ω) = Tνb (s0 , Ω) e−τν (s0 ) + Tνb (s0 , Ω) e−τν (s ) ds0 s0

6

Figure 3: Top panels: Two sampled raining vectors of spectral brightness temperatures (· · ∗ · ·) obtained from the TRMM microwave imager (TMI-1B11) over ocean (left) and land (right), respectively. The gray lines are 50-nearest spectral neighbors in an Euclidean sense, obtained from an independent learning set. Bottom panels: Surface rainfall probability histograms of the spectral neighbors, shown in the top panels, obtained from PR-2A25. In all plots the red squares () and the blue diamonds (−  −) show the 1-nearest neighbor in the spectral (1-nnT) and rainfall (1-nnR) spaces.

´s where τν (s) = 0 αν (s0 ) ds0 denotes the optical depth and αν (s) is the absorption coefficient [33]. As is evident, in this form of the radiative transfer equation, the flow of energy in terms of the brightness temperature is attenuated or smoothed out along its traveling path by the kernel Kν (s) = e−τν (s) , which characterizes the constituents of the traveling medium. Given that the first term in the right hand side of (2) is constant or vanishes in an optically thick medium at the origin, the above expression simply resembles the Fredholm integral equation of the first kind [34]. In a classic interpretation of this integral equation, given the medium absorptivity along the ray path and the brightness temperatures at s0 and s1 , the goal is to obtain the entire profile of the spectral brightness temperature along the path. However, there is a subtle difference between the classic treatment of the above integral equation and the problem of rainfall retrieval. Indeed in the rainfall retrieval problem, we are interested to estimate the kernel function for obtaining the absorptivity of the atmospheric medium and thus the precipitation profile along the radiation path. Furthermore, while the scattering is not negligible, the radiative transfer equation is more complex than that of expression (2). For a specific ray path, the incoming contribution of elastic scattering fluxes may be partially organized as follows: ˆ s1 ¨ S(ν) = γν (s0 ) pν (Ω0 → Ω) Tνb (s0 , Ω0 ) dΩ0 ds0 (3) s0

Ω0

where, γν (s) is the spectral scattering coefficient, and pν (Ω0 → Ω) is the phase function that characterizes the probability distribution of directional fluxes being scattered into the ray path [20, 15, 35]. One can easily see that even forward computation of this integral equation is difficult as it requires knowledge of the scattered energy fluxes from all involved directions. Therefore, accounting for both absorption and scattering phenomena, the rainfall retrieval problem is intrinsically very nonlinear and goes far beyond the scope of classic linear integral equations and inverse problems. To recast the above problem in a standard form of a discrete inverse problem, let us assume that each pixel-cube of T the observed spectral brightness temperatures and their corresponding rainfall profiles are y = (y1 , y2 , . . . , ync ) and T x = (x1 , x2 , . . . , xnr ) , respectively, where nc and nr denote the number of spectral channels and vertical layers of the rainfall intensity profile. As a result, in a finite dimension, one may express the above radiative transfer model as the following nonlinear observation model: y = F (x) + v

(4)

where, F (·) : x → y is a functional representation of the radiative transfer equations that maps the rainfall intensity profiles onto the space of spectral brightness temperatures, and v ∈ Rnc represents the observation error with a finite 7

energy. Obviously, the goal is to obtain an estimate of the rainfall profile x, given spectral brightness temperatures y, the radiative transfer functional F (·), and a priori information about the error. The search for a stable closed form solution of the above inverse problem seems almost a hopeless quest at least for now, given the fact that F (·) is extremely nonlinear in the rainfall retrieval problem and often observations of radiative intensities are highly corrupted with noise.

3.2

Algorithm

Motivated by our observations in Section 2, to bridge the explained complexities in rainfall retrieval, our algorithm relies M on a learning set, denoted by L = {(bi , ri )}i=1 . This set is populated by a large number of coincidental brightness T T temperatures bi = [b1i , b2i , . . . , bnc i ] ∈ Rnc and their corresponding rainfall profiles ri = [r1i , r2i , . . . , rnr i ] ∈ Rnr . In our algorithm, each of these pairs are indeed elementary atoms to be used for reconstruction of rainfall fields from their observed spectral signatures. For notational convenience, let us stack these atoms according to a fixed order in two joint matrices B = [b1 | . . . |bM ] ∈ Rnc ×M and R = [r1 | . . . |rM ] ∈ Rnr ×M , the so-called spectral and rainfall dictionaries.. In the detection step, we simply use a nearest neighborhood classification rule. In particular, for a given spectral pixelcube
y ∈ Rnc and the dictionary pair (B, R), the algorithm finds two joint sub-dictionaries BS ∈ Rnc ×K , RS ∈ Rnr ×K , K which are generated by the K = |S| nearest neighboring spectral {bk }k=1 atoms and their corresponding rainfall atoms K {rk }k=1 . Assuming that the last row of the rainfall sub-dictionary RS contains the near surface rainfall intensity values, the algorithm simply makes use of a probabilistic voting to declare y as raining or non-raining. In other words, choosing K a probability threshold p, the algorithm labels y as a raining pixelcube, if more than a p K number of {rk }k=1 are raining at the surface. In the estimation step, we assume that the true rainfall profile x of the given spectral pixelcube y can be well approximated by the RS ’s atoms, through the following linear model: x = RS c + e

(5)

where c ∈ RK is a vector of representation coefficients that linearly combines atoms of the rainfall sub-dictionary and e ∈ Rnr denotes a zero mean error with finite energy. As a result, given an estimate of the representation coefficients ˆ, the conditional expectation of the rainfall profile x ˆ can be obtained as follows: c ˆ = E (x|ˆ ˆ. x c) = R S c

(6)

Obviously, estimation of the representation coefficients solely from equation (5) is ambiguous as both sides of the equation are unknown. However, in the previous section, we provided evidence that estimation of the representation coefficients can be constrained by the information content of the neighboring spectral atoms of the observed spectral pixelcube y (see equation 1). To this end, we assume that the neighboring rainfall profiles and their spectral signatures live close to two joint manifolds with analogous geometric structure and thus similar locally linear representation. Therefore, the algorithm assumes a spectral observation model with the same linear representation coefficients as follows: y = BS c + v

(7)

where v ∈ Rnc denotes a zero mean observation error with finite energy. As is evident, estimation of representation coefficients from (7) is no longer an ill-defined problem. To estimate the representation coefficients in this linear observation model, the Minimum Mean Squared Error (MMSE) estimator, constrained to the probability simplex, seems to be the first choice as follows: minimize c

2

ky − BS ck2

subject to c  0, 1T c = 1

(8)

2

where the `2 -norm kck2 = Σi c2i and c  0 implies element-wise non-negativity. Note that the non-negativity constraint is required to be physically consistent as the brightness temperatures in Kelvin are always positive. Furthermore, the sum to one constraint assures that the estimates are locally unbiased. More importantly, this equality constraint makes the solution invariant to rotations, rescalings, and translations of the neighboring spectral pixelcubes [36, 37]. However, problem (8) is likely to be severely ill-posed due to the observation noise, especially when the column dimension of BS is larger than that of spectral bands nc . To make the problem well-posed and sufficiently stable, we suggest the following regularization scheme minimize c

2

1/2

2

W (y − BS c) + λ1 kck1 + λ2 kck2 2

subject to c  0, 1T c = 1 8

(9)

Algorithm 1 Shrunken Locally Linear Embedding Algorithm for Retrieval of Precipitation (ShARP).  N T n

Input: A spectral datacube Y containing yi = [y1i , y2i , . . . , ync i ] ∈ R c i=1 pixelcubes of spectral temperatures, spectral B ∈ Rnc ×M and rainfall R ∈ Rnr ×M dictionaries, spectral weight matrix W ∈ Rnc ×nc , detection probability p, number of nearest neighborhoods K, and regularization parameters λ1 , λ2 . Output: Precipitation field X containing {xi ∈ Rnr }N i=1 corresponding pixels of rainfall intensity profiles. For i := 1 to N step 1 do

• Find sub-dictionaries BS ∈ Rnc ×K and RS ∈ Rnr ×K , where S is the set of column indices of B which contain the K-nearest neighborhoods of yi .

• Standardize yi and atoms of BS , such that

Pnc j

yji = 0,

Pnc j

bjk = 0, and

Pnc j

b2jk = 1, for k = 1, . . . , K.

• Let RS (end, :) denotes the last row of RS containing neighboring surface rainfall. • If |supp (RS (end, :))| ≥np K , o

2

ˆi = argminci 0, 1T ci =1 W1/2 (yi − BS ci ) 2 + λ1 kci k1 + λ2 kci k22 c ˆ i = RS c ˆi x else ˆi = 0 x End If End For

where the `1 -norm is kck1 = Σi |ci |, λ1 , λ2 are non-negative parameters, and the positive definite W  0 in Rnc ×nc ˆ as the solution of the above problem, we can determines the relative weights of each channel. Obviously, obtaining c ˆ = RS c ˆ. retrieve the rainfall using expression (6) as x Note that problem (9) is a non-smooth convex problem. It is non-smooth as the `1 -norm is not differentiable at the origin. Convexity arises as it uses a conic combination of two well-known convex penalty functions to regularize a classic weighted least-squares problem over a convex set. These two regularization functions have been widely used to properly narrow down the solution of ill-posed inverse problems. In under-determined systems of equations, the `1 -norm penalty has proven to be an effective regularization for obtaining sparse solutions. In other words, it turns out that this regularization promotes sparsity as it uses a minimal number of atoms in the BS , while retains maximum amount of information [38, 39, 40, 41]. On the other hand, the `2 -norm penalty is the most widely used regularization approach to stabilize the solutions of dense ill-posed inverse problems as it incorporates all atoms in BS [42, 43]. Confining the regularization in (9) solely to the `1 -norm (λ2 = 0) is restrictive for rainfall retrieval because of two main reasons. First, the the number of selected columns of BS , or say non-zero elements of the solution, is bounded in this case by the number of available spectral bands. Second, as shown in Fig. 3, the spectral atoms in sub-dictionary BS are typically highly correlated and are clustered in groups. In this condition, the `1 -norm regularization typically fails to take into account the contribution of clustered atoms. On the other hand, the `2 penalty function tends to incorporate all of the spectral atoms for obtaining the solution. Thereby, solely relying on this penalty function (λ1 = 0) for regularization of problem (8) may lead to selection of irrelevant atoms and overly smooth rainfall retrieval. However, the proposed mixed penalty removes the limitation on the number of selected atoms by the `1 -norm, stabilizes the problem regularization path, and encourages grouping effects by shrinking the clusters of correlated atoms together and averaging their representation coefficients [44]. Throughout this paper, we consider a convex combination of regularization penalty functions by assuming λ2 = λα and λ1 = λ (1 − α) for all α ∈ [0, 1]. As we use the concept of locally linear embedding in [36] together with a mixed shrinkage estimation, we call our retrieval technique the Shrunken Locally Linear Embedding Algorithm for Retrieval of Precipitation (ShARP), which is summarized in Algorithm 1. Note that, given the induced non-negativity constraint in problem (9) allows us to solve it via a constrained quadratic programming (QP) as follows: minimize c



T T T cT BT c S WBS + λ2 I c + λ1 1 − BS W y

subject to c  0, 1T c = 1 T

where 1 = [1, . . . , 1] ∈ RK .

9

(10)

Figure 4: Different earth surface classes used in the current version of the ShARP algorithm, namely inland water body (In); coastal zone (c); land (l); and ocean (o). The classification is adopted based on the available data in the version 7 of the PR-1C21 product, which are mapped onto a regular graticule with 0.05◦ × 0.05◦ grids.

4

Retrieval experiments using TRMM data

Before presenting the results of the ShARP algorithm and delving into its quantitative comparison with version 7 of the GPROF operational product 2A12, we need to emphasize two important points: 1) GPROF uses an observationally consistent physically-based database over ocean [24] and thus in this sense is a stand-alone algorithm which does not firmly rely on any empirical relationships or learning strategies derived from the past observations; 2) GPROF retrieves the entire precipitation vertical profile and not just surface rainfall. On the other hand, in the present implementation of the ShARP algorithm, we only confine our consideration to empirical dictionaries collected from coincidental observations of the TRMM-PR and TMI sensors and only retrieve surface rainfall. Note that the core part of the ShARP algorithm is sufficiently flexible to accommodate physically-based dictionaries that can also encapsulate the precipitation vertical profiles as well.

4.1

Algorithm setup

In this study, for the TRMM orbital retrievals, we exploit four different earth surface classes, namely: ocean, land, coast, and inland-water (Fig. 4). In other words, we collect four dictionaries over each surface class and use them in Algorithm 1 depending on the geolocation of the given spectral pixelcubes. This stratification of the surface is obtained from standard surface classification data in the PR-1C21 product (version 7) at ∼ 5 × 5 km grid size. In this classification, the coastal areas are referred to those locations on the globe, where the presence of water is not permanent due to the seasonal variations or tidal effects. To construct spectral and rainfall dictionaries, we randomly sampled 750 orbits from our rainfall database. In these sampled orbits, more than 25 × 106 pairs of non-raining background and raining spectral signatures were used to construct the required dictionaries. 4.1.1

Detection step

Rainfall is a discontinuous process in time and space and thus a storm event might be considered as a composition of rain/no-rain binary patterns and rainfall distribution within raining areas. Therefore, in developing rainfall retrieval techniques we naturally have a choice to either first detect the storm support (raining areas) and then estimate the rainfall intensities or just use an estimation scheme that automatically recovers the storm support. In general, rainfall retrieval with a sequential detection estimation scheme is advantageous in the sense that it allows us to separate the estimation and detection error, while confining the computational expense of estimation only to the detected raining areas. Rain/no-rain classification from microwave observations and its induced error on the quality of rainfall retrieval has been addressed in many studies [45, 46, 47, 48, 49, 50], and reported as a known problem, which is not easy to mitigate, especially over land [24]. Basically, in the majority of these methods over land, the scattering of channel 85 GHz plays a very crucial role. Typically, a scattering index (SI) is computed by which we can separate the background non-raining radiances from the raining signatures. For example, in the early version of the GPROF, the scattering index is considered 85v ◦ to be SI = T22v b − Tb , where the TRMM spectral pixel-cubes with SI < 8 K are classified as non-raining [23]. The effectiveness and relevance of this simple classification rule can be easily understood by looking into the grouped scatter plot of T22v versus T85v over land in Fig. 1. To improve the standard discriminant analysis of the GPROF algorithm b b over land, [50] proposed a variable thresholding approach and reported 63% of chance for rainfall binary detection, considering the PR-2A25 as the reference datum.

10

Figure 5: Rainfall Receiver Operating Characteristic (ROC) curve over ocean (left panel) and land (right panel) for different probability of detection p ∈ [0, 1] and number of nearest neighborhoods K ∈ {5, 10, 20, 40, 100} in the ShARP algorithm. The blue circles show the probability of hit and false alarm for the standard TRMM-2A12 (version 7) product and the red dash-dotted lines show the 0.95 probability of hit as a datum.

Fig. 5 shows the estimated probability of hit (PrH ) versus probability of false alarm (PrF ) for the classification scheme of our algorithm as the model parameters are varied. Here, for brevity, we only present the results over ocean and land. In signal detection theory, this graph is often called Receiver Operating Characteristic (ROC) curve. As is evident, the best classification algorithm yields a point at the upper left corner with PrH = 1 and PrF = 0. To derive the rainfall ROC curves, we considered PR-2A25 as the reference observation and computed the probabilities of interest by applying the classification scheme of our algorithm to more than 3 × 105 randomly chosen spectral pixelcubes from our rainfall database. Note that, these pixelcubes are selected randomly from our rainfall database and have not been used in the construction of the retrieval dictionaries. In Fig. 5, we can see that the classification rule is not very sensitive to the number of chosen nearest neighborhoods as all of the curves are nearly collapsing onto each other. Over ocean, 2A12 (version 7) almost reaches a probability of detection close to 0.95 with a relatively large probability of false alarm greater than 0.20. These statistics imply that the 2A12 algorithm overestimates the raining area compared to the reference PR algorithm over ocean but the chance of missing a raining pixel is less than 0.05. On the other hand, over land, PrH = 0.62 in 2A12 product and PrF u 0. In other words, 2A12 misses almost 38% of the raining events compared to the radar algorithm while the chance for false alarm is very slim. The results of the ShARP for K = 20 and the majority vote rule, that is p = 0.5, are presented in Table 1. This table explains that over ocean, our algorithm detects raining pixels in 96% of the cases while the false alarm rate does not exceed 8%. Over land, as expected, the detection probability is slightly degraded to 0.90 while the probability of false detection reaches to 0.06 in our algorithm. We need to emphasize that the reference product 2A25 is not obviously free of error and all of the reported statistics are limited to the accuracy of this product. In particular, there is some evidence from the CloudSat satellite suggesting that the PR might underestimate the extent of light rain over ocean [51, 24]. Observation (2A25) Ocean

Detection (ShARP)

Land

rain

no-rain

rain

no-rain

rain

0.96

0.08

0.90

0.06

no-rain

0.04

0.92

0.1

0.94

Table 1: Probability of hit and false alarm for twenty nearest neighbors K = 20 and probability threshold of p = 0.5.

4.1.2

Estimation step

After finding the storm support, our algorithm moves toward estimation of the rainfall intensity values. Recall that, we use a positive definite weight matrix W in problem (9) that determines the relative importance of each channel over different surface classes. For all of the following rainfall estimation experiments, to design this weight matrix, we use the normalized coefficients of variation for each channel as reported in Fig. 2. The coefficients of variation are computed 11

Figure 6: Low- and high-frequency spectral brightness temperatures—except 21v GHz—of two storm events over ocean and coastal areas, captured by the TRMM orbit No. 03357 in 06/28/1998.

for all chosen earth surface classes, which are not shown for the coastal and inland-water bodies for brevity in Fig. 2. In particular, the relative weight
of the ith channel for a specific surface class is obtained by normalizing its coefficient of variation as wi = civ /max civ , where i = 1, . . . , 9. Obtaining each wi , as reported in Table 2, the weight matrix is i

assigned to be W = diag (wi ). Relative weights Surface Classes

Channels 10v

10h

19v

19h

21v

37h

37v

85v

85h

Ocean

0.39

1.00

0.35

0.76

0.19

0.14

0.40

0.49

0.45

Land

0.07

0.17

0.09

0.09

0.12

0.37

0.35

1.00

0.97

Coast

0.19

0.42

0.13

0.36

0.07

0.26

0.20

1.00

0.95

Inland-water

0.33

0.66

0.36

0.84

0.20

0.26

0.59

1.00

0.88

Table 2: The diagonal elements of the weight matrix W ∈ Rnc ×nc used in the ShARP algorithm for the chosen earth surface classes, where for the TMI data nc = 9.

To obtain the optimal representation coefficients in problem (9), we use a primal-dual interior-point method [52, chap. 11]. Basically, in this class of convex optimization methods, the inequity constrained quadratic problem (10) is reformulated into an equality constrained problem to which iterative Newton’s method can be applied. Specifically, we employed Linear-programing Interior Point SOLver (lIPSOL) [53] which is a variant of the algorithm by [54]. In this optimization sub-algorithm the maximum number of iterations in Newton’s steps is set to 200, the termination tolerance on the function value and magnitude of relative changes in the optimization variable are both set to 1e − 8. We set the algorithm regularization parameters to be λ = 0.001 and α = 0.1, which appears to work well for a wide range of rainfall retrieval experiments. This setting permits the algorithm to perform a full orbital rainfall retrieval in the order of 10 to 15 minutes on a contemporary desktop machine. Fig. 6 shows low- and high-frequency spectral brightness temperatures of two storm events captured by the TMI in 26/08/1998, over the north Atlantic Ocean and over the northern coastlines of the Gulf of Mexico. All spectral observations in this Figure are shown only over the TRMM inner swath for clearer comparison purposes. As is apparent, 12

Figure 7: Comparison of the reference 2A25 (top panels) with 2A12 (middle panels) and ShARP rainfall retrieval (bottom panels), given the spectral brightness temperatures shown in Fig. 6. Left panels show the rainfall retrievals while the right panels demonstrate the shape of the retrieved storm support.

the low-frequency channels (< 21 GHz) only respond to the bulk of surface rainfall over ocean (Fig. 6, top panels) while those with higher frequencies provide more detailed information of the storm structure, especially over coastal areas (Fig. 6, bottom panels). We can see that, frequency channels 10 GHz provide blurry signatures over ocean and are fairly non-informative with respect to the storm happening over the coastal areas. These low-frequency channels are apparently very responsive to the change of the background radiation over the ocean-land interface. In higher frequencies, we can see that the raining spectral signatures become more informative with noticeable variability. The storm boundaries are clearly delineated in brightness temperature fields at central frequencies 19 and 37 GHz. It is also seen that frequency channels 85 GHz do not appear to be very sensitive to the background radiation over coast lines and provide vital but partial information about the extent of the storm over coastal areas. In other words, near and over coastal areas multispectral brightness temperatures seem to be very crucial as we see that the storm exhibits a pronounced raining signature over the coast in 85 GHz channel, while its extension over ocean is well captured by the frequency channels 19 and 37 GHz. Fig. 7 compares 2A25 (top row) with 2A12 (middle row), and ShARP rainfall retrievals (bottom row) given the spectral brightness temperatures shown in Fig. 6. We can see that the shape of the recovered storm support in ShARP conforms well with the reference 2A25 and is smaller than that of 2A12. The extended raining areas in the 2A12, particularly over ocean, are typically filled with very low rainfall intensity values less than the PR accuracy (< 0.1 mm/hr) [47]. These extended low rainfall intensity values are the main reason for the 2A12 large probability of false alarm, reported in Table 1. A closer visual inspection over ocean reveals that the two retrieval algorithms are comparable over sufficiently high-intensity raining areas. However, in general, ShARP provides a rougher rainfall field with sharper boundaries and richer variability. Compared to the PR as the reference field, it seems that ShARP slightly overestimated the central parts of the storm over 70-75W. However, it better recovered extreme rainfall over the tiny band of cold ice scattering located on the western side of the main storm body over ocean. For the coastal storm, it seems that ShARP leads to improved retrieval both in terms of the recovered storm morphology and its rainfall variability.

4.2

Instantaneous experiments

Fig.s 8, 9, and 10 demonstrate the results of multiple instantaneous retrieval experiments using the TMI spectral observations over ocean, land, and coastal areas, respectively. Most of the experiments are conducted using important extreme events recorded in the TRMM extreme event archives (http://trmm.gsfc.nasa.gov/publications_dir/ extreme_events.html). Over ocean, we focused on rainfall retrieval using the TMI spectral datacubes of the hurricane Helene (09/15/2006), hurricane Danielle (08/29/2010), tropical storm Giri (10/22/2010), tropical storm Haruna (02/20/2013), and super

13

Figure 8: From left-to-right: PR-2A25, TMI-2A12, and ShARP retrieval products. Top-to-bottom panels: tropical storm Helene (orbit No. 50338) at 14:34 UTC; hurricane Danielle (orbit No. 72840) at 09:48 UTC; tropical storm Haruna (orbit No. 86960) at 07:17 UTC; tropical cyclone Giri (orbit No. 73676) at 15:34 UTC; super typhoon Usagi (orbit No. 90277) at 02:09 UTC.

typhoon Usagi (09/21/2013); see Fig. 8. The hurricane Helene formed on the 12th of September 2006 over south of the Cape Verde Islands and turned into the latest and strongest hurricane of the 2006 Atlantic hurricane season. Due to its long life span TRMM provided several unique images of its life cycle. Helene never made landfall while maximum instantaneous rainfall of about 120 mm/hr was recorded by the TRMM-PR overpasses and its maximum sustained wind speed reached up to 190 km/h. Hurricane Danielle is also a Cape Verde-type hurricane, which formed in 21th of August 2010 over eastern Atlantic Ocean and later intensified into a Category 4 storm on the 27th of August. In the image shown in Fig. 8, Danielle was a category 2 storm with one minute sustained wind speed of about 150 km/hr while its extreme rainfall rate reached to almost 100 mm/hr. The tropical cyclone Haruna was a fatal storm with severe social-economic disruptions and disease outbreak in southern Madagascar. Haruna developed in the Mozambique channel in February 18, 2013 with maximum sustained wind speed of 185 km/hr and peak rainfall of more than 110 mm/hr. Cyclone Giri was the most powerful tropical cyclone of 2010 and formed over the Bay of Bengal. Giri’s maximum sustained wind speed reached 250 km/hr and caused catastrophic damages in Myanmar. More than 157 direct fatalities and 360 million (USD) economic damages are reported for this storm. Super Typhoon Usagi was a category 5 super typhoon, which affected Taiwan, the Philippines and parts of China. Developed in 16th of September 2013, Usagi attained a maximum sustained wind speed of about 260 km/hr, caused tens of fatalities and 4.33 billion dollars in damages. Over land, we focused on a few thunderstorms and mesoscale convective systems. These systems include squall lines and high precipitation supercells, some of which are reported in the TRMM extreme event archives. These events include a local thunderstorm over Nigeria (06/28/1998); a squall line over Mali (08/29/2010), Africa; a summer season convective complex over northern Georgia and Alabama, U.S. (11/16/2011); a summer season convective system over northern Florida, U.S. (07/11/2012); a severe thunderstorm over the western boarder of Uruguay with Argentina (08/17/2012); and a spring season squall line containing tornadic activities over Georgia, U.S. (01/30/2013); see Fig. 9. 14

Over coastal areas, we studied typhoon Kai-tak over the Gulf of Tonkin, coastlines of Vietnam and southern China (08/17/2012); tropical storm Fernand over eastern coasts of Mexico (08/26/2013); hurricane Issac over Mississippi delta, U.S. (28, 29/08/2012); a convective system over south east coastlines of Brazil (09/21/2013); and tropical cyclone Alessia over northern coasts of Australia (11/27/2013). Typhoon Kai-tak with a maximum sustained wind speed of 130 km/hr was a deadly storm with up to 40 fatalities which affected the coasts of China, Vietnam, and Laos during the 2012 Pacific typhoon season. This typhoon triggered severe flash flooding and landsliding over northern Philippines and approximately dumped 400 millimeters of rain over the Pearl River Delta in the Guangdong province of China. Tropical Storm Fernand formed over the southern Bay of Campeche and made heavy landfall just to the north-northwest of Veracruz, Mexico and took the lives of 14 people. A maximum sustained wind speed of more than 100 km/hr and total rainfall amounts of 120 to 230 millimeters are reported by the U.S. National Hurricane Center over the states of Veracruz. Hurricane Issac was also a very destructive and deadly storm that hit U.S. state of Louisiana in 2008 and took 41 lives and caused billions of dollars in damages. Severe beach erosion and flash flooding are reported in the affected southern states and more than 400 millimeters of total rainfall amount is recorded in the state of Florida.

Figure 9: From left-to-right: PR-2A25, TMI-2A12, and ShARP retrieval products. Top-to-bottom panels: a summertime thunderstorm over Nigeria, Africa (orbit No. 03357) at 17:43 UTC; a thunderstorm over Mali, Africa (orbit No. 72841) at 10:30 UTC; a summer season convective complex over northern Georgia and Alabama, U.S. (orbit No. 79756) at 02:05 UTC; a convective system with thunderstorms over northern Florida, U.S. (orbit No. 83479) at 22:59 UTC; a sever convective system over Uruguay and Argentina (orbit No. 84050) at 14:20 UTC; and a spring season squall line of precipitation supercells and tornadoes over Georgia, U.S. ( orbit No. 86639) at 16:22 UTC .

15

In general, our experiments in Fig. 8, 9, and 10 demonstrate that the results of the ShARP retrieval conform well with the reference PR-2A25 and TMI-2A12 products. As previously noticed, we typically see that the 2A12 retrieves much larger raining areas over ocean compared to the ShARP and PR-2A25. The ShARP retrieved rainfall fields are typically rougher than those of 2A12 and prone to recover higher intensity rain-cells with sharper representation of storm morphology. For example, in rainfall retrieval of the tropical storm Helene (Fig. 8, first row), we see improvements in recovery of the outer and inner rainbands surrounding the center of the cyclone. In Hurricane Danielle (Fig. 8, second row), ShARP shows improved retrieval of the storm curvature and multiband rainfall structure while the shape of the storm support is well matched with the reference PR observation. In tropical storm Haruna (Fig. 8, fourth row) the eyewall is fairly recovered while ShARP shows some improved results in capturing the cyclonic tails of Giri (Fig. 8, fourth row) near to the south west shorelines of Myanmar. In rainfall retrieval of the super typhoon Usagi (Fig. 8, fifth row), we also see some refinements in recovery of rain bands circling the eye, especially over southern Taiwan where the storm hits the shore.

Figure 10: From left-to-right: PR-2A25, TMI-2A12, and ShARP retrieval products. Top-to-bottom panels: typhoon Kai-tak (orbit No. 84050) at 13:35 UTC; tropical Storm Fernand (orbit No. 89874) at 05:34 UTC; hurricane Isaac (orbit No. 84227) at 22:12 UTC and (orbit No. 84230) at 03:07 UTC; a convective system over south eastern coastlines of Brazil (orbit No. 90277) at 02:55 UTC; and tropical cyclone Alessia (orbit No. 91321) at 02:25 UTC.

Over land, 2A12 typically underestimates the size of the storm support sets compared to that of ShARP (Fig. 9, first row). As previously explained, the probability of false alarm in the 2A12 product is close to zero over land, meaning that the raining areas and storm extent are most likely to be underestimated in this product. Furthermore, the 2A12

16

0.35 2A25 ShARP 2A12

0.3

Prob.

0.25 0.2 0.15 0.1 0.05 0 −10 −8

−6

−4

−2

0

2

4

6

8

10

log (x) Figure 11: Kernel estimates of rainfall probability density functions over the TRMM inner swath (orbit No. 03357) in 06/28/1998 for the PR-2A25 (solid black), ShARP (blue dash-dot), and 2A12 (red dashed). For clarity the densities are shown for the logarithm of rainfall intensity values x [mm/hr].

retrievals are commonly much smoother over land than those over ocean. We can see that ShARP can obtain improved estimates of low rain rates, rainfall variability, and rainfall morphological patterns in convective and frontal systems. For example, in the retrieved thunderstorm near Mali (Fig. 9, second row) and tornadic events over Georgia (Fig. 9, bottom row), we see that the ordinary cells, which are spread behind the leading edge of the squall lines, have been well retrieved by ShARP. Nevertheless, compared to the reference precipitation radar, both retrieval methods are unable to fully recover small scale high-precipitation supercells and remain constrained to the representative error due to the coarse spatial resolution of the TMI. Due to the interference of land-ocean background radiations, rainfall retrieval over coastal zones is naturally the most complex retrieval with respect to the underlying surface classes. Visual inspections of the retrieved rainfall at the ocean-land interface show that ShARP performs well and its results are comparable with the operational 2A12 product. As the studied 2A12 product uses a physically constrained retrieval approach over ocean while it remains empirical over land, we observe some irregularities and discontinuities in the retrieved rainfall fields over the land-ocean interfaces. In the 2A12, the algorithm typically retrieves a large mass of very low rain rates over ocean while over land the lower bound of the retrieved rainfall intensity is much greater. On the other hand, as the current version of ShARP uses the same approach over all chosen surface classes, the retrieval results remain fairly coherent over coastal areas. Some improved results are observed in the recovery of low rain rates over land and it seems that this version of the ShARP algorithm is less sensitive to the change in the background surface radiation regime. For example, in Hurricane Isaac (Fig. 10, third and fourth rows), we see some improvements in the retrieval of rainbands and cyclonic morphology of the storm. In addition, small scale convective activities surrounding the major body of the storm are also well retrieved (Fig. 10, fifth and sixth rows). For the entire orbits of the shown instantaneous retrieval experiments, a wide range of quality metrics or retrieval ˆ and x skills are reported in Table 3. Let us assume that the retrieved rainfall and the reference fields are denoted by x ˆi ∼ pXˆ (ˆ as vector forms in RN , while their marginal probability distributions are x xi ) and xi ∼ pX (xi ), i = 1, . . . , N , respectively. In particular, to study the magnitude of error and correlation field with respect to the

of the retrieved √

ˆ chosen reference one, we used: (1) Root Mean Squared Error, RMSE = (x − x) / N ; (2) Mean Absolute Error, 2

ˆ ) /N k1 ; (3) Peak Retrieval to Error Ratio, PRER = 20 log10 (max (ˆ MAE = k(x − x xi ) /RMSE); and (4) classic Pearson correlation. To evaluate the proximity between the marginal distributions of retrieved rainfall and reference fields, we also used Kullback-Leibler (KL) Divergence KL =

N X

 log

i

pX (ˆ xi ) pXˆ (xi )

 pX (ˆ xi )

(11)

and Kolmogorov-Smirnov (KS) statistic KS = sup FX (z) − FXˆ (z) z

17

(12)

Product

ShARP

2A12

Exp.

Retrieval Skills RMSE

MAE

PRER

CORR

KS

KL

(a)

1.46

0.18

33.67

0.63

0.08

0.08

(b)

1.23

0.15

34.36

0.59

0.08

0.20

(c)

1.35

0.17

33.62

0.57

0.10

0.10

(a)

1.54

0.20

30.14

0.61

0.72

3.25

(b)

1.32

0.17

31.30

0.55

0.70

3.25

(c)

1.40

0.19

31.99

0.53

0.71

2.81

Table 3: Averaged retrieval skills for the entire orbits containing the experiments (Exp.) demonstrated in (a-Fig. 8),(b-Fig. 9), and (c-Fig. 10), respectively. Shown statistics are Root Mean Squared Error (RMSE) in mm/hr, Mean absolute Error (MAE) in mm/hr, Peak Rainfall to Error Ratio (PRER) in decibel of rainfall intensity, Pearson linear correlation (CORR), KolmogorovSmirnov (KS), and Kullback-Leibler Divergence (KL). In all of the below statistics the PR-2A25 product is considered as the reference observation.

where FX (z) = pX (x ≤ z) denotes the cumulative probability density function. Note that, the KL divergence is not a true metric as it is neither symmetric nor satisfies the triangle inequality; however, it provides a non-negative measure that explains proximity of two probability density functions. Furthermore, we report some location statistics including mean and quantiles of the retrieved and reference rainfall distributions for the instantaneous experiments in Table 4. In Table 3, for brevity, the statistics represent the average values for all orbital retrievals shown in Fig. 8, 9, and 10. It is seen that even though the improvements in RMSE, MAE, and correlation are about 8% to 10% on average, in terms of KL and KS measures, distributions of the ShARP rainfall retrievals seem to be much more closer to the reference PR than that of the 2A12 (see, also Fig. 11). Although, it is not surprising as we use PR data to reconstruct the rainfall dictionary in this version of our algorithm. We also see that PRER metric is increased more than 2.5 decibels, which implies improvements in the retrieval of rainfall extremes in the ShARP products. Table 4 compares the average of location statistics for all orbital tracks containing the retrieval experiments demonstrated in Fig. 8, 9, and 10. As is evident, ShARP retrieves more rain on average over the recovered raining support than 2A12 and the quantiles are in better agreements with the reference 2A25 values. Note that the differences in location statistics do not necessarily imply the presence of bias in 2A12. Because, typically 2A12 has a larger support set populated with many close to zero rainfall values than that of ShARP, which gives rise to smaller mean values. Indeed, we see later on that total retrieved rainfall in 2A12 may even exceeds the total 2A25 rainfall. To shed more light on this issue, Fig. 11 compares kernel density estimates of the retrieved rainfall probability density functions with 2A25 for the orbital track partially shown in Fig. 7. These estimated densities reveal a very important distinction between the ShARP and 2A12 retrieval algorithms. Compared to the 2A25 product, both retrieval algorithms produce a larger mass of low rain rates while they fall short in capturing the tail of the rainfall probability distribution or the rainfall instantaneous extremes. We can see that 2A12 tends to produce a large probability mass below the rainfall rates of 0.15 to 0.20 mm/hr, while the probability of rainfall extremes is markedly smaller than that of the 2A25 product. Although, this problem also manifests itself in the ShARP product, its extent seems to be less pronounced. This improvement may be partially due to the fact that the current version of the ShARP algorithm uses 2A25 observations to construct the rainfall dictionary.

Product

ShARP

2A12

2A25

Exp.

Location Statistics Mean

Q05

Q25

Q50

Q75

Q95

(a)

3.72

0.52

1.06

1.84

4.03

13.73

(b)

3.03

0.49

1.00

1.71

3.32

10.36

(c)

3.00

0.50

1.10

1.72

3.27

9.93

(a)

0.95

0.01

0.02

0.08

0.36

4.93

(b)

0.76

0.01

0.02

0.08

0.29

3.69

(c)

0.72

0.01

0.02

0.07

0.42

3.70

(a)

4.51

0.38

0.87

1.85

4.71

17.41

(b)

3.65

0.35

0.78

1.59

3.74

13.85

(c)

3.66

0.35

0.78

1.59

3.66

13.88

Table 4: Averaged location statistics of the retrieved rainfall for the entire orbits containing the experiments (Exp.) demonstrated in (a–Fig. 8), (b–Fig. 9), and (c–Fig. 10). Shown are the mean and q th quantile (Qq) of the rainfall distribution in mm/hr over the support set of each product, where q ∈ {5, 25, 50, 75, 95}.

18

Figure 12: Annual estimates of total rainfall [mm] in 2013 mapped onto a graticule at grid size 0.1◦ × 0.1◦ . From top-to-bottom panels: TMI-2A12, reference 2A25, and ShARP products.

4.3

Cumulative experiments

To compare the results of our algorithm with those of the TMI-2A12 and the reference PR-2A25 in a cumulative sense, we focus on the TMI spectral observations in an annual scale. To unify the sampling rate of the TMI and PR sensor, we mapped datacubes of the spectral brightness temperatures onto the PR grid coordinates over the inner swath for all of the TRMM orbital tracks throughout the calendar year of 2013. To this end, we simply used a nearest neighborhood registration approach. In other words, for each orbital track, every point on the TMI grids, within the inner swath, was mapped onto the nearest point on the PR grids. Fig. 12, demonstrates annual rainfall estimates at spatial resolution 0.1◦ × 0.1◦ , obtained from 2A12, 2A25, and ShARP products in the calendar year of 2013. Note that, in this figure, for those areas where the PR does not provide validated rainfall data (i.e., over and near Australia), we did not report any retrieval results from both 2A12 and ShARP products. Therefore, the retrieved annul rainfall estimates in that region may not be realistic but provide a benchmark to compare the retrieval results over the raining areas detected and reported by the PR. We see that the global rainfall pattern is well retrieved by both retrieval approaches. At this resolution, comparing the retrieval products with the reference 2A25 reveals that the estimation error is still relatively large. Specifically, the normalized root mean squared error (RMSEn )1 is about 36% and 48% for ShARP (Fig. 12—bottom panel) and 2A12 (Fig. 12—top panel), respectively. In coarser resolution at 1◦ × 1◦ grids, this error metric reduces to 17% and 31% (see Fig. 13), while the overall correlation of the annual rainfall increases from 0.92 for 2A12 to 0.97 for ShARP (Fig. 14). Zonal mean values 1 Normalized root mean squared error RMSE is the mean squared error between the retrieval product and the reference 2A25 normalized n by the square root of the sum of squared values of the 2A25 product.

19

Figure 13: Annual estimates of total rainfall error [mm] in 2013. From top-to-bottom panels: The difference between the TMI-2A12, and ShARP retrieval products with the reference 2A25 at grid size 1◦ × 1◦ . Hot (red) and cold (blue) colors denote intensity of positive and negative retrieval error.

are also demonstrated and compared in Fig. 15, while quantitative comparisons are reported in Table 5. Over ocean, except over north Atlantic mid-latitude storm tracks, both retrieval methods are subject to over estimation while most of the underestimation regions are concentrated over land, especially near coastal zones, over islands, and peninsulas. Marked overestimation can be seen over central Africa and South America in both retrieval methods. Generally speaking, both 2A12 and ShARP recover well the rainfall patterns over the tropics. However, Fig. 12 and 13 show that both 2A12 and ShARP slightly overestimate (~ 300 mm/hr) the narrow ridge of high precipitation over the Intertropical Convergence Zone (ITCZ) across the Pacific Ocean. As is evident, over the South Pacific, Atlantic, and Indian Ocean convergence zones, we also see some over estimation by both retrieval algorithms, while the positive bias is markedly mitigated for ShARP (Fig. 13). In the north Atlantic mid-latitude storm tracks both retrievals slightly underestimate the annual rainfall while 2A12 is less biased and shows some improvements compared to ShARP. The noticeable improvements by our algorithm seem to be over land and coastal zones, where detection and estimation of low rainfall amounts from microwave spectral signatures is extremely challenging. Over the subtropical hot desert, arid, and semi-arid climates (e.g., Sahara, Arabian, and Syrian deserts, central Iran plateau, and central Australia), we see that ShARP retrieves well the low rainfall amounts and performs better than 2A12 (Fig. 12). Over central Africa, 2A12 severely overestimates annual rainfall while ShARP shows noticeable improvements. Moreover, over South America, ShARP shows improved rainfall estimation over Brazil and southern Amazon, while some under estimation can be seen over the northern Amazon basin, compared to 2A12. Noticeable differences also occur over the Tibetan highlands and Himalayas, which are notoriously problematic in rainfall microwave retrieval partly due to ground high emissivity. We can see that 2A12 severely overestimated (>300 mm/hr) the annual rainfall over Tibetan highlands while ShARP retrieval is less biased and the extent of error is dramatically reduced. Note that, we have used minimal number of earth surface classifications and have not defined any specific class for example over Tibetan Plateau. Indeed, due to the nearest neighborhood selection of sub-dictionaries, our algorithm is relatively insensitive to the effects of earth surface emissivity on the background radiation regime. Over Southeast Asia, where the presence of land-ocean interface dominates and mixes the background radiation, both algorithms are subject to notable underestimation. However, we can see that the negative bias in ShARP is markedly reduced compared to 2A12, especially over Indonesia, Malaysia, and Philippines.

20

4000

r 2 = 0.94 RMSE n = 17%

3000

corr = 0.97

0

0

1000

2000

3000

Rainfall [mm]−2A25

corr = 0.92

2000

RMSE n = 31%

1000

Rainfall [mm]−2A25

4000

r 2 = 0.84

0

1000

2000

3000

4000

0

1000

Rainfall [mm]−2A12

2000

3000

4000

Rainfall [mm]−ShARP

Figure 14: Smooth scatter plots of total annual retrieved rainfall [mm] versus the reference 2A25 at grid size 1◦ × 1◦ . Left-toright panels: Pairs of 2A12 and ShARP versus the reference 2A25, respectively. Hot (red) and cold (blue) colors denote higher and lower density of the available rainfall pairs.

Annual zonal mean 2013 2A25 ShARP 2A12

Latitude

30N

Annual zonal mean 2013 − over ocean

Annual zonal mean 2013 − over land

30N

30N

20

20

20

10

10

10

0

0

0

−10

−10

−10

−20

−20

−20

−30S

−30S

−30S

500

1000

1500

Rainfall [mm]

2000

500

1000

1500

Rainfall [mm]

2000

500

1000

1500

2000

Rainfall [mm]

Figure 15: Estimates of annual rainfall zonal mean values [mm] obtained from estimates of annual rainfall shown in Fig. 12. From left-to-right panels: zonal mean values computed over all surface classes, over ocean, and over land plus coastal areas.

21

Product

ShARP

2A12

Annual Zonal Mean

Surface Class

RMSE [mm]

BIAS [mm]

Total

40.20

-6.53

Ocean

47.61

7.15

Land + Coast

95.67

-41.02

Total

103.04

73.63

Ocean

99.50

69.42

Land + Coast

137.62

79.43

Table 5: Retrieval skills for annual zonal mean values shown in Fig. 15.

Comparison of total annual zonal mean values over all surface classes (Fig. 15–left panel) shows that ShARP well approximates the average latitudinal rainfall distribution. We can see that not only over tropics but also over midlatitudes, where the stratiform rainfall is dominant, ShARP well reconstructs the reference 2A25 product and exhibits appreciable improved results (see Table 5). Over the ocean part of the ITCZ, both retrieval methods are apparently subject to some over estimation (Fig. 15–middle panel) while ShARP closely follows the reference 2A25 over midlatitudes to a better extent than 2A12. Over land, ShARP underestimates annual rainfall over a narrow band (latitudes 5◦ S-N) around the tropics, while it captures well the rainfall latitudinal patterns over the subtropical climate zones (Fig. 15–right panel). Quantitative comparison of these zonal profiles is presented in Table 5.

5

Concluding remarks

We proposed a new supervised microwave rainfall retrieval algorithm the called Shrunken Locally Linear Embedding Algorithm for Retrieval of Precipitation (ShARP). This algorithm makes use of a sequence of detection-estimation techniques. In the detection step, the algorithm uses a nearest neighborhood classification rule, while in the estimation step it learns the local correspondence of spectral and rainfall manifolds using a pair of dictionaries. These dictionaries are indeed collections of spectral brightness temperatures and their corresponding rainfall profiles. The algorithm exploits a modern constrained shrinkage estimator to ensure physical consistency and locally unbiased retrieval. We reported the performance of our algorithm using the data provided by the multi-sensor instruments aboard the Tropical Rainfall Measuring Mission (TRMM) satellite. We emphasized its pros and cons by comparing its results with the standard 2A12 (version 7) retrieval product of the Goddard Profiling Algorithm (GPROF). While we have examined the algorithmic performance using empirical dictionaries obtained from coincidental TRMM precipitation radar (PR) and Microwave Imager (TMI), the core of our algorithm is flexible enough and can exploit physically-based generated dictionaries using cloud resolving and radiative transfer models. Motivated by the results of our comparisons, it would be worthwhile to optimally integrate the ShARP retrieval products with the results of other algorithms (e.g., GPROF, UW) in future research. This optimal integration allows us to take advantage of all viable algorithms to improve quality of spaceborne microwave rainfall retrieval. Further refinements in implementation technicalities, such as smarter choices of surface classes by considering the ground emissivity, skin temperature, and rainfall climatological patterns, can definitely improve the algorithmic performance. In addition, we are planning to develop a new version of the ShARP algorithm that uses compact dictionaries for faster and more accurate retrieval with particular emphasis on the available spectral observations (10.65 to 183 GHz) and dual frequency precipitation radar aboard the successfully launched Global Precipitation Measuring (GPM) satellites.

Acknowledgments First author would like to thank Prof. C. D. Kummerow for his invaluable advice at the early stage of this paper and Prof. G. Lerman for his very insightful and helpful discussions. This work was supported mainly by a NASA Earth and Space Science Fellowship under the contract NNX12AN45H, the K. Harrison Brown Family Chair, and the Ling Endowed Chair funding. Furthermore, the support provided by two NASA Global Precipitation Measurement grants and the Belmont Forum DELTAS grant under the contract NNX13AG33G, NNX13AH35G and ICER-1342944 are also greatly recognized. The TRMM 2A12 and 2A25 data were obtained through the anonymous File Transfer Protocol publicly available at ftp://trmmopen.gsfc.nasa.gov/pub/trmmdata.

22

References [1] G. E. Stout and E. A. Mueller, “Survey of Relationships Between Rainfall Rate and Radar Reflectivity in the Measurement of Precipitation,” J. Appl. Meteor., vol. 7, no. 3, pp. 465–474, Jun. 1968. [2] W. Krajewski and J. Smith, “Radar hydrology: rainfall estimation,” Adv. Water Resour., vol. 25, no. 8–12, pp. 1387–1394, 2002. [3] M. New, M. Todd, M. Hulme, and P. Jones, “Precipitation measurements and trends in the twentieth century,” Int. J. Climatol., vol. 21, no. 15, pp. 1889–1922, 2001. [4] Barrett, “The Estimatipo of Monthly Rainfall from Satellite,” Mon. Wea. Rev., vol. 98, no. 4, pp. 322–327, Apr. 1970. [5] R. H. Blackmer, “Correlation of Cloud Brightness and Radiance with Precipitation Intensity.” Stanford Res. Inst. Menlo Park, CA. Final Report–Task B (NEPRF TR8–75 (SRI)), Tech. Rep., 1975. [6] B. J. Kilonsky and C. S. Ramage, “A Technique for Estimating Tropical Open-Ocean Rainfall from Satellite Observations,” J. Appl. Meteor., vol. 15, no. 9, pp. 972–975, Sep. 1976. [7] P. A. Arkin, “The Relationship between Fractional Coverage of High Cloud and Rainfall Accumulations during GATE over the B-Scale Array,” Mon. Wea. Rev., vol. 107, no. 10, pp. 1382–1387, Oct. 1979. [8] S. Lovejoy and G. Austin, “The delineation of rain areas from visible and IR satellite data for GATE and mid– latitudes,” Atmosphere-Ocean, vol. 17, no. 1, pp. 77–92, 1979. [9] P. A. Arkin and P. E. Ardanuy, “Estimating Climatic-Scale Precipitation from Space: A Review,” J. Climate, vol. 2, no. 11, pp. 1229–1238, Nov. 1989. [10] K. Hsu, X. Gao, S. Sorooshian, and H. Gupta, “Precipitation estimation from remotely sensed information using artificial neural networks,” J. Appl. Meteorol., vol. 36, no. 9, pp. 1176–1190, 1997. [11] H. V. G. X. G. Hsu, K. and S. Sorooshian, “Estimation of physical variables from multichannel remotely sensed imagery using a neural network: Application to rainfall estimation,” Water Resour. Res., vol. 35(5), pp. 1605–1618, 1999. [12] S. Sorooshian, K. Hsu, G. Xiaogang, H. Gupta, B. Imam, and D. Braithwaite, “Evaluation of PERSIANN system satellite-based estimates of tropical rainfall,” Bull. Amer. Meteor. Soc., vol. 81, no. 9, pp. 2035–2046, 2000. [13] Y. Hong, K. Hsu, S. Sorooshian, and X. Gao, “Precipitation estimation from remotely sensed imagery using an artificial neural network cloud classification system,” J. Appl. Meteorol., vol. 43, no. 12, pp. 1834–1853, 2004. [14] R. R. Ferraro and G. F. Marks, “The Development of SSM/I Rain-Rate Retrieval Algorithms Using Ground-Based Radar Measurements,” J. Atmos. Oceanic Technol., vol. 12, no. 4, pp. 755–770, Aug. 1995. [15] T. Wilheit, C. Kummerow, and R. Ferraro, “Rainfall algorithms for AMSR-E,” IEEE Trans. Geosci. Remote Sens., vol. 41, no. 2, pp. 204–214, feb. 2003. [16] N.-Y. Wang, C. liu, R. Ferraro, D. Wolff, E. Zipserand, and C. Kummerow, “TRMM 2A12 Land Precipitation Product - Status and Future Plans,” Journal of the Meteorological Society of Japan. Ser. II, vol. 87A, pp. 237–253, 2009. [17] J. Bytheway and C. Kummerow, “A Physically Based Screen for Precipitation Over Complex Surfaces Using Passive Microwave Observations,” IEEE Trans. Geosci. Remote., vol. 48, no. 1, pp. 299–313, 2010. [18] G. W. Petty and K. Li, “Improved Passive Microwave Retrievals of Rain Rate Over Land and Ocean. 1. Algorithm description,” J. Atmos. Oceanic Technol., 2013. [19] G. W. Petty and K. Li, “Improved Passive Microwave Retrievals of Rain Rate Over Land and Ocean. 2. Validation and Intercomparison,” J. Atmos. Oceanic Technol., 2013. [20] T. T. Wilheit, A. T. C. Chang, M. S. V. Rao, E. B. Rodgers, and J. S. Theon, “A Satellite Technique for Quantitatively Mapping Rainfall Rates over the Oceans,” J. Appl. Meteor., vol. 16, no. 5, pp. 551–560, May 1977.

23

[21] T. T. Wilheit, A. T. C. Chang, and L. S. Chiu, “Retrieval of Monthly Rainfall Indices from Microwave Radiometric Measurements Using Probability Distribution Functions,” J. Atmos. Oceanic Technol., vol. 8, no. 1, pp. 118–136, Feb. 1991. [22] C. Kummerow, W. S. Olson, and L. Giglio, “A simplified scheme for obtaining precipitation and vertical hydrometeor profiles from passive microwave sensors,” IEEE Trans. Geosci. Remote., vol. 34, no. 5, pp. 1213–1232, 1996. [23] C. Kummerow, Y. Hong, W. Olson, S. Yang, R. Adler, J. McCollum, R. Ferraro, G. Petty, D. Shin, and T. Wilheit, “The evolution of the Goddard Profiling Algorithm (GPROF) for rainfall estimation from passive microwave sensors,” J. Appl. Meteorol., vol. 40, no. 11, pp. 1801–1820, 2001. [24] C. D. Kummerow, S. Ringerud, J. Crook, D. Randel, and W. Berg, “An Observationally Generated A Priori Database for Microwave Rainfall Retrievals,” J. Atmos. Oceanic Technol., vol. 28, no. 2, pp. 113–130, Sep. 2011. [25] M. Grecu and E. N. Anagnostou, “Use of Passive Microwave Observations in a Radar Rainfall-Profiling Algorithm,” J. Appl. Meteor., vol. 41, no. 7, pp. 702–715, Jul. 2002. [26] M. Grecu, W. S. Olson, and E. N. Anagnostou, “Retrieval of Precipitation Profiles from Multiresolution, Multifrequency Active and Passive Microwave Observations,” J. Appl. Meteor., vol. 43, no. 4, pp. 562–575, Apr. 2004. [27] M. Grecu and W. S. Olson, “Bayesian Estimation of Precipitation from Satellite Passive Microwave Observations Using Combined Radar-Radiometer Retrievals,” J. Appl. Meteor. Climatol., vol. 45, no. 3, pp. 416–433, 2006. [28] Z. Haddad, E. Smith, C. Kummerow, T. Iguchi, M. Farrar, S. Durden, M. Alves, and W. Olson, “The TRMM ’day-1’ radar/radiometer combined rain-profiling algorithm,” J. Meteor. Soc. Japan, vol. 75, no. 4, pp. 799–809, 1997. [29] K. F. Evans, J. Turk, T. Wong, and G. L. Stephens, “A Bayesian Approach to Microwave Precipitation Profile Retrieval,” J. Appl. Meteor., vol. 34, no. 1, pp. 260–279, jan 1995. [30] C. Kummerow, W. Barnes, T. Kozu, J. Shiue, and J. Simpson, “The Tropical Rainfall Measuring Mission (TRMM) Sensor Package,” J. Atmos. Oceanic Technol., vol. 15, no. 3, pp. 809–817, Jun. 1998. [31] T. Iguchi, T. Kozu, R. Meneghini, J. Awaka, and K. i. Okamoto, “Rain-Profiling Algorithm for the TRMM Precipitation Radar,” J. Appl. Meteor., vol. 39, no. 12, pp. 2038–2052, Dec. 2000. [32] T. Oguchi, “Electromagnetic wave propagation and scattering in rain and other hydrometeors,” Proceedings of the IEEE, vol. 71, no. 9, pp. 1029–1078, 1983. [33] M. A. Janssen, Atmospheric remote sensing by microwave radiometry, M. A. Janssen, Ed. Wiley & Sons, Inc., 1993.

New York, NY: John

[34] C. D. Rodgers, “Retrieval of atmospheric temperature and composition from remote measurements of thermal radiation,” Rev. Geophys., vol. 14, no. 4, pp. 609–624, 1976. [35] E. A. Sharkov, Passive Microwave Remote Sensing of the Earth: Physical Foundations. Publishing, Springer, 2003.

Chichester, UK: Praxis

[36] S. T. Roweis and L. K. Saul, “Nonlinear Dimensionality Reduction by Locally Linear Embedding,” Science, vol. 290, no. 5500, pp. 2323–2326, 2000. [37] A. M. Ebtehaj, E. Foufoula-Georgiou, and G. Lerman, “Sparse regularization for precipitation downscaling,” J. Geophys. Res., vol. 116, 2012. [38] D. Donoho, “De-noising by soft-thresholding,” IEEE Trans. Inform. Theory., vol. 41, no. 3, pp. 613–627, may 1995. [39] R. Tibshirani, “Regression Shrinkage and Selection via the Lasso,” J. R. Stat. Soc. Ser. B Stat. Methodol., vol. 58, no. 1, pp. 267–288, 1996. [40] S. S. Chen, D. L. Donoho, and M. A. Saunders, “Atomic decomposition by basis pursuit,” SIAM J. Sci. Comput., vol. 20, pp. 33–61, 1998. [41] M. Elad, Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. New York, NY.: Springer Verlag, 2010.

24

[42] A. Tikhonov, V. Arsenin, and F. John, Solutions of ill-posed problems. [43] P. Hansen, Discrete inverse problems: insight and algorithms. Applied Mathematics (SIAM), 2010, vol. 7.

Winston & Sons. Washington, DC, 1977.

Philadelphia, PA, USA: Society for Industrial &

[44] H. Zou and T. Hastie, “Regularization and variable selection via the elastic net,” J. R. Stat. Soc. Ser. B Stat. Methodol., vol. 67, no. 2, pp. 301–320, 2005. [45] N. C. Grody, “Classification of snow cover and precipitation using the special sensor microwave imager,” Journal of Geophysical Research: Atmospheres, vol. 96, no. D4, pp. 7423–7435, 1991. [46] C. Kummerow and L. Giglio, “A Passive Microwave Technique for Estimating Rainfall and Vertical Structure Information from Space. Part I: Algorithm Description,” J. Appl. Meteor., vol. 33, no. 1, pp. 3–18, Jan. 1994. [47] Q. Li, R. L. Bras, and D. Veneziano, “Passive microwave remote sensing of rainfall considering the effects of wind and nonprecipitating clouds,” Journal of Geophysical Research: Atmospheres, vol. 101, no. D21, pp. 26 503–26 515, 1996. [48] M. D. Conner and G. W. Petty, “Validation and Intercomparison of SSM/I Rain-Rate Retrieval Methods over the Continental United States,” J. Appl. Meteor., vol. 37, no. 7, pp. 679–700, Jul. 1998. [49] R. R. Ferraro, E. A. Smith, W. Berg, and G. J. Huffman, “A Screening Methodology for Passive Microwave Precipitation Retrieval Algorithms,” J. Atmos. Sci., vol. 55, no. 9, pp. 1583–1600, May 1998. [50] S. Seto, N. Takahashi, and T. Iguchi, “Rain/No-Rain Classification Methods for Microwave Radiometer Observations over Land Using Statistical Information for Brightness Temperatures under No-Rain Conditions,” J. Appl. Meteor., vol. 44, no. 8, pp. 1243–1259, Aug. 2005. [51] T. D. Ellis, T. L’Ecuyer, J. M. Haynes, and G. L. Stephens, “How often does it rain over the global oceans? The perspective from CloudSat,” Geophys. Res. Lett., vol. 36, no. 3, 2009. [52] S. Boyd and L. Vandenberghe, Convex optimization.

New York: Cambridge University Press, 2004.

[53] Y. Zhang, “Solving Large-Scale Linear Programs by Interior-Point Methods Under the MATLAB Environment,” Department of Mathematics and Statistics, University of Maryland, Baltimore County, Baltimore, MD, Technical Report TR96-01, July, Tech. Rep., 1995. [54] S. Mehrotra, “On the Implementation of a Primal-Dual Interior Point Method,” SIAM Journal on Optimization, vol. 2, pp. 575–601, 1992.

25