Meridian Filtering for Robust Signal Processing - Google Sites

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 8, AUGUST 2007

3949

Meridian Filtering for Robust Signal Processing Tuncer Can Aysal, Student Member, IEEE, and Kenneth E. Barner, Senior Member, IEEE

Abstract—A broad range of statistical processes is characterized by the generalized Gaussian statistics. For instance, the Gaussian and Laplacian probability density functions are special cases of generalized Gaussian statistics. Moreover, the linear and median filtering structures are statistically related to the maximum likelihood estimates of location under Gaussian and Laplacian statistics, respectively. In this paper, we investigate the well-established statistical relationship between Gaussian and Cauchy distributions, showing that the random variable formed as the ratio of two independent Gaussian distributed random variables is Cauchy distributed. We also note that the Cauchy distribution is a member of the generalized Cauchy distribution family. Recently proposed myriad filtering is based on the maximum likelihood estimate of location under Cauchy statistics. An analogous relationship is formed here for the Laplacian statistics, as the ratio of Laplacian statistics yields the distribution referred here to as the Meridian. Interestingly, the Meridian distribution is also a member of the generalized Cauchy family. The maximum likelihood estimate under the obtained statistics is analyzed. Motivated by the maximum likelihood estimate under meridian statistics, meridian filtering is proposed. The analysis presented here indicates that the proposed filtering structure exhibits characteristics more robust than that of median and myriad filtering structures. The statistical and deterministic properties essential to signal processing applications of the meridian filter are given. The meridian filtering structure is extended to admit real-valued weights utilizing the sign coupling approach. Finally, simulations are performed to evaluate and compare the proposed meridian filtering structure performance to those of linear, median, and myriad filtering. Index Terms—Generalized Cauchy distribution (GCD), generalized Gaussian distribution (GGD), heavy-tails, impulsive noise, maximum likelihood estimation, ratio distributions, robust filtering.

I. INTRODUCTION

T

HE CLASS OF maximum likelihood type estimators ( -estimators) of location, which were developed in the theory of robust statistics [1], has been of fundamental importance in the development of robust signal processing techniques [2]. Robust nonlinear estimators are critical for applications involving impulsive processes (e.g., radar clutter, ocean acoustic noise, and multiple access interference in wireless system

Manuscript received June 19, 2006; revised November 13, 2006. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Marcelo G. S. Bruno. T. C. Aysal was with the Department of Electrical and Computer Engineering, University of Delaware, Newark, DE 19716 USA. He is now with the Electrical and Computer Engineering Department, McGill University, Montreal, QC H1T 3E2 Canada (e-mail: [email protected]). K. E. Barner is with the Department of Electrical and Computer Engineering, University of Delaware, Newark, DE 19716 USA (e-mail: [email protected]. udel.edu). Digital Object Identifier 10.1109/TSP.2007.894383

communications), where heavy-tailed non-Gaussian distributions model the underlying signals [3]–[6]. Given a set of observations (input samples) , an -estimate of their common location is given by (1) is the cost function of the -estimators. Maxwhere imum likelihood location estimates form a special case of -estimators, with the observations being independent and , where is identically distributed and the common density function of the samples. The weighted mean (linear), weighted myriad, and weighted median filter families are well-derived from maximum likelihood location estimator under Gaussian, Cauchy, and Laplacian statistics, respectfully, [7]–[9]. The cost functions to minimize, in these , , where cases, are given by is the linearity parameter [8], [9] and for mean, myriad, and median estimators, respectively. In this paper, we note that the Gaussian and Laplacian distributions are special cases of the generalized Gaussian distribuand , retions (GGD) family corresponding to the spectively, where is the tail parameter. Then, we focus on the well-established statistical relation between the Gaussian and Cauchy distributions, indicating that the ratio of two independent Gaussian random variables is Cauchy distributed. We note that the Cauchy distribution is a special case of the generalized Cauchy distributions (GCD) family corresponding to , where is the tail parameter. An analogous statistical relationship is constructed here for the Laplacian distribution, where the distribution function of the random variable formed as the ratio of two independent Laplacian distributed random variables is derived. Interestingly, it is shown that the obtained statistics, referred to as the Meridian distribution, is also a member of the GCD with . Hence, a connection between the GGD and GCD families is formed. The maximum likelihood estimate under the new statistics is analyzed, where the cost function, in this case, is given by (2) with controlling the robustness of the meridian estimator.1 The fact that the meridian estimator is likelihood-based guarantees that the estimate is (at least asymptotically) unbiased, consistent, and efficient in Meridian statistics. Properties and robustness of the meridian estimator are analyzed. Specifically, an influence function study shows that the meridian estimator exhibits desirable features. Derived properties also show that 1The “meridian” nomenclature is adopted since meridian means peak and the obtained density function is peaky.

1053-587X/$25.00 © 2007 IEEE

3950

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 8, AUGUST 2007

the meridian estimator provides characteristics desired in robust signal processing applications. From a statistical signal processing point of view, plain location -estimators, i.e., those using i.i.d., observation samples, do not adequately capture the true statistical relationship among samples of a typical input signal. A more appropriate estimator formulation is obtained by considering samples that are independent but not identically distributed. In particular, we consider those with common location , but varying scale factors [7]–[9]. Using this approach, meridian estimators are generalized to include non-negative cost function weights. Properties derived for unweighted estimator are extended to the weighted case. Also, utilizing the sign coupling approach [8], the weighted meridian filter is extended to admit real-valued weights. The remainder of this paper is organized as follows. Optimal filtering is approached from a maximum likelihood perspective in Section II. Utilizing the fact that Gaussian and Laplacian distributions are special cases of generalized Gaussian distribution, the statistical relationship between the Gaussian and Cauchy distributions is determined and the maximum likelihood analysis under Cauchy statistics is given. Noting that Cauchy distribution is a special case of the GCD, an analogous relationship for the Laplacian distribution is constructed. This leads to the Meridian distribution and form a connection between GGD and GCD. Section III presents the maximum likelihood analysis under Meridian statistics, along with the statistical and robustness analysis of the corresponding cost function. The meridian estimator is also generalized to admit non-negative weights. In Section IV, the meridian filter is generalized to admit real-valued weights. Simulations evaluating and comparing the performance of the proposed meridian filtering structure to those of linear, median, and myriad filters are given in Section V. Finally, concluding remarks and some discussions about the future works are given in Section VI.

A. Generalized Gaussian Distribution and Filtering A broad range of statistical processes can be characterized by the generalized Gaussian probability density function (PDF)

(3) where is the Gamma function, is a constant defined as , and is the standard deviation. In this representation, the scale of the PDF is , and the impulsiveness determined by the scale parameter is determined by the tail parameter . This representation . includes the standard Gaussian PDF as a special case , the PDF’s tail decays slower than in the Gaussian For case, resulting in a heavier-tailed PDF. Indeed, a second special case that is of particular interest is the Laplacian PDF, which is . realized when Consider the problem of estimating the constant amplitude of noisy obsignal from the samples servation data . Let (4) where the terms are independent and identically distributed zero-mean noise. The ML estimate of is given by (5) In the following, the ML estimate of location under Gaussian and Laplacian statistics are briefly reviewed. Since the results corresponding to these cases are well-known [1], [7], [8], the proofs are not given. independent samples Theorem 1: Consider a set of each obeying the Gaussian distribution with variance . The ML estimate of location is given by

II. STATISTICAL PROCESSES, MAXIMUM LIKELIHOOD ESTIMATION, AND FILTERING This section discusses statistical models of observed samples obeying generalized Gaussian statistics and relates the filtering problem to maximum likelihood (ML) estimation. Specifically, the Gaussian and heavy-tailed Laplacian cases, corresponding to and , respectively, where is the tail parameter, are considered. Subsequently, we note the statistical relation between the Gaussian and Cauchy distributions, where the latter correspond to the special case under generalized Cauchy statistics, and is the tail parameter. In particular, we note that the Cauchy distributed random variable can be formed as the ratio of two independent Gaussian distributed random variables. Statistical models of observed samples under Cauchy statistics are discussed and the filtering problem under such statistics is related to ML estimation. An analogous relationship for Laplacian statistics is formed here and, surprisingly, another distribution from GCD family is obtained, i.e., the distribution of the random variable formed as the ratio of two independent Laplacian distributed random variables is derived. The filtering problem under the obtained statistics is then approached from an ML perspective.

(6) , where This is a mean filter structure denotes the filter output. The ML estimate of location can be generalized to admit weights to enable flexible filtering characteristics. independent samples Theorem 2: Consider a set of each obeying the Gaussian distribution with (possibly) different variances . The ML estimate of location is given by

(7) . where This is a normalized linear filter structure , where the ’s are the filter weights. Enforcing the positivity

AYSAL AND BARNER: MERIDIAN FILTERING FOR ROBUST SIGNAL PROCESSING

constraint on the weights constrains the resulting filters to be smoothers. In practice, however, this constraint is relaxed, enabling FIR filters to take on a wide array of spectral characteristics. Note that the tails of the Laplacian distribution is heavier than that of the Gaussian distribution . The location estimation under heavier-tailed Laplacian statistics is considered next. Theorem 3: Consider a set of independent samples each obeying the Laplacian distribution with variance . The ML estimate of location is given by (8) This is a median filter structure , where denotes the filter output. Similar to the mean filtering, median filtering can also be generalized to admit weights. independent samples Theorem 4: Consider a set of each obeying Laplacian distribution with (possibly) different variances . The ML estimate of location is given by (9) where as

and

is the replication operator defined

The weight positivity constraining the filters to smoothers can, as in the FIR filter case, be relaxed to enable more general filtering characteristics [8] (10) where if , , if and , if . Considerable analysis is available in the literature on the detail preservation and outliers rejection characteristics of weighted median (WM) filters [7], [8], [10]–[12].

3951

slower than in the Cauchy case, resulting in a heavier-tailed PDF. In the following, the well-established statistical relation between Gaussian and Cauchy distributions is discussed [16]. Also, the recently proposed myriad filtering structure [6], [9], [17], which is based on the Cauchy distribution, is presented. Proposition 1: The RV formed as the ratio of two independent zero-mean Gaussian distributed RVs and , with variances and , respectively, is Cauchy distributed (general) with . ized Cauchy distribution with Weighted myriad filters have recently been proposed as a class of robust, nonlinear filters based on the Cauchy distribution. They have been used in robust communications and image processing applications [6], [9], [18]. These filters have been derived as extensions of the sample myriad, defined as the ML estimate of location for the Cauchy distribution [9], [17], [18]. Remark 1: It should be noted that the original authors derived the myriad filter starting from -stable distributions [3]. They noted that there is only two closed-form expression for -stable and corresponding to Gaussian and distributions, Cauchy distributions, respectively. This original development [6], [9], [17], [18] did not mention or utilize the statistical relation between Gaussian and Cauchy distributions. independent samples Theorem 5: Consider a set of each obeying the Cauchy distribution with common parameter . The ML estimate of location, or sample myriad, is given by

(13) where is the linearity parameter [18]. Note that, unlike the mean or median, the definition of the myriad involves the free-tunable parameter . Interestingly, the myriad includes the mean as a limiting case of [18]. As in the previous cases, the sample myriad is generalized to admit weights [9]. independent samples Theorem 6: Consider a set of each obeying the Cauchy distribution with (possibly) varying scale factors [6], [9]. The ML estimate of location, or weighted myriad is given, in this case, by

B. Generalized Cauchy Distribution and Filtering The generalized Cauchy distribution was first proposed by Rider in 1957 [13] and “rediscovered” under a different parametrization by Miller and Thomas in 1972 [14]. The GCD is used in several studies of impulsive radio noise [5], [14], [15]. The generalized Cauchy PDF is given by (11) with (12) In this representation, is the scale parameter and is the tail constant. This representation includes the standard Cauchy PDF . For , the PDF’s tail decays as a special case

(14) where denotes the weighting operation in the minimization problem. As in the previous cases, the weight positivity constraint restricting the filters to be smoothers can be relaxed to enable more general filtering characteristics [6], [9] (15) Next, a relationship similar to that between the Gaussian and Cauchy statistics is constructed here for the Laplacian case. That is, the distribution of the RV formed as the ratio of two independent Laplacian RVs is considered.

3952

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 8, AUGUST 2007

Fig. 1. Cauchy (solid line) and Meridian (dotted line) probability density functions. Also shown is the plot zoomed into the tail information.

Proposition 2: The RV formed as the ratio of two independent zero-mean Laplacian distributed RVs and , with scale and , respectively, is a member of the GCD parameters and , and is referred to as the family, with meridian distribution. Proof: Let be the RV formed as the ratio of two RVs and : . The PDF of the RV , , is given by [16] (16) where and

is the joint PDF of and . In the case where are independent, (16) reduces to (17)

where and denote the PDF of and , reand with Laplacian spectively. Solving (17) for RVs and PDFs , gives (18) Performing some manipulations and setting

yields (19)

A careful inspection of the observed distribution shows that belongs to the GCD family, corresponding specifically to case. We refer to as the Meridian distribution. the The standard Cauchy and Meridian PDFs are plotted in Fig. 1, . Fig. 1(a) shows the PDFs, whereas Fig. 1(b) where zooms in to show the tail weight. It can be seen that the Meridian distribution has a peaky shape, unlike the Cauchy distribution,

Fig. 2. Determined relationships between generalized Gaussian and generalized Cauchy distributions. g (1) denotes the ratio operation of two RVs.

which has a bell shape. Also, as expected, the meridian PDF exhibits tails heavier than that of the Cauchy PDF. Remark 2: It is very interesting to note that the ratio of two , yields the GCD, generalized Gaussian distributions, with with . Also, the ratio of two generalized Gaussian distri, yields the GCD, with .2 The deterbutions, with mined connections between generalized Gaussian and Cauchy , distribution families are shown in Fig. 2, where and denote the ratio operation of two RVs, the generalized Gaussian PDF, and generalized Cauchy PDF, respectively. III. MERIDIAN AND WEIGHTED MERIDIAN FILTERING In this section, location estimation from observed samples under Meridian statistics is considered and the filtering problem is related to ML estimation in an analogous fashion to the previous Gaussian, Laplacian, and Cauchy cases. Given the tail weight of the considered distribution, Meridian filtering is proposed for robust signal processing. Following determination of meridian filtering, several properties critical to signal processing applications are given. The meridian filtering is then extended to weighted meridian filtering. independent samples Theorem 7: Consider a set of each obeying the Meridian distribution with 2The generalization of this statistical relation for any k remains an open problem.

AYSAL AND BARNER: MERIDIAN FILTERING FOR ROBUST SIGNAL PROCESSING

3953

1) 2) 3) 4)

1)

( is strictly increasing) for , and ( is strictly decreasing) for . lie within the range of input All local extrema of samples, . has a finite number of local The objective function minima [input samples]. , i.e., The meridian is one of the local minima of one of the input samples. Proof: yields Differentiating (26)

Fig. 3. Sketch of a typical meridian objective function. Input samples are x = [4:9; 0:0; 6:5; 10:0; 9:5; 1:7; 1] and  = 1.

Note that that reduces to

for

. This also implies . Thus,

(27)

common scale parameter . The ML estimate of location, , or sample meridian, is given by

(20) where is referred to as the medianity parameter. Proof: The ML estimate is formulated as . Replacing the Meridian distribution for each sample yields (21) Utilizing basic properties of the argmax function and noting that maximizing the fraction is equivalent to minimizing the denominator, (21) reduces to (22) (23) Taking the natural log of (23) yields the final result. The performance of the meridian filtering is directly related to the objective function that arises naturally from the PDF. The following proposition presents several key properties of the meridian objective function. The properties described in the following are illustrated by Fig. 3, which illustrates the objective that results from a set of observed samples in the function case. window size of denote the order statistics [19] Proposition 3: Let the smallest and the largest. of the input vector , with Also, define

The desired result follows by noting that . Similarly, . In this case, to

(28) which gives the desired result. if or 2) From 1), we see that . That is, the local extremas of , lie in the range of the input samples. for any . 3) Let is rewritten as The objective function

(29) Taking the natural log of (29) yields

(30) Differentiating (30) with respect to twice yields the first given by and second derivative of (31) and

(24) and (25) The following statements hold.

reduces

(32) respectively. Note that for any is concave in . Since any points of the objective function

for indicating that intervals for are critical [note that is

3954

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 8, AUGUST 2007

not differentiable for

] and is concave in for any , the values, hence the values, are the only possible local minimas of the objective function . 4) Since is the global minimum of , and from 1)–3), it follows that the global minimum must occur at one of its local minima, i.e., one of the input samples. This completes the proof of the proposition. The meridian estimator output is hence the input sample that function value. The selective nature of yields the smallest the meridian estimator, shared with the median estimator, facilitates the filter output computation which is formulated as (33) (34) Since the sample meridian is optimal for the heavy-tailed Meridian distribution, we should intuitively expect the estimator to be robust in the presence of impulsive noise. In addition to its rich set of properties, we show that the sample meridian and derived filtering methods, satisfy important optimality conditions under various impulsive noise models, including the Laplacian. Note that, unlike the sample mean and median, the definition of sample meridian involves the free-tunable parameter , which is similar in function to the sample myriad parameter . This parameter plays a fundamental role in the theory of meridian filters, and is the topic of study in the following subsection. A. Medianity Parameter The behavior of the meridian estimator is markedly dependent on the value of its medianity parameter . We show that for large values of , the sample meridian is equivalent to the sample median, whereas for small , the estimator acquires the form of the sample mode. That is, in the latter case, the output is equal to one of the most repeated values. Property 1: (Median Property) Given a set of samples , the sample meridian converges to the sample . This is median as

Since (40) it follows that (41) This concludes the proof, since (41) corresponds to the objective function of the sample median estimator. Plainly, an infinite value of converts the meridian into the sample median. This behavior explains the choice of medianity for the name of the parameter. It is important to emphasize that the family of meridian estimators subsumes the sample median as a limiting case. This simple fact makes the meridian filter class inherently more efficient than (or at least equally efficient to) median filters over all noise distribution, including the Laplacian. As the meridian moves, in function, away from the median region (large values of ) to lower medianity values, the estimator becomes more robust to the presence of impulsive noise. In the limit, when tends to zero, the meridian acquires its maximum resistance to impulsive noise. As shown below, this estimator treats every observation as a possible outlier, assigning more credibility to the most repeated values in the sample set. This “mode-type” characteristic leads to the mode-meridian name assigned to this estimator. Property 2: (Mode Property) Given a set of samples , the sample meridian converges to a . This is mode-type estimator as (42) is the set of most repeated values. where Proof: Since is a positive constant, the definition of the sample meridian can be reformulated as (43) where (44)

(35) Proof: Using the basic properties of the the estimator can be expressed as

function,

When

is very small, it is easy to check that (45)

(36)

(38)

is the number of times the value is repeated in where . In the sample set, and denotes the asymptotic order as must be minimized in order the limit, the exponent to minimize . Therefore, the mode-meridian is one of the . most repeated values in the sample. Now, let , expanding the product in (44) gives Then for

(39)

(46)

(37)

AYSAL AND BARNER: MERIDIAN FILTERING FOR ROBUST SIGNAL PROCESSING

Since the first term in (46) is negligible for small lated as

3955

, the second term is , and can be calcu(47) (48)

(49)

This property is also shared by the myriad estimator, as the myriad estimator also converges to a mode-type estimator for [18]. B. Robustness of the Sample Meridian Since the sample meridian belongs to the class of -estimators [1], many robust statistics tools are available for evaluating its robustness [1], [20]. M-estimators, also called genis eralized ML estimators, are formulated by (1) where an arbitrary objective function to be designed. Assuming that exists, the -estimator is obtained by solving (50) is proportional to the influence function. The influwhere ence function of an estimator determines the effect of contamination on the estimator. To further characterize M-estimates, it is useful to list the desirable features of a robust influence function [1], [20]. • B-robustness: An estimator is B-robust if the supremum of the absolute value of the influence function is finite. • Rejection Point: The rejection point, defined as the distance from the center of the influence function to the point where the influence function becomes negligible, should be finite. Rejection point measures whether the estimator rejects outliers and, if so, at what distance. The influence function for the sample mean, median, and , , and myriad can be shown to be , respectively. The influence function of the sample meridian is discussed in the following. Proposition 4: The influence function of the meridian estimator is given by

Fig. 4. Influence functions for (solid:) the sample mean, (dashed:) the sample median, (dotted:) the sample myriad, and (dash-dotted:) the sample meridian.

outliers. The myriad estimate is clearly B-robust and the effect of the errors decreases as the error increases. The meridian estimate is also B-robust, and in addition, the rejection point is smaller than that of myriad as it has a higher influence function decay rate. This indicates that the meridian is more robust than the myriad. In addition to desirable influence function features, the meridian possesses the followings properties important in signal processing applications. . Then Property 3: (Outlier Rejection) Let (52) Proof: The result can be shown as follows:

(53) (54)

(55)

(51) for the sample Proof: Note that is simply obtained meridian case. The influence function with respect to . by differentiating The influence functions for the sample mean, median, myriad, and meridian are depicted in Fig. 4. Since the influence function of the mean is unbounded, a gross error in the observations leads to severe distortion in the estimate. The mean is clearly not B-robust and its rejection point is infinite. On the other hand, a similar gross error has a limited effect on the median estimate. While the median is B-robust, its rejection point, like the mean, is not finite. Thus, the median estimate is always affected by

(56) Evaluating the limit yields

(57) (58) which concludes the proof.

3956

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 8, AUGUST 2007

According to Property 3, large errors are efficiently eliminated by a meridian estimator with a finite medianity parameter. Note that this is not the case for the median estimate in which large positive or negative errors can always shift the value of the smoother. In robust statistics, M-estimators satisfying the outlier rejection property are called redescending. It can be proven that a necessary and sufficient condition for an M-estimator to be . Note that this redescending is that condition holds for meridian and myriad estimators, whereas it does not for the mean and median. Property 4: (No undershoot/overshoot) The output of the meridian estimator is always bounded by (59) and . where The proof follows directly from the proposition 3 and thus is omitted for brevity. The shift and sign invariance property is essential in showing the unbiasedness of the meridian filtering. These properties are discussed next, following which the unbiasedness of the meridian filter is shown. . Then, Property 5: (Shift and sign invariance) Let for any , it follows that: 1) ; 2) . The proofs are trivial from the definition of the meridian. Property 6: (Unbiasedness) Given a set of samples that are independent and symmetrically distributed around the symmetry center , is also symmetrically distributed around . In particular, if exists, then . is symmetric around , then has the Proof: If has the same same distribution as . Thus, distribution as , which according to Property 5, is equal to . It follows that is symmetric around .

For a given nominal scale factor , the underlying random variables are assumed to be independent and Meridian distributed with a common location parameter , but varying scale , where factors (60) A larger value for the weight (smaller scale ) makes the distribution of more concentrated around , thus increasing the reliability of the sample . Note that the special case when all the weights are equal to unity corresponds to the sample myriad at the nominal scale factor, with all the scale factors reducing . The weighted meridian smoother output can now to be defined as the value that maximizes the likelihood function . independent samples Theorem 8: Given a set of , each obeying the Meridian distribution with , the ML estimate (possibly) varying scale parameters of location, or weighted meridian, is given by

(61) where denotes the weighting operation in the minimization problem. Proof: Replacing the meridian distribution in the likelihood function and performing manipulations yield (62) (63) Taking the natural gives

and utilizing basic properties of

C. Weighted Meridian Filtering The sample meridian can be generalized to the weighted meridian smoother by assigning nonnegative weights to the input samples (observations), where the weights can be interpreted as reflecting the varying levels of sample “reliability.” To this end, the observations are assumed to be drawn from independent Meridian random variables that are not identically distributed. It is important to realize that the location estimation problem being considered here is intimately related to the using a sliding problem of filtering a time-series , at time , can be interpreted window. The output as an estimate of location based on the input samples . Further, the aforementioned model of independent but not identically distributed samples can capture the temporal relationships usually present among the input samples. To see this, , as an estimate of location, typically note that the output , comrelies more on (give more weight to) the sample pared with samples that are further away in time. By assigning varying scale factors in modeling the input samples, leading to different weights, their varying reliabilities or temporal locations can be effectively accounted for.

(64) (65) (66) which completes the proof. The properties given for the meridian estimator can be easily extended to the weighted meridian estimator case. Similar to the unweighted case, for large values of , the weighted meridian is equivalent to the weighted median, whereas for small , the estimator acquires the form of a weighted mode. Property 7: (Median Property) Given a set of samples , and weights , the weighted meridian converges to the weighted median as . This is

(67)

AYSAL AND BARNER: MERIDIAN FILTERING FOR ROBUST SIGNAL PROCESSING

Following the steps of the proof for the unweighted version, one can easily see that (67) holds. The previous property provides a fundamental link between meridian and median filters. Accordingly, a weighted median weights, can be seen as a meridian filter filter with positive . Conversely, when tends to with the same weights and zero, the weighted meridian becomes what we will call weighted mode-meridian. Weighted mode-meridian filters maintain the same mode-like behavior of the unweighted mode-meridian, as stated in the following. Property 8: (Mode Property) Given a set of samples , and positive weights , the weighted meridian is equal to one of the most repeated values . Furthermore in the sample set as

3957

TABLE I M-ESTIMATOR AND M-SMOOTHER (WEIGHTED M-ESTIMATOR) OBJECTIVE FUNCTIONS AND OUTPUTS FOR VARIOUS FILTER FAMILIES

(68) where is the set of most repeated values and is the number is repeated in the sample set. of times a member of Proof: Similar steps of the proof of unweighted version, it is straightforward that for small values of (69)

Dividing by gives the desired result. Let us define the following: (70) From (70), we see that is small if is large (which is small means that is emphasized) or if are close to ). Since (which happens when many of the has to be the smallest over the set for the filter output to be , it is clearly seen that the filter favors input samples (having significant weights) that are clustered together. Large errors are efficiently eliminated by the weighted meridian estimate with a finite medianity parameter and positive finite weights. In addition, as in the meridian case, and . weighted meridian estimate is also bounded by The following discuss these properties. and let denote Property 9: (Outlier Rejection) Let a vector of positive and finite weights. Then

(71) Following the steps for the unweighted meridian case, one can easily obtain the desired result. Property 10: (No undershoot/overshoot) The output of the weighted meridian estimator with positive and finite weights, , is always bracketed by (72) where

and

.

The proof follows straight from the unweighted case, and thus is omitted here. The shift and sign invariance, and unbiasedness properties of the meridian estimator is extended to the weighted meridian case. Since the proofs are straightforward extensions of the meridian estimate, they are omitted. . Property 11: Shift and Sign Invariance. Let Then, for any and weight vector , it follows that: 1) ; 2) . Property 12: (Unbiasedness) Given a set of samples that are independent and symmetrically distributed around the symmetry center , is also symmetrically distributed around . In particular, if exists, then . Table I summarizes the M-estimators and M-smoothers for existing and proposed filter families discussed in the paper. The rows number (one, two) and (five, six) of the table corresponds to the generalized Gaussian statistics cases occurring for and . The objective function related to independent (not motivates (weighted) identical) Gaussian statistics mean filtering, whereas the objective function related to indemotivates pendent (not identical) Laplacian statistics (weighted) median filtering. Note that the RV formed as the ratio of two independent Gaussian distributed RVs is Cauchy distributed, where the Cauchy distribution is a special case of . The rows numthe generalized Cauchy distribution for bered three and seven correspond to this case, where the ML estimate under independent (not identical) Cauchy statistics motivates the (weighted) myriad filtering. In this paper, it is also shown that the RV formed as the ration of two independent Laplacian distribution is also a member of generalized . This distribution is Cauchy distribution family for referred to as Meridian distribution. The ML estimate under (possibly varying variances) Meridian statistics is determined, motivating the (weighted) meridian filtering. The row numbers four and eight consider these cases. The (weighted) meridian

3958

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 8, AUGUST 2007

filtering structure hence completes the missing link of the estimator/smoother quadruplet which is now composed of: (weighted) mean, (weighted) median, (weighted) myriad, and (weighted) meridian.

and [6], [9]

(78) IV. MERIDIAN FILTERS ADMITTING REAL-VALUED WEIGHTS Considered first is the sign coupling approach that allows to introduce real-valued weights to median- and myriad-type filters. The weighted meridian structure is then generalized into meridian filters admitting real-valued weights. Prior to extending the general class of M-smoothers, it is instructive to examine the special cases of the weighted mean, median, and myriad smoothers. Referring to Table I, the output of a weighted mean smoother, with input vector and a weight vector of non-negative weights, is given by (73) Notice from Table I that minimizes the objective function . When the weights are allowed to be both , a natural extension of (73) positive and negative is given by [8] (74) which is the weighted mean filter. Note that the normalization factor in (74) is the sum of the absolute weight values. We can rewrite (73) as [6], [8] (75) Thus, the sign of the weight is uncoupled from its magnitude and attached to the corresponding input sample. Comparing (75) with (73), we can rewrite (75) as [6] (76) and . The where weighted mean filter can, therefore, be thought of as a weighted mean smoother that applies the non-negative weights to the modified set of samples . Referring to the objective function for the weighted mean smoother in Table I, we can infer that the weighted mean filter output minimizes the . This operation objective function is referred to as sign coupling [6], [8]. Following the previous approach, the weighted median and myriad smoothers were recently extended to a class of weighted median and myriad filters with real-valued weights [8], [9]. Thus, referring to Table I, and by analogy to (75), the outputs of weighted median and myriad filters admitting real-valued weights are written as [6], [8]

Utilizing the sign coupling approach in a fashion similar to the mean, median and myriad filter cases, the weighted meridian smoother is generalized to the class of weighted meridian filters admitting real-valued weights. The output of the weighted meridian filter thus minimizes the following objective function:

(79) Since the weighted meridian filter allows for positive as well as negative weights, it overcomes the limitations of the weighted meridian smoother to the extent that it can accomplish a wide range of frequency-selective filtering operations, including bandpass and highpass filtering. V. APPLICATIONS OF MERIDIAN FILTERS The proposed meridian filtering is tested and evaluated through simulations. Addressed first is the baseband communication problem in channels corrupted with impulsive noise. The meridian filter is then utilized in a powerline communication application. The weighted meridian filter is finally evaluated in a frequency-selective application, the highpass filtering of a multitone signal. In each case, the proposed meridian filtering structure is compared to the linear, myriad and median filtering structures. The linearity and the medianity parameter of the myriad and meridian filters are set to unity throughout the simulations since unity linearity parameter is shown to be effective . in varying tail heaviness conditions [9], i.e., Recall that the meridian operator derives its optimality from , a distribution referred to the algebraic-tailed GCD for as the Meridian. Thus, in such environments, the meridian is the optimal estimator. Experiments validating this expected result have been carried out, but are not presented due to their expectation nature and space constraints. Rather, results are presented for the commonly utilized -stable density family, which is also algebraic-tailed. This provides a fairer comparison and illustrates the performance of the meridian filter operating on a widely used heavy-tailed distribution, but one that shares only tail order with the Meridian. The -stable RVs, like Gaussian RVs, are very appealing for the modelling of real-life phenomena because they constitute the only variables that can be described as the superposition of many small independent and identically distributed (i.i.d.) effects [3]. The class of symmetric -stable distributions is usually described by its characteristic function (80)

(77)

The parameter , which is commonly called the dispersion, is a positive constant related to the scale of the distribution and

AYSAL AND BARNER: MERIDIAN FILTERING FOR ROBUST SIGNAL PROCESSING

3959

TABLE II MATCHED LINEAR, MEDIAN, MYRIAD, AND MERIDIAN FILTERS OUTPUT MAES AND MSES

Fig. 5. Baseband communication model: s(t) denotes the combined impulse response of the transmitter and the channel, and s(t) is corrupted by additive white noise n(t).

is the scale parameter of the distribution. In order for (80) to define a characteristic function, the values of must be restricted to the interval (0,2]. Importantly, determines the impulsiveness, or tail heaviness, of the distribution, where smaller values of indicate increased levels of impulsiveness. The limit case corresponds to the zero-mean Gaussian distribution with variance . All other values of correspond to impulsive distributions with infinite variance and algebraic tail behavior [3]. Indeed, -stable RVs are successfully utilized to model impulsive environments across a wide array of applications [3]–[6], [8], [9], [12], [21], [22]. A. Baseband Communications Consider the baseband communication model given in Fig. 5 [23]. Suppose that (real) is to be communicated over the channel. Denoted as is the combined impulse response to be of the transmitter and channel, and take the pulse corrupted by additive white Meridian noise. The received pulse is then given by [23] (81) which after sampling at rate

corresponds to the sequence (82)

Taking the common case assumption that only for , the communications goal is to estimate using the sam. Note that for a fixed , is meridian ples . By the whiteness asdistributed centered around the are indepensumption, the random variables dent with identical meridian distributions. This implies that the ML estimate for is given by (83) Taking the natural log of (83) and rewriting the sum yields (84) from which it can be seen that the ML estimate is the weighted meridian of normalized received signal values, where pulse shape determines the weights. Thus, we define the matched meridian filter (85)

that is matched to the pulse shape , , as the value minimizing (84). To evaluate and compare the matched meridian filter to its linear, median [23], and myriad [15] counterparts, 10 000 parameGaussian distributed ters are generated, sent through the baseband communication , and filtered with matched channel, sampled with linear, median, myriad, and meridian filters to obtain the esti. The corrupting channel mates noise is -stable distributed with . The pulse carrying the symbol is taken to be rectangular. The mean absolute and squared errors of the matched filter outputs are tabulated in Table II. It is clear from the results that the matched meridian filter provides the best performance under both the mean absolute (MAE) and mean squared error (MSE) criterions. The performance improvement provided by the matched meridian filter especially stands out in the MSE case since this criteria is sensitive to outliers. B. Powerline Communications In recent years, the use of existing power lines for transmitting data and voice has been receiving interest [24]–[27]. The advantages of power line communications (PLCs) are obvious due to the ubiquity of power lines and power outlets. The potential of power lines to deliver broadband services, such as fast internet access, telephone, fax services, and home networking is emerging in new communications industry technology. However, there remains considerable challenges for PLCs, such as communications channels that are hampered by the presence of large amplitude noise superimposed on top of traditional white Gaussian noise. The overall interference is appropriately modeled as an algebraic tailed process, with -stable often chosen as the parent distribution [24]. The performance of the meridian estimator is evaluated and compared to that of mean, median, and myriad estimators in a PLC problem. Consider a set of voltage levels, unknown at the receiver.3 A signal randomly composed of these voltage levels, shown in Fig. 6(a), is transmitted through a power line. The observed signal is given in Fig. 6(b), where the noise is -stable distributed with . The observed powerline signal is processed with mean, median, myriad, and meridian estimators, each utilizing , the results of which are given in window length Fig. 6(c)–(f), respectively. As shown in the figure, the mean and median are vulnerable to impulsive noise. The myriad, on the other hand, provides better impulse suppression, albeit with a higher output fluctuation than the median. The figure also shows that the meridian provides the best impulsive noise suppression and the smallest output fluctuation. 3In the case where the signal alphabet is known at the receiver, the linear, median, myriad, and meridian estimators are readily extended to detectors.

3960

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 8, AUGUST 2007

Fig. 6. Powerline communications signal enhancement: (a) transmitted signal; (b) observed signal corrupted by  = 0:5 noise output of the; (c) mean; (d) median; (e) myriad; and (f) meridian estimators.

C. Highpass Filtering of a Multitone Signal As a final example, consider the problem of preserving a high-frequency tone while removing all low-frequency terms. Fig. 7(a) depicts a two-tone signal with normalized frequencies 0.02 and 0.4 Hz. The signal is 1000 samples long, although a cropped version of the original signal {0,200} is shown for presentation purposes. Fig. 7(b) shows the multitone signal filtered by a 40-tap linear FIR filter designed by the MATLAB fir1 command with a normalized cutoff frequency of 0.3 Hz. The myriad and median filter weights are optimized in this case utilizing the adaptive methods detailed in [8] and [9].4 Noting that the median filter is a limiting case of the meridian filter, the meridian filter weights are set equal to those of the median. The clean multitone input signal is then corrupted by additive stable noise , Fig. 7(c). The noisy multi-tone signal is processed with by the FIR, median, myriad and meridian filters with the results given in Fig. 7(d), (e), (f), and (g), respectively. Similarly to the previous cases, the meridian filter preserves the primary signal detail and yields the most robust performance. D. Comparison to Median Filtering In a final example, the meridian filtering is compared to its limiting case, median filtering, in varying -stable environments . The performance of the meridian estimator where 4The clean multitone signal and the desired high-frequency signal is utilized as the input and desired signals, respectively. For more detail see [8] and [9].

is evaluated and compared to that of median estimators in a PLC problem as defined in Section V-B for varying values (see Fig. 8). The results show that the meridian estimator outperforms the median estimator for low values. The performances of the meridian and median estimators, as expected converge together as tends to two. VI. CONCLUDING REMARKS AND FUTURE WORK It is noted here that the Gaussian and Cauchy distributions are statistically related. An analogous statistical relation is constructed here for the Laplacian distribution yielding the Meridian distribution. Noting that the Gaussian and Laplacian distributions are special cases of generalized Gaussian distriand , respectively, a similar butions family with relation is brought to light here as the Cauchy and Meridian distributions are, indeed, special cases of generalized Cauchy and , respectively. Thus, distributions family with connections between the generalized Gaussian and Cauchy distributions are constructed. Noting that the linear, myriad, and median filters are statistically related to the maximum likelihood estimate under Gaussian, Cauchy and Laplacian statistics, a novel meridian filtering structure is proposed based on the maximum likelihood estimate under Meridian statistics. The objective function of the meridian filter is statistically analyzed. The robustness of the meridian filter is shown through evaluation of the influence function. Several properties of the

AYSAL AND BARNER: MERIDIAN FILTERING FOR ROBUST SIGNAL PROCESSING

3961

Fig. 7. High frequency-selection filtering in impulsive noise: (a) clean two-tone input signal; (b) (desired) highpass component of the input signal (processed with lowpass FIR filter); (c) two-tone signal corrupted with stable noise ( = 0:4), noisy two-tone input signal processed with; (d) linear; (e) median; (f) myriad; and (g) meridian filters.

approach. The effectiveness of the proposed filter is demonstrated through simulations including matched filtering, powerline signal enhancement, and frequency-selective filtering in stable noisy environments. The statistical analysis, varying properties of the meridian filter and simulation results indicate that the meridian filter is a very promising filter structure that can effectively remove the impulsive noise and preserve the signal detail. Although the meridian estimator shows promise for processing of signals embedded in very impulsive noise, the appropriate filter, e.g., median, myriad, or meridian, with appropriate tuning parameters, depends on the application at hand. Current work focuses on the optimal weight and robustness parameter design utilizing stochastic gradient approaches. REFERENCES Fig. 8. Powerline communications signal enhancement in varying -stable environments with N = 9.

meridian filter essential to signal processing applications are given. The meridian filter is also extended to admit positive weights utilizing the maximum likelihood estimate under (possibly) varying Meridian statistics. The meridian filter is then extended to admit real-valued weights utilizing the sign coupling

[1] Huber, Robust Statistics. New York: Wiley, 1981. [2] S. A. Kassam and H. V. Poor, “Robust techniques for signal processing,” Proc. IEEE, vol. 73, no. 3, pp. 433–482, Mar. 1985. [3] C. L. Nikias and M. Shao, Signal Processing With Alpha-Stable Distributions and Applications. New York: Wiley, 1995. [4] J. Ilow, “Signal proceesing in alpha-stable noise environments: Noise modeling, detection and estimation,” Ph.D. dissertation, Elect. Comput. Eng. Dept., Univ. Toronto, Toronto, ON, Canada, 1995. [5] S. A. Kassam, Signal Detection in Non-Gaussian Noise. New York: Springer-Verlag, 1985.

3962

[6] G. R. Arce, Nonlinear Signal Processing: A Statistical Approach. New York: Wiley, 2005. [7] L. Yin, R. Yang, M. Gabbouj, and Y. Neuvo, “Weighted median filters: A tutorial,” IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process., vol. 43, no. 3, pp. 157–192, Mar. 1996. [8] G. R. Arce, “A general weighted median filter structure admitting negative weights,” IEEE Trans. Signal Process., vol. 46, no. 12, pp. 3195–3205, Dec. 1998. [9] S. Kalluri and G. R. Arce, “Robust frequency-selective filtering using weighted myriad filters admitting real-valued weights,” IEEE Trans. Signal Process., vol. 49, no. 11, pp. 2721–2733, Nov. 2001. [10] R. Yang, L. Yin, M. Gabbouj, J. Astola, and Y. Neuvo, “Optimal weighted median filters under structural constraints,” IEEE Trans. Signal Process., vol. 43, no. 3, pp. 591–604, Mar., 1995. [11] I. Shmulevich, O. Yli-Harja, J. Astola, and A. Korshunov, “On the robustness of the class of stack filters,” IEEE Trans. Signal Process., vol. 50, no. 7, pp. 1640–1649, Jul. 2002. [12] K. E. Barner and T. C. Aysal, “Polynomial weighted median filtering,” IEEE Trans. Signal Process., vol. 54, no. 2, pp. 636–650, Feb. 2006. [13] P. R. Rider, “Generalized cauchy distributions,” Annals Inst. Stat. Math., vol. 9, pp. 215–223, 1957. [14] J. H. Miller and J. B. Thomas, “Detectors for discrete-time signals in non-Gaussian noise,” IEEE Trans. Inf. Theory, vol. 18, no. 2, pp. 241–250, Mar. 1972. [15] J. G. Gonzalez, “Robust techniques for wireless communications in non-Gaussian environments,” Ph.D. dissertation, Elect. Comput. Eng. Dept., Univ. Delaware, Newark, 1997. [16] A. Papoulis, Probability, Random Variables, and Stochastic Processes. New York: McGraw-Hill, 1984. [17] S. Kalluri and G. R. Arce, “Adaptive weighted myriad filter algorithms for robust signal processing in -stable environments,” IEEE Trans. Signal Process., vol. 46, no. 2, pp. 322–334, Feb. 1998. [18] J. G. Gonzalez and G. R. Arce, “Optimality of the myriad filter in practical impulsive-noise environments,” IEEE Trans. Signal Process., vol. 49, no. 2, pp. 438–441, Feb. 2001. [19] H. A. David and H. N. Nagaraja, Order Statistics. New York: Wiley, 2003. [20] F. Hampel, E. Ronchetti, P. Rousseeuw, and W. Stahel, Robust Statistics: The Approach Based on Influence Functions. New York: Wiley, 1986. [21] E. Masry, “Alpha-stable signals and adaptive filtering,” IEEE Trans. Signal Process., vol. 48, no. 11, pp. 3011–3016, Nov. 2000. [22] T. C. Aysal and K. E. Barner, “Second-order heavy-tailed distributions and tail analysis,” IEEE Trans. Signal Process., vol. 54, no. 7, pp. 2827–2832, Jul. 2006. [23] J. Astola and Y. Neuvo, “Matched median filtering,” IEEE Trans. Commun., vol. 40, no. 4, pp. 722–729, Apr. 1992. [24] Y. H. Ma, P. L. So, and E. Gunawan, “Performance analysis of OFDM systems for broadband power line communications under impulsive noise and multipath effects,” IEEE Trans. Power Delivery, vol. 20, no. 2, pp. 674–682, Apr. 2005. [25] O. G. Hooijen, “A channel model for the residential power circuit used as a digital communications medium,” IEEE Trans. Electromagn. Compat., vol. 40, no. 4, pp. 331–336, Nov. 1998. [26] O. G. Hooijen, “On the channal capacity of the residential power circuit used as a digital communications medium,” IEEE Commun. Lett., vol. 2, no. 10, pp. 267–268, Oct. 1998. [27] M. Zimmerman and K. Dostert, “Analysis and modeling of impulsive noise in broadband power line communications,” IEEE Trans. Electromagn. Compat., vol. 44, no. 1, pp. 249–258, Feb. 2002.

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 8, AUGUST 2007

Tuncer Can Aysal (S’05) received the B.E. degree (high honors) from Istanbul Technical University, Istanbul, Turkey, in 2003, and the Ph.D. degree from the University of Delaware, Newark, in 2007, both in electrical and computer engineering. He is currently a Postdoctoral Research Fellow with the Electrical and Computer Engineering Department, McGill University, Montreal, QC, Canada. His research interests include distributed/ decentralized signal processing, sensor networks, consensus algorithms, and robust, nonlinear, statistical signal and image processing. Dr. Aysal was a recipient of the University of Delaware Competitive Graduate Student Fellowship, a Signal Processing and Communications Graduate Faculty Award (award is presented to an outstanding graduate student in this research area), and a University Dissertation Fellowship. He was also a Best Student Paper finalist at the International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2007. His Ph.D. dissertation was nominated by the Electrical and Computer Engineering Department for Allan P. Colburn Dissertation Prize in Mathematical Sciences and Engineering for the most outstanding doctoral dissertation in the mathematical and engineering disciplines.

Kenneth E. Barner (S’84–M’92–SM’00) was born in Montclair, NJ, on December 14, 1963. He received the B.S.E.E. degree (magna cum laude) from Lehigh University, Bethlehem, PA, in 1987, and the M.S.E.E. and Ph.D. degrees from the University of Delaware, Newark, in 1989 and 1992, respectively. He is currently a Professor in the Department of Electrical and Computer Engineering, University of Delaware. He was the duPont Teaching Fellow and a Visiting Lecturer at the University of Delaware in 1991 and 1992, respectively. From 1993 to 1997, he was an Assistant Research Professor in the Department of Electrical and Computer Engineering at the University of Delaware and a Research Engineer at the duPont Hospital for Children. His research interests include signal and image processing, robust signal processing, nonlinear systems, communications, haptic and tactile methods, and universal access. Dr. Barner was a recipient of a 1999 National Science Foundation CAREER Award. He was the co-chair of the 2001 IEEE-EURASIP Nonlinear Signal and Image Processing (NSIP) Workshop and a Guest Editor for a special issue of the EURASIP Journal of Applied Signal Processing on Nonlinear Signal and Image Processing. For his dissertation “Permutation Filters: A Group Theoretic Class of Non-Linear Filters,” he received the Allan P. Colburn Prize in Mathematical Sciences and Engineering for the most outstanding doctoral dissertation in the engineering and mathematical disciplines. He is a member of the Nonlinear Signal and Image Processing Board and is the co-editor of the book Nonlinear Signal and Image Processing: Theory, Methods, and Applications (CRC, 2003). He was the Technical Program Co-Chair for ICASSP’05. He is serving as an Associate Editor of the IEEE TRANSACTIONS ON SIGNAL PROCESSING, the IEEE Signal Processing Magazine, and the IEEE TRANSACTION ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING. He is a member of the Editorial Board of the EURASIP Journal of Applied Signal Processing. He is a member of Tau Beta Pi, Eta Kappa Nu, and Phi Sigma Kappa.