IEEE SIGNAL PROCESSING LETTERS, VOL. 21, NO. 11, NOVEMBER 2014
1385
Convex Combination of Adaptive Filters under the Maximum Correntropy Criterion in Impulsive Interference Liming Shi, Student Member, IEEE, and Yun Lin
Abstract—A robust adaptive filtering algorithm based on the convex combination of two adaptive filters under the maximum correntropy criterion (MCC) is proposed. Compared with conventional minimum mean square error (MSE) criterion-based adaptive filtering algorithm, the MCC-based algorithm shows a better robustness against impulsive interference. However, its major drawback is the conflicting requirements between convergence speed and steady-state mean square error. In this letter, we use the convex combination method to overcome the tradeoff problem. Instead of minimizing the squared error to update the mixing parameter in conventional convex combination scheme, the method of maximizing the correntropy is introduced to make the proposed algorithm more robust against impulsive interference. Additionally, we report a novel weight transfer method to further improve the tracking performance. The good performance in terms of convergence rate and steady-state mean square error is demonstrated in plant identification scenarios that include impulsive interference and abrupt changes. Index Terms—Adaptive filtering, convex combination, impulsive interference, maximum correntropy criterion, weight transfer.
I. INTRODUCTION
T
HE recently defined correntropy function can be viewed as a local similarity measure and has a close relationship with M-estimation [1]. These characteristics make the maximum correntropy criterion (MCC) more suitable for nonlinear, non-Gaussian signal processing than the minimum mean square error (MSE) criterion [2], [3], a known global similarity measure method. It is also shown in [4] that the maximum correntropy estimation is a smoothed maximum a posteriori (MAP) estimation. The MCC has recently been applied to adaptive filtering algorithm to improve the tracking performance in impulsive interference [5], [6], while MSE-based algorithms perform poorly [7]. The steady-state mean-square error of the MCC-based algorithm has been analyzed in [8]. Manuscript received June 05, 2014; accepted July 08, 2014. Date of current version July 16, 2014. This work was supported by Program for Changjiang Scholars and Innovative Research Team in University under Grant IRT1299) and by the special fund of the Chongqing Key Laboratory (CSTC). The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Sascha Spors. L. Shi and Y. Lin are with the Chongqing Key Lab of Mobile Communications Technology, Chongqing University of Posts and Telecommunications (CQUPT), Chongqing 400065, China (e-mail:
[email protected];
[email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/LSP.2014.2337899
The steady-state error of MCC-based algorithm is mainly controlled by the step-size, while its convergence rate by both the step-size and kernel width. Zhao etc. propose an adaptive update method to select the appropriate kernel width [9]. When the kernel width is chosen as a fixed value, a large step-size leads to a fast convergence rate, but a large mean-square error at the steady state. In contrast, algorithm with small step-size shows a small mean square error at the steady state, but a slow convergence rate. Therefore, it requires a tradeoff between fast convergence rate and small steady-state mean-square error. Recently, convex combination of adaptive filters enjoys a great popularity [10]–[12]. Its development is motivated by the idea of combing the good performance of different adaptive filters to offer complementary capabilities (such as a fast convergence speed filter and a small steady-state mean square error one). The mean square performance of the convex combination of the least mean square algorithms has been analyzed in [13]. In this Letter, we report a novel robust adaptive algorithm by convexly combining two MCC-based adaptive algorithms with different step-sizes to satisfy the conflicting requirements between convergence rate and the steady-state mean square error. Our objective is to derive a novel robust adaptive algorithm to outperform each of the separate filters by obtaining the fast convergence rate of the large step-size filter and the low steady state mean square error of the small step-size one. However, updating the mixing parameter by minimizing the squared error in conventional convex combination theory cannot be directly applied to the proposed algorithm, which will lead to performance degeneration in the presence of non-Gaussian impulsive interference. Therefore, the maximization of the correntropy method is introduced to offer a more robust solution. In addition, we present a weight transfer method to further improve the performance of the convex combination scheme. The rest of the paper is organized as follows. In Section II, we briefly review the MSE-based adaptive algorithm proposed by Widrow-Hoff [7] and the MCC-based adaptive algorithm presented by Singh and Principe [5], respectively. In Section III and IV, we present the convex combination of two MCC-based algorithms and its improved scheme, respectively. The simulation results are given in Section V, and the conclusion is drawn in Section VI. Notation: Boldface symbols in lowercase letters denote vectors. The notation is used for transpose, and for taking expectation.
1070-9908 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
1386
IEEE SIGNAL PROCESSING LETTERS, VOL. 21, NO. 11, NOVEMBER 2014
II. THE MSE-BASED AND THE MCC-BASED ADAPTIVE ALGORITHMS
III. PROPOSED CONVEX COMBINATION SCHEME AND ITS IMPROVED SCHEME
Consider the desired signal that arises from a plant , where is an unidentification model known plant to be identified, denotes the input vector and is the background noise plus impulsive noise, respectively. The error signal is defined , where is the estimate of at iteration as . The MSE, a known global similarity measure between two random variables, is defined as follows:
We propose to convexly combine two MCC-based adaptive algorithms using (7) with different step-sizes, a large one and a small one . Following the conventional idea of convex combination [10]–[12], the two adaptive filters with different stepsizes are adapted separately. The overall output is obtained by combining the output of the component filters with a so-called mixing parameter
(1) The cost function of the MSE-based adaptive algorithm can be denoted as follows: (2) A stochastic gradient method of the MSE-based algorithm, namely the well-known least mean square (LMS) adaptive algorithm can be expressed as
(8) denotes the mixing parameter, , , and and are defined as the weight vectors of the large step-size filter and the small one, respectively. According to (8), the overall filter output error can be expressed as where
(9) The overall filter weight vector can be considered as the form
(3)
(10)
where denotes the step-size. Although the LMS algorithm may perform well in Gaussian noise environments, it suffers severely performance degeneration in non-Gaussian impulsive interference [7]. The MCC-based adaptive algorithm has recently been proposed to offer a more robust solution in impulsive interference. The correntropy, a local similarity measure between two random variables, is defined as follows [1]:
As can be seen from (10), our objective is to make the mixing parameter as close to 1 as possible when algorithm starts, and make it as close to 0 as possible when algorithm begins to converge. Following [10]–[12], Instead of directly adapting , we update it by using a sigmoidal function as
(4)
is based on a stochastic negative graand the adaptation of dient method. Unfortunately, updating the in conventional combination method by minimizing the squared error is not robust against impulsive noise [10]. Here, we modify the adaptive rule by using the correntropy maximisation form with the stochastic positive gradient method, that is
where is a shift-invariant Mercer Kernel, and is the joint distribution function of . A widely used kernel is the Gaussian kernel:
(11)
(5) where , and is the kernel width. The cost function of the MCC-based algorithm can be expressed as follows [8]: (6) Using the stochastic gradient method, the MCC-based adaptive algorithm can be expressed as [5], [8]: (7) Although (7) is more robust against impulsive interference than (3), it still exists the performance trade-off problem like traditional adaptive algorithms. The appropriate selection of the kernel width can be found in [9]. When the kernel width is chosen as a fixed value, the tracking performance of (7) is mainly controlled by the step-size [8]. The conflicting requirement between fast convergence rate and small steady-state mean square error is the major drawback of the MCC-based adaptive algorithm.
(12) denotes the step-size. However, the adaptation will where be very slow when the value of is too close to 1 or 0. Following [10]–[12], we also restrict the range of in the symmetric interval . The convex combination scheme can be further improved by introducing the weight transfer idea, using the faster filter to speed up the slower one in some instants [14]. However, the estimation of the squared error method used in [14] cannot be applied to proposed algorithm, due to the presence of impulsive interference. We modify the weight transfer rule as follows:
(13)
SHI AND LIN: CONVEX COMBINATION OF ADAPTIVE FILTERS
Whenever obtained as
.
and
1387
are
(14) In (14), and denote the output errors of the fast and the slow filters at iteration , is a smoothing factor, and is a positive constant value. and can be viewed as the correntropy estimations of the two filters. According to (13) and the weight transfer condition, the weight transfer can be set to occur when the performance of the slower filter is significantly worse than the faster one. The selections of the factors and are simple. From our extensive experiments, and are good choices. Also note that when is set to 1, proposed algorithm degenerate to the original convex combination scheme without introducing any weight transfer.
Fig. 1. Performance of the CMCC with different smoothing factor ). SNR is set to 30 dB. impulsive interference (
under
IV. SIMULATION RESULTS IN PLANT IDENTIFICATION SCENARIOS The plant to be identified is randomly generated with 32 taps, and the adaptive filter is assumed to have the same number of taps. The input signals are set to Gaussian white noise with unit variant. The output signal-to-noise ratio (SNR) is set to 10, 20, or 30 dB for additive Gaussian white noise. The impulsive noise is generated as [15], where is a Bernoulli process with a probability of success . The probability of the occurrence of impulsive noise is selected as 0.05, 0.1, or 0.5, respectively. We set the power of to , is the power of the system output: . where is set to 4.5 for all the experiments. The tracking performance is evaluated by the normalized mean square deviation (NMSD): , where . We compare proposed convex combination of maximum correntropy criterion-based algorithms (CMCC) with maximum correntropy criterion-based algorithm (MCC) in scenarios that include impulsive noise and abrupt changes. The learning curves are obtained by the ensemble averages over 100 independent runs. The work first tests the performance of the CMCC with different , as shown in Fig. 1, where and , are set to 3 and is set to 0.1, respectively. As can be seen, the CMCC without introducing any weight transfer ( ) achieves both the fast convergence rate of the large step-size filter and the low steady-state mean square error of the small step-size one. Moreover, the reported weight transfer method is effective to improve the overall tracking performance. Furthermore, 0.8 is a good choice for the weight transfer smoothing factor . We will use this value for the rest of the experiments. The work also examines the performance of the CMCC and MCC with different levels of impulsiveness in Fig. 2. is selected as 0.05, 0.1, and 0.5. As shown in Fig. 2, the CMCC and MCC are robust against impulsive interference, and the CMCC outperforms the MCC algorithm in terms of the convergence rate and the steady-state mean square error in various levels of
Fig. 2. Performance of the CMCC and MCC in different levels of impulsiveare chosen as 0.05, 0.1, and 0.5. , , ness. The values of , and . SNR is set to 30 dB.
Fig. 3. Performance of the CMCC and MCC in different levels of SNR. The , values of SNR are chosen as 10, 20, and 30 dB, respectively. , , and . is set to 0.1.
impulsiveness. Similar conclusions can be drawn from Fig. 3, which shows the performance of the CMCC and MCC in different levels of SNR.
1388
IEEE SIGNAL PROCESSING LETTERS, VOL. 21, NO. 11, NOVEMBER 2014
to further improve the performance. Simulation results show that the proposed algorithm achieves a better performance (in terms of convergence rate and steady-state mean square error) as compared with the original single filter algorithm. Therefore, it can be potentially applied to many adaptive filter applications. REFERENCES
Fig. 4. Optimal weights are suddenly changed from to are chosen as 0.1. SNR is set to 30 dB. 5000. The values of
in iteration
The comparisons of the proposed CMCC with MCC in abrupt change scenario are shown in Fig. 4 Under impulsive interference with a fixed value of . Different step-size combinations and , and are used in the simulations. As can be seen, there is no losing of steady-state performance for the CMCC and MCC algorithms. However, both the CMCC and MCC algorithms with various step-sizes need a longer time to converge after the abrupt change, due to larger deviations from current weight vector (close to ) to the optimal weight vector than from initial state to the optimal weight vector . Moreover, the MCC algorithms with smaller step-sizes suffer from severe performance degeneration of the tracking speed. The proposed CMCC presents a better tracking performance. V. CONCLUSION In this letter, a convex combination scheme is utilized to improve the performance of the maximum correntropy criterionbased adaptive algorithm. In order to provide robustness against impulsive interference, the adaptive rule of the mixing factor is derived by maximising the correntropy with the stochastic positive gradient method, instead of the conventional minimisation of the squared error with the stochastic negative gradient method. Additionally, a weight transfer approach is introduced
[1] W. Liu, P. P. Pokharel, and J. C. Principe, “Correntropy: Properties, and applications in non-Gaussian signal processing,” IEEE Trans. Signal Process., vol. 55, no. 11, pp. 5286–5298, 2007. [2] R. He, W. S. Zheng, and B. G. Hu, “Maximum correntropy criterion for robust face recognition,” IEEE Trans. Patt. Anal. Mach. Intell., vol. 33, no. 8, pp. 1561–1576, 2011. [3] R. He, B. G. Hu, W. S. Zheng, and X. W. Kong, “Robust principal component analysis based on maximum correntropy criterion,” IEEE Trans. Image Process., vol. 20, no. 6, pp. 1485–1494, 2011. [4] B. Chen and J. C. Principe, “Maximum correntropy estimation is a smoothed MAP estimation,” IEEE Signal Process. Lett., vol. 19, no. 8, pp. 491–494, 2012. [5] A. Singh and J. C. Principe, “Using correntropy as a cost function in linear adaptive filters,” in Proc. Int. Joint Conf. Neural Networks (IJCNN), 2009, pp. 2950–2955. [6] S. Zhao, B. Chen, and J. C. Principe, “Kernel adaptive filtering with maximum correntropy criterion,” in Proc. Int. Joint Conf. Neural Networks (IJCNN), 2011, pp. 2012–2017. [7] A. H. Sayed, Fundamentals of Adaptive Filtering. Hoboken, NJ, USA: Wiley, 2003. [8] B. Chen, L. Xing, J. Liang, N. Zheng, and J. C. Principe, “Steady-state mean-square error analysis for adaptive filtering under the maximum correntropy criterion,” IEEE Signal Process. Lett., vol. 21, no. 7, pp. 880–884, 2014. [9] S. Zhao, B. Chen, and J. C. Principe, “An adaptive kernel width update for correntropy,” in Proc. Inter. Joint Conf. Neural Netw., 2012, pp. 1–5. [10] J. Arenas-García, A. R. Figueiras-Vidal, and A. H. Sayed, “Steady state performance of convex combinations of adaptive filters,” in Proc. ICASSP, Philadelphia, PA, USA, Mar. 2005. [11] L. M. Shi, Y. Lin, and X. Z. Xie, “Combination of affine projection sign algorithms for robust adaptive filtering in non-Gaussian impulsive interference,” Electron. Lett., vol. 50, no. 6, pp. 466–467, 2014. [12] M. Ferrer, A. Gonzalez, M. deDiego, and G. Pinero, “Convex combination filtered-x algorithms for active noise control systems,” IEEE Trans. Audio, Speech, Lang. Process., vol. 21, no. 1, pp. 156–167, 2013. [13] J. Arenas-Garcia, A. R. Figueiras-Vidal, and A. H. Sayed, “Mean-square performance of a convex combination of two adaptive filters,” IEEE Trans. Signal Process., vol. 54, no. 3, pp. 1078–1090, Mar. 2006. [14] J. Arenas-García, V. Gómez-Verdejo, and A. R. Figueiras-Vidal, “New algorithms for improved adaptive convex combination of LMS transversal filters,” IEEE Trans. Instrum. Meas., vol. 54, no. 6, pp. 2239–2249, Dec. 2005. [15] L. R. Vega, H. Rey, J. Benesty, and S. Tressens, “A new robust variable step-size NLMS algorithm,” IEEE Trans. Signal Process., vol. 56, no. 5, pp. 1878–1893, May 2008.
