Robust-COMET for Covariance Estimation in Convex Structures ...

Robust-COMET for Covariance Estimation in Convex Structures: Algorithm and Statistical Properties Bruno Meriaux, Chengfang Ren, Mohammed Nabil El Korso, Arnaud Breloy, Philippe Forster

To cite this version: Bruno Meriaux, Chengfang Ren, Mohammed Nabil El Korso, Arnaud Breloy, Philippe Forster. Robust-COMET for Covariance Estimation in Convex Structures: Algorithm and Statistical Properties. 2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP 2014), Dec 2017, Curaçao, Netherlands. .

HAL Id: hal-01651772 https://hal.archives-ouvertes.fr/hal-01651772 Submitted on 29 Nov 2017

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Robust-COMET for Covariance Estimation in Convex Structures: Algorithm and Statistical Properties Bruno M´eriaux∗ , Chengfang Ren∗ , Mohammed Nabil El Korso† , Arnaud Breloy† and Philippe Forster‡ ∗

†

SONDRA, CentraleSup´elec, 91192 Gif-sur-Yvette, France Universit´e Paris-Nanterre/LEME, 92410 Ville d’Avray, France ‡ SATIE, ENS Paris-Saclay, CNRS, 94230 Cachan, France

Abstract—This paper deals with structured covariance matrix estimation in a robust statistical framework. Covariance matrices often exhibit a particular structure related to the application of interest and taking this structure into account increases estimation accuracy. Within the framework of robust estimation, the class of circular Complex Elliptically Symmetric (CES) distributions is particularly interesting to handle impulsive and spiky data. Normalized CES random vectors are known to share a common Complex Angular Elliptical distribution. In this context, we propose a Robust Covariance Matrix Estimation Technique (RCOMET) based on Tyler’s estimate and COMET criterion for convexly structured matrices. We prove that the proposed estimator is consistent and asymptotically efficient while computationally attractive. Numerical results support the theoretical analysis in a particular application for Hermitian Toeplitz structure. Index Terms—Robust covariance estimation, elliptical distributions, Tyler’s M-estimator, structured covariance matrix.

I.

I NTRODUCTION

Covariance Matrix (CM) estimation plays a central role in adaptive signal processing. Besides the obvious Hermitian and positive characters, CM’s often exhibit a particular structure related to the application of interest: a well-known example is the Toeplitz structure for uniform linear arrays. Taking this structure into account increases estimation accuracy. In a Gaussian context, this problem has been widely investigated. In particular, an estimation procedure known as Covariance Matching Estimation Technique (COMET) [1] is an interesting alternative to Maximum Likelihood (ML) estimation: it allows one to handle easily linear structures and provides asymptotically efficient CM estimates. However, COMET is based on the Sample Covariance Matrix (SCM) estimate: therefore, it is neither robust to departures from Gaussianity nor to outliers. Within the framework of robust CM estimation, the class of circular Complex Elliptically Symmetric distributions (CES) has attracted much attention since the pioneering works of Maronna [2] and Tyler [3]. Indeed, CES distributions encompass a large number of non Gaussian distributions that are met in various applications such as radar or sonar [4]: Generalized Gaussian, compound Gaussian, t-distribution and K-distribution... Within this CES framework, Tyler proposed an unstructured distribution free estimate of the scatter matrix [3] which may be interpreted as the ML estimate of the covariance matrix of normalized CES data [5]. The latter random vectors are known to share a common Complex Angular Elliptical (CAE) distribution [5]. Furthermore estimation of CM’s with convex structures has been recently addressed for a CAE distribution [6, 7]. A COnvexly ConstrAined (COCA)

CM estimator has been recently proposed in [6] in which the proposed estimator is consistent but suffers from heavy computational cost. Moreover, COCA is not asymptotically efficient. Iterative Majorization-Minimization algorithms for the computation of structured CM estimates are developped in [7]. However, the latter approach may be computationally demanding (except for several very specific structures yielding closed form iterations). In order to fulfill this lack, we propose in this paper a Robust Covariance Matrix Estimation Technique (RCOMET) based on Tyler’s estimate and COMET criterion for convexly structured CM matrices. The proposed criterion is convex and its minimization admits a unique solution that can be efficiently computed (notably in a straightforward manner for linear structures). Our main contribution is to show that it yields consistent and asymptotically efficient CM estimates for CAE distributions. This paper is organized as follows. In section II, the data model is presented. Section III focuses on the proposed algorithm. The performance analysis is also treated. Section IV gives a particular application considering a Hermitian Toeplitz structure with some simulation results. Finally, a brief conclusion is given in Section V. d

In what follows, the notation = indicates ”has the same distribution as”. Convergence in distribution and in probability L P are respectively denoted by → and →. For a matrix A, |A| and Tr (A) denote the determinant and the trace of A, AT (respectively AH ) stands for the transpose (respectively conjugate transpose) matrix. The vec-operator vec (A) stacks all columns of A into a vector. The operator ⊗ refers to Kronecker matrix product. The subscript ”e” refers to the true value. The notations < and = denote the real and imaginary parts. II.

P ROBLEM SETUP

Let x ∈ Cm be a circular CES random vector [5] with scatter matrix R. If it exists, the covariance matrix of x is x proportional to R. The normalized vector y = , x 6= 0, kxk follows a CAE distribution, denoted by y ∼ U (R). The vector y has the following probability density function (p.d.f.) w.r.t. spherical measure which is the natural Borel measure on the unit complex sphere CSm , {z ∈ Cm | kzk = 1} [6]
−m p(y | R) ∝ |R|−1 yH R−1 y (1) where the shape matrix R is defined up to an arbitrary scale factor. To avoid scaling ambiguity, R is normalized according

to Tr (R) = m. It is worth noting that R is not the scaled covariance matrix of y unless R = I, though it is the scatter matrix of the underlying CES vector x: that is why we refer to R as the shape matrix of y. We assume that the latter matrix belongs to a convex subset S of Hermitian positive-definite matrices, and that there exists a one-to-one differentiable mapping µ 7→ R(µ) from RP to S . The vector µ is the unknown interest parameter with exact value µe , and Re = R(µe ) corresponds to the exact shape matrix. Considering N i.i.d. CAE distributed observations, yn ∼ U(Re ), n = 1, . . . , N , the log-likelihood function is given, up to an additive constant by L(y1 , . . . , yN ; µ) = −m

N X

  −1 log yH yn − N log |R(µ)| (2) n R(µ)

n=1

The above log-likelihood is a non-convex function of R. Therefore its maximization is a difficult and time consuming problem. To overcome this issue, we propose in the next section a new estimation procedure that provides unique, consistent and asymptotically efficient estimates. Moreover, closed form expressions of the estimates are easily obtained for linear structures of the shape matrix.

A. Tyler’s estimate: an overview The unstructured ML estimate of the shape matrix is known b T [5], it maximizes (2) over to be Tyler’s estimate R  the  set of b Hermitian positive matrices under constraint Tr RT = m. b T is the unique Assuming N > m, it is well known that R solution of the fixed point equation N   X yn yH n bT = m bT = m R , subject to Tr R −1 N n=1 yH R b y n n T

(4)

bT It is obtained by a simple algorithm which converges to R b T is also known to be a consistent, unbiased estimate [3, 8]. R of Re . Its asymptotic distribution is related to the Complex b W be a complex Wishart matrix Wishart distribution. Let R m with N degrees of freedom and parameter matrix Re . m+1   b √ m RT  − Re  and Then, both random matrices N   b Tr R−1 R T e   bW √ mR  − Re  converge to the same asymptotic N  −1 b Tr Re RW Gaussian distribution [8]. This result plays a central role in the proof of Theorem 2 at the next section.

III.

RCOMET: A ROBUST ASYMPTOTICALLY EFFICIENT COVARIANCE MATCHING ESTIMATE

b  0 be an unstructured shape matrix estimate based Let R on N CAE distributed observations. Consider the following function     −1   −1  b αR(µ) = Tr R b − αR(µ) R b b − αR(µ) R b d R, R ,

(3)

where α > 0 is needed for the purpose of theoretical derivations,though theshape matrix is parameterized as R(µ). Given b d R, b αR(µ) is a convex function of αR. Therefore, for R, R ∈ S convex set and α > 0, the minimization of (3) w.r.t. αR is a convex problem that admits a unique solution. In addition, the constraint on the trace matrix, avoiding scaling ambiguity, ensures the uniqueness of R(µ). Finally, the oneto-one mapping yields a unique solution for µ. Function (3) is a COMET type criterion that has been originally introduced in a Gaussian framework [1]. In that former b the SCM, the minimization Gaussian data context, taking for R   b αR(µ) w.r.t. α and µ yields an efficient estimate for of d R, the covariance matrix model αR(µ). In this paper, we address the study of criterion (3) in the context of CAE observations. In this non-Gaussian context, b T in (3) for R b we will show that taking Tyler’s estimate R ensures asymptotic statistical efficiency for CAE distribution:   b T , αR(µ) will be referred to as the RCOMET criterion. d R Properties of Tyler’s estimate are recalled in section III-A, and  b T , αR(µ) w.r.t. α we will show in III-B that minimizing d R and µ leads to an asymptotically efficient estimate of µe for b will be referred as the CAE observations. In the following, µ RCOMET estimate of µ.

B. Consistency and asymptotic efficiency of the RCOMET estimator This section provides a statistical analysis of the   b T , αR(µ) b which minimizes d R RCOMET estimator µ where minimization is carried out w.r.t. α > 0 and b is unique. µ = (µ1 , . . . , µP )T ∈ RP . As already noticed, µ b is a consistent esTheorem 1. The RCOMET estimator µ timator of µe . Likewise, R (b µ) is a consistent estimator of R(µe ) Proof: See Appendix A The following Lemma is needed for proving Theorem 2. Lemma 1. Let CRBCAE be the P × P Cram´er-Rao Bound (CRB) on µ for y ∼ U (R(µe )). Let CRBG be the CRB on µ for z ∼ CN (0, αe R(µe )) where αe , µe are both unknown. Then m+1 CRBCAE = CRBG m Proof: See Appendix B Remark: note that αe is not part of our model U (R(µe )) while it appears in the Gaussian model CN (0, αe R(µe )). b be the RCOMET estimator of µe based Theorem 2. Let µ b is asymptotically on N i.i.d. observations, yn ∼ U (R(µe )). µ Gaussian and efficient: √ L N (b µ − µe ) → N (0, CRBCAE ) Proof: See Appendix C

IV.

A PPLICATION AND NUMERICAL RESULTS

This section presents the RCOMET algorithm in the particular case of an Hermitian Toeplitz shape matrix. Simulation results are given in order to assess the statistical analysis and to compare performance with a state of the art algorithm.

A. Toeplitz structure

Our algorithm will be compared to the COCA covariance estimator proposed in [6], which consists in solving the following problem:

N

1 X

H minimize R − di yi yi

R∈S,di N i=1 F ( 1 H R  di yi yi , ∀i = 1, . . . N subject to m di > 0, ∀i = 1, . . . N

kE [b µ − µe ]k2

Let Re = R(µe ) ∈ Cm×m belong to S, the convex subset The standard semi-definite program solver, CVX, is used of Hermitian positive-definite matrices, with Toeplitz structure to compute this estimator. The algorithm proposed in [7, and trace equal to m. A natural parameterization is as follows: Algorithm 3] cannot be directly transposed to a complex  valued matrix and cannot be used here.   < (R2 ) 1 R2 · · · Rm Performance of our RCOMET estimator and COCA are  = (R2 )   ..     R∗ . . . . . . compared to the CRB via MSE(b µ) and Tr (CRBCAE ).  .  .   2   ∈ R2m−2 . R(µ) =   and µ =   .   ..  .. ..   . . R2   . 10−1 < (Rm ) RCOMET ∗ ∗ Rm · · · R2 1 = (Rm ) COCA 10−2   1 Let us introduce λ = α ∈ R2m−1 and R0 (λ) = αR(µ). 10−3 µ 2 Note that there exists a matrix, J ∈ Cm ×2m−1 , which relates 10−4 the vectorized matrix R0 (λ) to λ as
r0 (λ) = vec R0 (λ) = vec (αR(µ)) = Jλ 10−5 Consequently, the RCOMET criterion (3) reads

10−6 1 10

−1/2

2

b = arg min bT b −1/2 brT − W λ Jλ

W

T

102

103

Number of samples N

λ

Fig. 1: Bias simulation  b T and W bT = R b TT ⊗ R b T . The well known where brT = vec R analytical solution gives 

In this particular case, using lemma 1 the CRB on µ for a single observation is easily shown to be

H ! vec R−1 vec R−1 m e e −1 H H −1 K J We − JK CRBCAE = m+1 m where K is the identity matrix deprived of its first row.

B. Simulations For m = 8, the Toeplitz shape matrix is generated from its first row according to {Re }1,` = ρ|`−1| , ` = 1, . . . , m and ρ = 0.8 + 0.3i. We generate 1000 sets of N independent m-dimensional Compound-Gaussian distributed samples, xn ∼ τ CN (0, Re ), n = 1, . . . , N [5]. These data are xn , normalized, to obtain CAE distributed samples, yn = kxn k n = 1, . . . , N , thereby getting rid of the random texture τ which plays no role anymore in what follows.

Fig. 1 presents the Euclidean norm of the estimated bias for b based on 1000 runs for each N . As shown previously, our µ RCOMET algorithm is asymptotically unbiased with a higher rate than COCA. 5 Tr (CRBCAE ) COCA RCOMET

0 Tr {MSE (b µ)} (dB)

h i  b    λ α b = −1 H b −1 1 b = JH W b −1 b r thus W J J λ T T T 1 hbi  µ b= λ α b 2:2m−1

−5 −10 −15 −20 −25 −30 101

102 Number of samples N

Fig. 2: Efficiency simulation

103

The asymptotic efficiency of RCOMET is checked on Fig. 2: it gets closer and closer to the CRB as N increases, unlike COCA estimator. However, we can note that COCA performs better at small number of samples, probably due to a bias in finite samples [6]. 102

RCOMET COCA

Time (s)

101

A PPENDIX B P ROOF OF L EMMA 1 For the Gaussian problem, the Slepian-Bang Formula can be directly applied for the Fisher Information Matrix (FIM):
−1 [Fµµ ]k,` = Tr R−1 ∀ k, ` = 1, . . . , P e ∂k RRe ∂` R
m 1 [Fµα ]k = Tr R−1 e ∂k R and Fαα = 2 α α The Schur complement gives us the CRB on µ ∀ k, ` = 1, . . . , P

100



−1  
Tr R−1 e ∂k R Tr Re ∂` R −1 −1 CRB−1 = Tr R ∂ RR ∂ R − k ` e e G k,` m

10−1 10

For the CAE problem, by following the same methodology as in [9], we obtain

−2

101

102

103

Number of samples N

Fig. 3: Average calculation time

The average computing times are reported in Fig. 3. As pointed out in [7], the number of constraints grows linearly in N for the COCA algorithm which becomes computationally prohibitively expensive when N increases: this is not the case for our proposed RCOMET.

V.

C ONCLUSION

In this paper, we addressed robust structured covariance estimation for convex structures. We proposed a robust extension of the classical Gaussian COMET for CAE distributed observations. The proposed RCOMET method is consistent, asymptotically unbiased and efficient. Numerical simulations confirm the theoretical analysis and the practical interest of this approach.

A PPENDIX A P ROOF OF T HEOREM 1 b)  α, µ be the unique values ofα and µ which  Let (b  minimize b b b ) = arg min d RT , αR(µ) . d RT , αR(µ) : (b α, µ α,µ

b T that we denote by: b is a function of R µ   bT b=g R µ Function g(·) satisfies g (Re ) = µe since d (Re , R(µe )) = 0. Moreover, for a smooth parameterization R(µ), g(·) is differb T [8] entiable and thus continuous. Then, the consistency of R  P b T → g (Re ) = µe . b =g R and the continuity of g imply µ P

Consequently, R(b µ) → R(µe ).




−1 −1 −1  mTr R−1 e ∂k RRe ∂` R − Tr Re ∂k R Tr Re ∂` R −1  CRBCAE k,` = m+1 m  −1  CRBG k,` ∀ k, ` = 1, . . . , P = m+1

Therefore CRBCAE =

m+1 CRBG which proves the lemma. m

A PPENDIX C P ROOF OF T HEOREM 2 b is independent of the scaling facFirst, remark that µ  b T . Indeed, if (b b T , αR(µ) b ) = arg min d R tor on R α, µ α,µ   b T , αR(µ) b ) = arg min d λR then (λb α, µ ∀ λ ∈ R∗ . α,µ   b T is thus an homogeneous function of degree 0: b = g R µ     bT = g R b T ∀ λ ∈ R∗ . g λR m b W be a complex Wishart matrix with N Let R m+1 degrees matrix    of freedom and parameter  Re . It is known that b b √ √ m RT m RW  − Re  and N    − Re  N  −1 b b Tr Re RT Tr R−1 R W e converge to the same asymptotic Gaussian distribution [8]. It follows from the Delta method [10,  Chapter 3] and √ b T − g (Re ) and the homogeneity of g(·) that N g R   √   b W − g (Re ) , where g (Re ) = µe , converge N g R also  to the same asymptotic Gaussian distribution. But bW b W is COMET’s estimate of µe based g R = µ m on K = N independant complex Gaussian samples m+1 b W is an asympCN (0, αe R(µe )), and it is known [1] that µ totically Gaussian efficient estimator. Therefore, √

√ N (b µW

L

K (b µW − µe ) → N (0, CRBG )   m+1 L − µe ) → N 0, CRBG = N (0, CRBCAE ) m √ L N (b µ − µe ) → N (0, CRBCAE )

R EFERENCES [1]

[2] [3] [4]

[5]

[6]

[7]

[8]

[9]

[10]

B. Ottersten, P. Stoica, and R. Roy, “Covariance matching estimation techniques for array signal processing applications,” Digital Signal Processing, vol. 8, no. 3, pp. 185–210, 1998. R. A. Maronna, “Robust m-estimators of multivariate location and scatter,” Ann. Statist., vol. 4, no. 1, pp. 51–67, 01 1976. D. E. Tyler, “A distribution-free m-estimator of multivariate scatter,” The Annals of Statistics, pp. 234–251, 1987. K. J. Sangston, F. Gini, and M. S. Greco, “Adaptive detection of radar targets in compound-gaussian clutter,” in Radar Conference (RadarCon), 2015 IEEE. IEEE, 2015, pp. 0587–0592. E. Ollila, D. E. Tyler, V. Koivunen, and H. V. Poor, “Complex elliptically symmetric distributions: Survey, new results and applications,” IEEE Transactions on signal processing, vol. 60, no. 11, pp. 5597–5625, 2012. I. Soloveychik and A. Wiesel, “Tyler’s covariance matrix estimator in elliptical models with convex structure,” IEEE Transactions on Signal Processing, vol. 62, no. 20, pp. 5251–5259, 2014. Y. Sun, P. Babu, and D. P. Palomar, “Robust estimation of structured covariance matrix for heavy-tailed elliptical distributions,” IEEE Transactions on Signal Processing, vol. 64, no. 14, pp. 3576–3590, 2016. F. Pascal, P. Forster, J.-P. Ovarlez, and P. Larzabal, “Performance analysis of covariance matrix estimates in impulsive noise,” IEEE Transactions on Signal Processing, vol. 56, no. 6, pp. 2206–2217, 2008. O. Besson and Y. I. Abramovich, “On the fisher information matrix for multivariate elliptically contoured distributions,” IEEE Signal Processing Letters, vol. 20, no. 11, pp. 1130–1133, 2013. A. W. Van der Vaart, Asymptotic statistics. Cambridge university press, 2000, vol. 3.