Asymptotic Eigenvalue Distribution of Wishart Matrices whose Components are not Independently and Identically Distributed Takashi Shinzato∗
arXiv:1605.06840v1 [q-fin.PM] 22 May 2016
Mori Arinori Center for Higher Education and Global Mobility, Hitotsubashi University, Tokyo, 1868601, Japan. (Dated: May 24, 2016) In the present work, eigenvalue distributions defined by a random rectangular matrix whose components are neither independently nor identically distributed are analyzed using replica analysis and belief propagation. In particular, we consider the case in which the components are independently but not identically distributed; for example, only the components in each row or in each column may be identically distributed. We also consider the more general case in which the components are correlated with one another. We use the replica approach while making only weak assumptions in order to determine the asymptotic eigenvalue distribution and to derive an algorithm for doing so, based on belief propagation. One of our findings supports the results obtained from Feynman diagrams. We present the results of several numerical experiments that validate our proposed methods. PACS number(s): 89.90.+n, 75.10.Nr, 89.65.Gh, 02.50.-r
I.
INTRODUCTION
Random matrices, in which each component is regarded as a random variable, are widely used and investigated, both theoretically and practically, in many fields of research, including number theory, combinatorial theory, nuclear physics, condensed matter physics, bionomics, mathematical finance, and communication theory [1–5]. In particular, the mathematical structure of random square matrices has been investigated; topics of investigation include the eigenvalue distribution and distribution of the level spacings of a Gaussian unitary ensemble (GUE) characterized by a Hermitian random matrix, and those of a Gaussian orthogonal ensemble (GOE) characterized by an orthogonal random matrix. For random rectangular matrices, topics of investigation have included singular values and the asymptotic eigenvalue distribution of a Wishart matrix that is defined by an autocovariance matrix [6–12]. For instance, Marˇcenko and Pastur consider the asymptotic eigenvalue distribution when each entry of a given random rectangular matrix is independently and identically drawn from a population with a probability distribution with mean 0 and variance 1/N , and N × N autocovariance matrix, and the eigenvalue distribution of the autocovariance matrix is sufficiently close to the asymptotic distribution when it is sufficiently large; this is known as the Marˇcenko– Pastur law [6]. Silverstein and Choi used the Stieltjes transformation to rederive the asymptotic eigenvalue distribution for the Marˇcenko–Pastur law [7, 8]. Sengupta and Mitra expanded the resolvent of a random matrix in which the components are correlated with one another, using the inverse of the matrix size, and they used Feynman diagrams to derive the fixed-point equations that would determine the asymptotic eigenvalue distribution [9]. Burda, G¨orlich, Jarsoz, and Jukiewicz derived the
∗
[email protected]
relationship between the asymptotic eigenvalue distribution of a correlation matrix, that was obtained using Feynman diagrams and an eigenvalue distribution estimated from a practical dataset [10]. In addition, Burda, Jurkiewicz, and Waclaw successfully derived the relation between those estimates by considering the resolvent and a moment-generating function [11]. Recher, Kieburg, and Guhr used supermatrix theory to assess the eigenvalue distribution of small, random matrices in which the components were correlated, and they compared the theoretical results with those obtained from numerical experiments [12]. As discussed above, there have been many studies that use Feynman diagrams or supermatrix theory to evaluate the asymptotic eigenvalue distribution defined by a random matrix ensemble, but few studies have used replica analysis or belief propagation to investigate the asymptotic eigenvalue distribution of a Wishart matrix in which the components are independently but not identically distributed, or in which they are correlated with each other. It has been assumed that the resolvent can be expanded to the inverse of the matrix size and that the ensemble average of each term is independent; in addition, it has been implicitly assumed that, in the Feynman diagram approach, a recursive relation with respect to the irreducible self-energy is a primary part of the resolvent. Moreover, since it is necessary to compute the inverse matrix in order to use the Feynman diagram approach, the required computational time is excessive. We note that the portfolio optimization problem is widely considered to be one of the most important applications of random matrix theory. If we consider an investment market in which the variance of the return rate of assets is not identical, then we need to use a random matrix ensemble in which the components are not identically distributed [13–19]. Furthermore, since we can use the asymptotic eigenvalue distribution defined by the Wishart matrix to evaluate the typical behavior of two quantities that characterize the optimal portfolio (defined as the portfolio that minimizes the investment
2 risk), in order to solve the portfolio optimization problem, we need to systematically examine the asymptotic eigenvalue distribution of nonidentically distributed random matrices. Thus, in the present paper, our goal is to determine the asymptotic eigenvalue distribution of a random matrix ensemble in which the entries are independently but not identically distributed or in which they are correlated with one another; we will do this using replica analysis, since it does not require the computation of the inverse matrix, and belief propagation, which does not require second-order statistics. We verify the effectiveness of our proposed method by presenting the results of several numerical experiments. This paper is organized as follows. In Section II, we consider the relationship between Green’s function and the eigenvalue distribution; we do this in order to analytically derive the asymptotic eigenvalue distribution and to explain the approach used in various previous studies. In Section III, we develop a methodology based on replica analysis in order to evaluate the asymptotic eigenvalue distribution of a random matrix ensemble in which the entries are neither independently nor identically distributed. In Section IV, in a similar way, we derive an algorithm based on belief propagation. In Section V, we present the results of numerical simulations that show the consistency and accuracy of our proposed methods. Section VI is devoted to a summary of our findings and a discussion of areas for future work.
II.
A.
EIGENVALUE DISTRIBUTIONS AND GREEN’S FUNCTIONS
Asymptotic eigenvalue distribution and Green’s functions
In this subsection, as a preparation for deriving the asymptotic eigenvalue distribution, we discuss the relationship between the eigenvalue distribution of a Wishart matrix defined by a random rectangular matrix and Green’s functions. Similar to the discussion of Wishart et al. [1, 20, n 21],o we consider a random rectangular max trix, X = √iµ ∈ RN ×p , (i = 1, · · · , N, µ = 1, · · · , p). N For simplicity, we will assume that by random matrix we mean a random rectangular matrix; we will assume that the expectation of each entry of a random matrix is√(or is normalized to be) 0; and we will assume that 1/ N is a scaling coefficient determined by the maximum or minimum eigenvalue of the variance-covariance matrix, which is a random matrix (Wishart matrix) in which α = p/N ∼ O(1). From these settings, the eigenvalue distribution of the Wishart matrix XX T ∈ RN ×N , ρ(λ|X), can be written using the N eigenvalues λ1 , · · · , λN as follows: ρ(λ|X) =
N 1 X δ(λ − λk ), N k=1
(1)
where δ(x) is the Dirac delta function, and a superscript T indicates the transposition of a vector or matrix. In addition, by using the trace operator, Tr, Eq. (1) can be rewritten as ρ(λ|X) = N1 Trδ λIN − XX T , where 1 (Y − iεIN )−1 − (Y + iεIN )−1 , δ(Y ) = limε→+0 2πi Y ∈ RN ×N ; and IN is the identity matrix, i.e., IN = diag{1, 1, · · · , 1} ∈ RN ×N (hereafter, Im ∈ Rm×m will be used to denote the m-dimensional identity matrix). Next, in order to derive the eigenvalue distribution, we define two kinds of Green’s function (or resolvent), as follows: −1 1 GR (λ|X) = lim Tr (λ + iε)IN − XX T , (2) ε→+0 N −1 1 GA (λ|X) = lim Tr (λ − iε)IN − XX T , (3) ε→+0 N where GR (λ|X) is the retarded Green’s function, and GA (λ|X) is the advanced Green’s function. From these definitions, we can have the following relations for the real and imaginary parts of these Green’s functions: ReGR (λ|X) = ReGA (λ|X), and ImGR (λ|X) = −ImGA (λ|X) From this, the eigenvalue distribution of a Wishart matrix XX T ∈ RN ×N , ρ(λ|X), can be rewritten using GR (λ|X) and GA (λ|X) as follows: 1 GR (λ|X) − GA (λ|X) 2πi 1 = − ImGR (λ|X). π
ρ(λ|X) = −
(4)
From Eq. (4), it can be seen that if we could analytically assess the retarded Green’s function GR (λ|X), then we could derive the eigenvalue distribution ρ(λ|X) from its imaginary part. Finally, when N is sufficiently large, the asymptotic eigenvalue distribution ρ(λ) is said to be self-averaging, that is, ρ(λ|X) = EX [ρ(λ|X)], EX [f (X)] means the expectation of f (X) on random variables X. Thus, we will not analyze the eigenvalue distribution of a Wishart matrix ρ(λ|X), but instead, we will determine its asymptotic eigenvalue distribution, ρ(λ) = EX [ρ(λ|X)]. B.
Previous studies
We now present some findings obtained in previous studies for the asymptotic eigenvalue distribution. Several previous studies have considered the case in which each entry, xiµ , of a random matrix is independently and identically distributed. For example, when the distribution of the entries has mean 0 and variance 1, and α = p/N , the asymptotic eigenvalue distribution (for large N ) converges to the Marˇcenko-Pastur law, as follows: p [λ+ − λ]+ [λ − λ− ]+ + ρ(λ) = [1 − α] δ(λ) + , (5) 2πλ √ where λ± = (1 ± α)2 , and [u]+ = max(u, 0) [6].
3 In a more general case, which will be discussed in detail below, if the asymptotic eigenvalue distribution of a random matrix ensemble has EX [xiµ xjν ] = mij θµν , the asymptotic eigenvalue distribution can be derived by expanding the generating function in terms of the characteristic parameters. From this property, previous studies have determined the asymptotic eigenvalue distribution by using recursive relations with respect to the 1/N expansion of the generating functions based on Feynman diagrams [10]. There are two key properties: (1) Green’s functions are related to self-energy, and (2) self-energy can be divided into irreducible self-energy; from these properties, we can obtain simultaneous equations for the order parameters, as follows [9]: −1 1 , (6) TrQt Qw = (λ + iε)IN + M N Qs = M Qw , (7) −1 1 , (8) Qu = Θ TrQs − Ip N Qt = ΘQu , (9) where EX [xiµ xjν ] = mij θµν , M = {mij } ∈ RN ×N , and Θ = {θµν } ∈ Rp×p are replaced by the N -dimensional matrices Qw , Qs and p-dimensional matrices Qu , Qt . We can solve these simultaneous equations to obtain the asymptotic eigenvalue distribution: 1 1 ρ(λ) = − Im lim TrQw . (10) ε→+0 N π Comparing this with the Feynman diagram method, we note that Qw , Qu correspond to Green’s functions and Qs , Qt correspond to irreducible self-energy. In order to simplify the calculation of Eq. (6) to Eq. (9), we rewrite them as follows: 1 χw = TrQw , (11) N 1 χs = TrQs , (12) N 1 χu = TrQu , (13) p 1 χt = TrQt . (14) p Using the novel order parameters χw , χs , χu , χt ∈ C, we can write the following simultaneous equations: 1 −1 (15) χw = Tr ((λ + iε)IN + αχt M ) , N 1 − (λ + iε)χw χs = , (16) αχt 1 −1 (17) χu = Tr (χs Θ − Ip ) , p 1 + χu χt = . (18) χs Then, from χw , we have 1 (19) ρ(λ) = − Im lim χw , ε→+0 π
that is, the limit of one of these parameters gives the asymptotic eigenvalue distribution. These newly defined parameters allow us to solve the simultaneous equations comparatively easily, compared to solving the original matrix formulation. However, it is still necessary to calculate the inverse matrix in Eq. (15) and Eq. (17), and thus it is difficult to implement this approach and to calculate the inverse matrices in Eq. (6) to Eq. (9) when N, p are large [9, 10]. It is not sufficient to discuss the adequacy of the assumption that we can expand Qw , Qs , Qu , Qt over 1/N , based on the Feynman diagram method. Thus, we will use replica analysis for a quenched ordered system in order to directly solve the asymptotic eigenvalue distribution of a random matrix ensemble; this approach will not require the calculation of an inverse matrix, and it validates the adequacy of their approaches [9, 10]. As an alternative approach, we propose an algorithm based on the belief propagation method; this approach allows us to determine the eigenvalue distribution without the need to calculate an inverse matrix, when N, p are sufficiently large but not infinite.
III.
REPLICA ANALYSIS A.
Replica trick
We now discuss the use of replica analysis to solve the asymptotic eigenvalue distribution ρ(λ); this is done in a way to similar to that presented in previous studies [22–25]. We can rewrite the retarded Green’s function as GR (λ|X) = −2 lim
ε→+0
∂φ(λ + iε|X) , ∂λ
(20)
where the partition function Z(λ + iε|X) and the generating function φ(λ + iε|X) are defined as follows: − 1 Z(λ + iε|X) = det (λ + iε)IN − XX T 2 , 1 log Z(λ + iε|X). φ(λ + iε|X) = N
(21) (22)
From Eq. (4) and Eq. (20), the eigenvalue distribution can be derived as ρ(λ|X) =
2 ∂φ(λ + iε|X) Im lim . ε→+0 π ∂λ
(23)
Moreover, since its asymptotic eigenvalue distribution ρ(λ) is evaluated as ρ(λ) = EX [ρ(λ|X)] ∂ 2 = Im lim EX [φ(λ + iε|X)], ε→+0 ∂λ π
(24)
in order to implement Eq. (24), we need to assess EX [φ(λ + iε|X)] =
1 EX [log Z(λ + iε|X)]. N
(25)
4 That is, we need to average the generating function φ(λ+ iε|X) over all configurations of the random matrix X. We note that, in general, it is more difficult to assess the configurational average of the logarithm of the partition function. Thus, we can use an identity function known n as the replica trick, log Z = limn→0 Z n−1 , and we obtain EX [Z n (λ + iε|X)] − 1 ; (26) n→0 n from this, we can compute the configurational average of the logarithm of the partition function EX [log Z(λ + iε|X)] from the configurational average of a power function of the partition function EX [Z n (λ + iε|X)]. Moreover, using l’Hopital’s rule with respect to Eq. (26), the replica trick can be rewritten as ∂ EX [log Z(λ + iε|X)] = lim log EX [Z n (λ + iε|X)], n→0 ∂n (27) EX [log Z(λ + iε|X)] = lim
where, from the definition in Eq. (21), the partition function is Z ∞ dw ~ − 21 w~ T ((λ+iε)IN −XX T )w~ . (28) Z(λ + iε|X) = N e −∞ (2π) 2 Furthermore, when the power n is a natural number, one can expand the power function of the partition function in order to assess the configurational average comparatively easily: EX [Z n (λ + iε|X)] "Z # n ∞ Y dw ~ a − 12 Pna=1 w~ aT ((λ+iε)IN −XX T )w~ a = EX . Nn e −∞ a=1 (2π) 2
(29)
Thus, we can (comparatively) easily evaluate EX [Z n (λ + iε|X)] for n ∈ N with respect to each of the three statistical properties of each component of the random matrix, and thus, we can determine the asymptotic eigenvalue distribution.
and the conjugate order parameters: q˜wab , q˜sab , (a, b = 1, 2, · · · , n). In this setting, using n-dimensional square ˜w, Q ˜ s ∈ Cn×n whose components are matrices Qw , Qs , Q the order parameters qwab , qsab , q˜wab , q˜sab , we have 1 log EX [Z n (λ + iε|X)] N α 1 ˜w − log det |In − Qs | + TrQw Q = Extr ˜ w ,Qs ,Q ˜s 2 2 Qw ,Q D E 1 ˜ w + sQ ˜ s| ˜ s − 1 log det |(λ + iε)In + Q + TrQs Q , 2 2 s (34) lim
N →∞
where α = p/N ∼ O(1) and N 1 X f (si ). N →∞ N i=1
(35)
hf (s)is = lim
Furthermore, we use the notation that ExtrΛ {g(Λ)} means the extremum with respect to Λ. Note that using the saddle-point method to evaluate this by expanding the order parameters is comparatively tight for sufficiently large N . From the extremum of the order parameters, we obtain −1 ˜ ˜ Qw = (λ + iε)In + Qw + sQs , (36) s −1 ˜ w + sQ ˜s , (37) Qs = s (λ + iε)In + Q s
˜ w = 0, Q ˜ s = α(Qs − In )−1 . Q
(38)
(39)
If we substitute Eq. (38) and Eq. (39) into Eq. (36) and Eq. (37), then we have D −1 E Qw = (λ + iε)In + αs(Qs − In )−1 , (40) s E D −1 . (41) Qs = s (λ + iε)In + αs(Qs − In )−1 s
B.
Independent but not identically distributed; case 1
We consider the case in which each entry, xiµ , of the random matrix is distributed such that the probability has covariance EX [xiµ xjν ] = si δij δµν , and the higherorder moments are finite. That is, from EX [xiµ xjν ] = si δij δµν , we have M = diag {s1 , s2 , · · · , sN } ∈ RN ×N , Θ = Ip ∈ Rp×p .
(30) (31)
We prepare the order parameters: qwab = qsab =
N 1 X wia wib , N i=1
N 1 X wia wib si , N i=1
(32)
(33)
From this solution, we assume the following replicasymmetric solution: Qw = χw In + qw Dn , Qs = χs In + qs Dn ,
(42) (43)
where In is the identity matrix, and Dn ∈ Rn×n is an n-dimensional square matrix in which each of the entries is unity. From this, we obtain * + 1 χw = , (44) λ + iε + χsαs −1 s 1 , (45) qw = αqs c s * + s χs = , (46) λ + iε + χsαs −1 s DsE , (47) qs = αqs c s
5 where c = 1s ((λ+iε)(χs −1)+αs)((λ+iε)(χs −1+nqs )+ αs). Next, from Eq. (45) and Eq. (47), we have qw = 0, qs = 0.
(30) and Eq. (31) into Eq. (6) and Eq. (14) to obtain χw =
(48) (49)
χs = That is, the off-diagonal elements of Qw and Qs are estimated to be 0. From Eq. (44) and Eq. (46), since χw and χs do not depend on n, Eq. (44) and Eq. (46) hold for any n. Thus, by using χs in Eq. (46), we can analytically assess χw in Eq. (44). Then, if we substitute qw = qs = 0 into Eq. (34), we have 1 log EX [Z n (λ + iε|X)] N nα χs nα = − log(1 − χs ) 2 χs − 1 2 n αs − log λ + iε + . 2 χs − 1 s
N 1 X 1 , N i=1 λ + iε + αχt si N si 1 X , N i=1 λ + iε + αχt si
1 , χs − 1 1 . χt = χs − 1
χu =
(53)
(54) (55) (56)
In the limit of large N , the results obtained by our proposed replica approach are consistent with those obtained in [10].
lim
N →∞
C.
(50)
Independent but not identically distributed; case 2
We now consider the case in which the covariance is EX [xiµ xjν ] = tµ δij δµν , that is, we set
From this, we have the asymptotic limit 1 log EX [Z n (λ + iε|X)] N n = lim log (EX [Z(λ + iε|X)]) . N →∞ N
M = IN ∈ RN ×N , Θ = diag {t1 , · · · , tp } ∈ Rp×p ,
lim
N →∞
(51)
(57) (58)
and then proceed in a way similar to what we did in the previous subsection. That is, we begin by obtaining 1 log EX [Z n (λ + iε|X)] N →∞ N 1 ˜w| − log det |(λ + iε)In + Q = Extr ˜ ˜ 2 Qw ,Qw ,Qt ,Qt 1 ˜ w − α TrQt Qw + α TrQt Q ˜t + TrQw Q 2 D 2 2 E o α ˜ t| − log det |In − tQ , (59) 2 t lim
In a previous study [15], it was found that this result implies that the distribution of the partition function is concentrated on a single point, the expectation of the partition function. That is, roughly speaking, in the thermodynamic limit, for an arbitrary function of the partition function, f (Z(λ + iε|X)), EX [f (Z(λ + iε|X))] = f (EX [Z(λ + iε|X)]) holds asymptotically. From this, we can assess the configurational average of the generating function, as follows: 1 log EX [Z(λ + iε|X)] EX [φ(λ + iε|X)] = lim N →∞ N α χs α = − log(1 − χs ) 2 χs − 1 2 1 αs − log λ + iε + . 2 χs − 1 s (52) Note that we do not let n → 0 in Eq. (52), but we take limN →∞ N1 EX [log Z(λ + iε|X)] = limN →∞ N1 log EX [Z(λ + iε|X)]. From Eq. (24) and Eq. (52), we obtain ρ(λ) = − π1 Im limε→+0 χw for this case. One point should be noted here. Based on a result obtained in the previous work, we should substitute Eq.
where
p 1X f (tµ ). p→∞ p µ=1
hf (t)it = lim
(60)
From the extremum of Eq. (59), we obtain ˜ w )−1 , Qw = ((λ + iε)In + Q ˜ w = αQt , Q −1 ˜ t − In , Q t = t tQ
(61) (62) (63)
t
˜ t = Qw . Q
(64)
If we substitute Eq. (63) and Eq. (64) into Eq. (61) and Eq. (62), we obtain simultaneous equations in terms of ˜ w . Using Qw = χw In + qw Dn in Eq. (42) and Qw and Q ˜ Qw = χ ˜w In − q˜w Dn , we obtain the following saddle-point
6 equations: χw =
1 , λ + iε + χ ˜w
q˜w , (λ + iε + χ ˜w )(λ + iε + χ ˜w − n˜ qw ) t , χ ˜w = α tχw − 1 t t2 q˜w = αqw . (tχw − 1)(tχw − 1 + ntqw ) t qw =
(65)
These are obtained using a similar approach to that used earlier in this paper (see Appendix A for details). We note that this is not dependent on either W or U . From this, we obtain the saddle-point equations:
(66) Qw = (67)
1 , λ + iε + αχt χ ˜w = αχt , p tµ 1X . χt = p µ=1 tµ χs − 1
˜ w + sQ ˜s (λ + iε)In + Q
(68)
(69) (70)
(74) (75) (76)
(79)
˜ t = Qs . Q
(80)
(77) (78)
˜w, Q ˜ s, Q ˜ u, Q ˜ t into Eq. (73) to Eq. (76), If we substitute Q we obtain Qu = χu In − qu Dn and Qt = χt In − qt Dn . In a similar way, since the off-diagonal elements of the order parameter matrices are 0, we obtain χw χs
Kronecker product correlation; case 3
1 log EX [Z n (λ + iε|X)] N α 1 ˜w − TrQs Qt + TrQw Q = Extr ˜ ˜ ˜ ˜ 2 2 Qw ,Qs ,Qu ,Qt ,Qw ,Qs ,Qu ,Qt 1 ˜ s + α TrQu Q ˜ u + α TrQt Q ˜t + TrQs Q 2 2 2 E 1D ˜ w + sQ ˜ s| log det |(λ + iε)In + Q − 2 D s Eo α ˜ u − tQ ˜ t| log det |In − Q . (72) − 2 t lim
,
s
˜ w = 0, Q ˜ s = αQt , Q ˜ Qu = 0,
χu
N →∞
(73)
t
(71)
As a more general case, we consider the asymptotic eigenvalue distribution when the covariance of the components of the random matrix is given by EX [xiµ xjν ] = mij θµν , that is, the components are mutually correlated. Since the correlation EX [xiµ xjν ] = mij θµν is represented as a Kronecker product, we can diagonalize M = {mij } ∈ RN ×N and Θ = {θµν } ∈ Rp×p with the diagonal matrices S = diag {s1 , · · · , sN } ∈ RN ×N and T = diag {t1 , · · · , tp } ∈ Rp×p and the orthogonal matrices W ∈ RN ×N and U ∈ Rp×p , such that M = W SW T ∈ RN ×N , Θ = U T U T ∈ Rp×p , and
, s
−1 ˜ w + sQ ˜s s (λ + iε)In + Q −1 ˜t + Q ˜ u − In , Qu = tQ t −1 ˜t + Q ˜ u − In , Q t = t tQ
That is, when N is sufficiently large, the use of replica analysis in this case produces results that are consistent with those obtained in previous studies.
D.
−1
Qs =
From Eq. (66) and Eq. (68), we estimate qw = q˜w = 0. Furthermore, since Eq. (65) and Eq. (67) hold for any n, limN →∞ N1 log EX [Z n (λ + iε|X)] = n limN →∞ N log EX [Z(λ + iε|X)] holds approximately. From this, we can also find the asymptotic eigenvalue distribution for this case. To compare this with the result of the previous work [10], we substitute Eq. (57) and Eq. (58) into Eq. (15) to Eq. (18) to obtain χw (= χs ) =
χt
1 = , λ + iε + αsχt s s = , λ + iε + αsχt s 1 , = tχs − 1 t t = . tχs − 1 t
(81) (82) (83) (84)
Note that this finding includes the findings presented in the previous subsection. Furthermore, we verified that the proposed method includes as a special case the approach based on Feynman diagrams.
IV.
BELIEF PROPAGATION ALGORITHM A.
Multivariate Gaussian distribution
Replica analysis is one way to analyze a quenched ordered system by using self-averaging and/or the assumption that the matrix size N is sufficiently large. However, an arbitrary random matrix ensemble is not always selfaveraging, the size N may be large but not infinite; for example, an assets return matrix in the mean-variance model of investment management is assumed to be finite, and so it is also important to be able to determine the eigenvalue distribution ρ(λ|X) when N is large but finite.
7 We use Eq. (23), as follows: ρ(λ|X) Z ∞ T 1 T ∂ 1 dwe ~ − 2 w~ ((λ+iε)IN −XX )w~ 2 Im lim log N ε→+0 ∂λ N π (2π) 2 −∞ Z ∞ 1 w ~ Tw ~ = − Im lim . (85) dwP ~ (w|λ, ~ X) ε→+0 −∞ π N
=
The expectation of w ~ Tw ~ using P (w|λ, ~ X) can be applied to determine the eigenvalue distribution, where the probability density function P (w|λ, ~ X) is a multivariate Gaussian distribution with N variables: 1
P (w|λ, ~ X) =
e− 2 w~
T
((λ+iε)IN −XX T )w ~
N
1
(2π) 2 det |(λ + iε)IN − XX T |− 2
.
(86)
Note that since we must directly determine the inverse matrix and determinant of (λ + iε)IN − XX T in order to average w ~ Tw ~ using P (w|λ, ~ X)w ~ Tw ~ in Eq. (86), when N is large, the calculation time will be excessive. In order to reduce the required computation time, we will consider a way to assess the expectation of w ~ Tw ~ with a trial distribution Q(w) ~ that, as evaluated by the Kullback-Leibler divergence, is approximately close to P (w|λ, ~ X).
B. Derivation from belief propagation algorithm based on the Kullback-Leibler information criterion
Based on the above discussion, we will derive Q(w), ~ which is an approximate trial distribution with respect to P (w|λ, ~ X) and is based on the Kullback-Leibler criterion [26]. In the context of belief propagation, P (w|λ, ~ X) in Eq. (86) is defined as follows: ! p N Y ~xT ~ 1 Y µw P (w|λ, ~ X) = P0 (wi ) g √ , (87) ZP i=1 N µ=1 ! Z ∞ p N Y Y ~xT ~ µw , (88) g √ P0 (wi ) ZP = dw ~ N −∞ µ=1 i=1 λ+iε
v2
2
where P0 (wi ) = e− 2 wi and g(v) = e 2 . On the other hand, the trial distribution Q(w) ~ is defined using beliefs bi (wi ), bµ (w) ~ as follows: 1 Q(w) ~ = ZQ ZQ =
Z
∞
−∞
N Y
!1−p
bi (wi )
i=1
dw ~
N Y
p Y
(89)
µ=1
!1−p
bi (wi )
i=1
bµ (w), ~ p Y
bµ (w), ~
(90)
µ=1
where beliefs bi (wi ) and bµ (w) ~ are defined as ∀i, µ,
bi (wi ) =
Z
∞
N Y
−∞ k=1,(k6=i)
dwk bµ (w), ~
(91)
R ∞ QN where −∞ k=1,(k6=i) dwk means the integral with respect to w ~ except for wi . Thus, the Bethe free energy, that is, the primary part of the Kullback-Leibler divergence between P (w|λ, ~ X) and Q(w) ~ is p Z ∞ X b ( w) ~ µ dwb ~ µ (w) ~ log ~xT w~ Q F = N µ √ −∞ g P (w ) µ=1 0 i i=1 N Z N ∞ X bi (wi ) dwi bi (wi ) log +(1 − p) . (92) P0 (wi ) i=1 −∞
That is, we determine bi (wi ) and bµ (w) ~ such that they minimize the Bethe free energy under the constraint given by Eq. (91). Although we can derive more approximate trial distributions Q(w), ~ we would rather evaluate the mean and the variance of wi with Q(w) ~ instead of Q(w) ~ so that we can analytically assess the eigenvalue distribution. From this, we obtain the mean and variance as follows: Z ∞ mwi = dwQ( ~ w)w ~ i, (93) −∞ Z ∞ (94) dwQ( ~ w)w ~ i2 − m2wi . χwi = −∞
In this setting, we used a previously developed algorithm based on the belief propagation method [16, 27], and obtained the following: hwk , (95) mwk = λ + iε + χ ˜wk p 1 X hwk = √ xkµ muµ + χ ˜wk mwk , (96) N µ=1 muµ = huµ
huµ , 1−χ ˜uµ
N 1 X = √ xkµ mwk − χ ˜uµ muµ , N k=1
1 , λ + iε + χ ˜wk p 1 X 2 = x χuµ , N µ=1 kµ
χwk = χ ˜wk
χuµ = χ ˜uµ =
1 , χ ˜uµ − 1
N 1 X 2 xkµ χwk , N
(97) (98) (99) (100) (101) (102)
k=1
where we note that the parameters other than mwk and χwk are auxiliary. It is easy to verify mwk = hwk = muµ = huµ = 0, and we determine χwk , χ ˜wk , χuµ , χ ˜uµ such that they satisfy Eq. (99) to Eq. (102). From Eq. (85), Eq. (93), and Eq. (94), the eigenvalue distribution ρ(λ|X) is found to be N 1 X 1 χwi . ρ(λ|X) = − Im lim ε→+0 N π i=1
(103)
8 The complexity of this algorithm is estimated to be O(N 2 ), since the complexity of calculating the inverse matrix is O(N 3 ), and thus our proposed approach is faster than the standard approach. This finding is consistent with an algorithm derived with the cavity method, in which the Bethe tree is assumed as the graphical model [27]. Finally, although we have considered a quenched disordered system, we also need to compare the results of our proposed method with those obtained in previous studies [9, 10]. Thus, we rewrite Eq. (99) as χuµ = −1 + χuµ χ ˜uµ ˜wk wk χ . It is then simand Eq. (101) as χwk = 1−χλ+iε ple to evaluate the configurational average with respect to randomness in Eq. (100) and Eq. (102) by using EX [x2kµ ] = mkk θµµ , and we obtain χ ˜uµ = θµµ χs , χ ˜wk = αmkk χt ,
(104) (105)
where χs = χt =
N 1 X mkk χwk , N k=1 p X
1 θµµ χuµ . p µ=1
From this, we obtain χw =
(106) (107)
N 1 X χwk N k=1
1 − αχt χs = , λ + iε p 1X χu = χuµ p µ=1 = −1 + χs χt ,
(108)
(109)
which correspond to Eq. (16) and Eq. (18), respectively. In addition, Qw = diag{χwk } ∈ CN ×N and Qu = diag{χuµ } ∈ Cp×p , and from Eq. (106) to Eq. (109), we obtain 1 (110) χw = TrQw , N 1 χu = TrQu , (111) p 1 χs = TrM Qw , (112) N 1 χt = TrΘQu ; (113) p these results are consistent with those obtained in previous studies [9, 10]. Note that if we are given EX [xiµ xjν ] = mij θµν , we can determine the asymptotic eigenvalue distribution with replica analysis and Feynman diagrams, and when covariance is unknown, we can determine the eigenvalue distribution with the belief propagation algorithm, Eq. (99) to Eq. (102). Note that this latter approach does not require knowledge of EX [xiµ xjν ].
V.
NUMERICAL EXPERIMENTS AND APPLICATIONS
We now consider the eigenvalue distributions obtained by the replica analysis and belief propagation algorithms, and we verify the proposed approaches by presenting the results of several numerical experiments. A.
Independent but not identically distributed; case 1
From the above arguments, since the mathematical structure of the second-order statistics of randomness is similar for all three cases (for example, we can simultaneously diagonalize M and Θ in replica analysis), we will first consider the independently but not identically distributed situation (case 1) in detail. We assume that the probability of sk follows the uniform distribution: 1 s ≤ sk ≤ smax P (sk ) = smax −smin min , (114) 0 otherwise and we will consider the following three cases: case (1,a): (smin , smax ) = (1, 5); case (1,b): (smin , smax ) = (2, 4); and case (1,c): (smin , smax ) = (2.5, 3.5) and α = p/N = 4. The results are shown in Fig. 1. In order to verify the effectiveness of our proposed approaches, we compared the results with the eigenvalue distributions derived from replica analysis and belief propagation (see appendix B). The matrix size used for the belief propagation experiments was N = 500, and each component in the random matrix was assigned from the Gaussian distribution defined by hyperparameter sk , which follows the random uniform distribution in Eq. (114); 100 samples were prepared. As shown in Fig. 1, the results were in compliance with each other. In a similar manner, we used the Householder method (and the Sturm theorem) [28], which can rigorously evaluate eigenvalue distributions; these results are also shown in Fig. 1. For the Householder method, we plotted the average of 100 samples with N = 500. The results shown in Fig. 1 verify that the eigenvalue distributions can be accurately obtained with replica analysis and belief propagation, since they are consistent with the results of the Householder method. As compared with independently and identically distributed case, Marˇcenko-Pastur (MP) law when the component xiµ is independently and identically distributed is defined as follows; p [λ+ − λ]+ [λ − λ− ]+ + ρ(λ) = [1 − α] δ(λ) + , (115) 2πλv √ where λ± = (1 ± α)2 v and the constant v = P P N p 1 2 For ini=1 µ=1 EX [xiµ ] = hsis htit are used. Np stance, if v = 3 and α = 4, then λ− = 3 and λ+ = 27. Shown in Fig. 1, it turns out that when |smax −smin | is becoming small, the eigenvalue distribution is close to MP
9 law. In addition, the results of the previous works which handled the market correlation and analyzed the eigenvalues (and eigensignals) of financial cross-correlation matrix in detail are supported by our proposed methods [5, 18, 19].
B.
Independent but not identically distributed; case 2
not identically distributed case, case 1. We begin with Z ∞ 1 1 = dλρ(λ) λ λ λ −∞ Z ∞ Z ∞ 1 1 dλ dsP (s) = − Im lim αs ε→+0 π 0 −∞ λ λ + iε + χs (λ)−1 Z ∞ 1 dsP (s) = − Im lim ε→+0 0 π ! Z Z −r
lim
R→∞,r→+0
Next, we also discuss another situation of independent but not identically distributed (case 2). We assume that the probability of tµ follows the uniform distribution: P (tµ ) =
1 tmax −tmin
0
tmin ≤ tµ ≤ tmax , otherwise
(116)
and we will consider the following three cases: case (2,a): (tmin , tmax ) = (1, 5); case (2,b): (tmin , tmax ) = (2, 4); and case (2,c): (tmin , tmax ) = (2.5, 3.5) and α = p/N = 4. The results are shown in Fig. 2. The effectiveness of our proposed approaches is verified from the comparation with the results of the eigenvalue distributions from replica analysis and belief propagation (see appendix B) and the one of Householder method. The numerical setting is similar to that of case 1. As shown in Fig. 2, the results were in compliance with each other. Moreover the eigenvalue distributions in three cases are close max to MP law in Eq. (115) with v = tmin +t = 3 be2 cause of the definition of Wishart matrix; its each eleP ment (XX T )ij = N1 pµ=1 xiµ xjµ . C.
dλf (λ) , (117)
−R
r
where, since χs is dependent on λ, we rewrite χs as χs (λ). Now, we have " # 1 1 1 f (λ) = . (118) − αs αs iε + χs (λ)−1 λ λ + iε + χs (λ)−1 From the Cauchy integral theorem, we have I 1 dz 0= αs z iε + C χs (z)−1 "Z −r 1 dz = lim αs R→∞,r→+0 −R z iε + χs (z)−1 Z 0 Z R 1 dz dθ +i + αs αs z iε + χs (z)−1 π iε + χs (reiθ )−1 r # Z π dθ , (119) +i αs iε + 0 χs (Reiθ )−1 where we already replaced z = reiθ in the third term and z = Reiθ in the fourth term. Furthermore, since 1 , 1−α lim χs (z) = 1, lim χs (z) =
z→0
Kronecker product correlation; case 3
Lastly, we also discuss the situation of Kronecker product correlation (case 3). We use the parameter probabilities P (sk ) in Eq. (114) and P (tµ ) in Eq. (116) with (smin , smax ) = (1, 5) and (tmin , tmax ) = (0, 2) bemax tmin +tmax cause of v = smin +s = 3. In Fig. 3, it turns 2 2 out that the results of three methods, replica analysis, belief propagation and Householder method, are consistent. Futhermore, from case (1,a); (smin , smax ) = (1, 5) and (tmin , tmax ) = (1, 1) to case (3); (smin , smax ) = (1, 5) and (tmin , tmax ) = (0, 2), the smallest and largest eigenvalues are varied from λmin (≃ 1.950) to λmin (≃ 1.606) and from λmax (≃ 32.487) to λmax (≃ 35.713) as compared with λ− = 3 and λ+ = 27 of MP law in Eq. (115).
R
dλf (λ) +
|z|→∞
(120) (121)
then lim
ε→+0
Z
∞
−∞
dz χs (0) − 1 1 = iπ αs z iε + χs (z)−1 αs =
iπ . s(1 − α)
Next, in a similar way, we estimate Z ∞ 1 dz lim = 0. αs αs ε→+0 −∞ z + iε + iε + χs (z)−1 χs (z)−1
(122)
(123)
Thus, we obtain
−1 . s 1 s = , λ λ α−1
(124)
Moreover, if we use D.
Applications: Expectations of λ−1 and λ−2
Finally, we consider the expectations of λ−1 and λ−2 for this eigenvalue distribution in the independently but
α s−1 s ∂χs (z) lim , =− z→0 ∂z (α − 1)3 ∂χs (z) lim = 0, |z|→∞ ∂z
(125) (126)
10 then we obtain Z ∞ 1 1 = dλρ(λ) 2 2 λ λ λ −∞
−1 2
−2 s s s s = + . (127) 3 (α − 1) (α − 1)2 See appendix C, they are consistent with the results from analysis of portfolio optimization problem. VI.
Appendix A: Replica calculation for the correlated case
When EX [xiµ xjν ] = mij θµν , we obtain
=
SUMMARY AND FUTURE WORK
In this paper, we considered the asymptotic eigenvalue distribution of a Wishart matrix defined by a random rectangular matrix. We considered three cases: (1) the components in each column are identically distributed, (2) the components in each row are not identically distributed, and (3) the components are correlated with one another. For each of these cases, we assessed the eigenvalue distribution using replica analysis, and we derived an algorithm for solving this based on belief propagation. Our proposed approaches reproduced the findings of the Feynman diagram approach, which has been discussed in previous works, and the effectiveness of our approaches was validated by numerical experiments. As an area of future work, since the random rectangular matrices considered in this paper can be regarded as dense, we also plan to analyze the asymptotic eigenvalue distribution for random rectangular sparse matrices, and to consider the case in which the entries are not identically distributed and that in which the entries are correlated with one another. In addition, since it is assumed in various applications (such as those in the crossdisciplinary fields of portfolio optimization, code division multiple access, and perceptron learning) that the components of the random rectangular matrix are i.i.d., our findings can be applied to the analysis of these problems, and the approaches discussed in previous works can be further developed for use with cases in which the entries are not i.i.d. and/or in which they are correlated.
EX [Z n (λ + iε|X)] Z ∞ Y n dw ~ a d~ua d~va −∞ a=1 n X
+ =
Z
λ + iε X T w ~ w ~a 2 a=1 a !# T ~va − X w ~a
exp − E Nn +pn X 2
n X 1 ~vaT~va + i ~uT a 2 a=1 a=1
∞
n Y dw ~ a d~ua d~va
−∞ a=1 n X
(2π)
Nn 2 +pn
a=1
n
exp −
1 T ~v ~va + i~uT va a~ 2 a
+
λ + iε X T w ~ w ~a 2 a=1 a n
n
p XX w ub ~ aT M w ~ b ~uT a Θ~ − 2 a=1 N p b=1
!
EX [Z n (λ + iε|X)] Z ∞ Y p n Y N n dwia dzia d¯ zia Y Y duµa dvµa dyµa d¯ yµa = 3N n 2pn (2π) (2π) 2 −∞ a=1 i=1 a=1 µ=1 exp − +i
p n N n λ + iε X X 2 1 XX 2 v wia + 2 i=1 a=1 2 µ=1 a=1 µa
p X n X
µ=1 a=1
+i +i
N X n X
i=1 a=1 p X n X
n
z¯ia y¯µa
n X n X
q˜wab
k=1 p X
yµa − N X i=1
b=1
1 XX q˜sab − 2 a=1
N X
1 − 2 a=1
µ=1
n
n
b=1 n X n X n
n
p XX qsab qtab 2 a=1 b=1 ! N X T zia − Wik wka
uµa vµa −
µ=1 a=1
1 − 2 a=1
.
(A1)
For the novel order parameters, we obtain qsab = T u ~ T Θ~ u w ~a Mw ~b and qtab = a p b , then ~za = W T w ~ a and N T ~ya = U ~ua , and we rewrite (A2) and (A3) as qsab = PN Pp 1 1 k=1 zia zib si and qtab = p µ=1 yµa yµb tµ , where N T T M = W SW and Θ = U T U . From this, we obPN PN tain qwab = N1 i=1 wia wib = N1 i=1 zia zib and quab = P P p p 1 1 µ=1 uµa uµb = p µ=1 yµa yµb . Thus, we can evaluate p
ACKNOWLEDGEMENTS
The author appreciates the fruitful comments of T. Nagao and K. Kobayashi. The work is in part supported by Grant-in-Aid Nos. 24710169 and 15K20999; the President Project for Young Scientists at Akita Prefectural University; research project No. 50 of the National Institute of Informatics, Japan; research project No. 5 of the Japan Institute of Life Insurance; research project of the Institute of Economic Research Foundation; Kyoto University; research project No. 1414 of the Zengin Foundation for Studies in Economics and Finance; research project No. 2068 of the Institute of Statistical Mathematics; research project of Mitsubishi UFJ Trust Scholarship Foundation; and research project No. 2 of the Kampo Foundation.
(2π)
n
"
q˜uab
b=1 n
1 XX q˜tab − 2 a=1 b=1
i=1 p X p X
µ=1
ν=1
T Uµν uνa
!
zia zib − N qwab
!
zia zib si − N qsab yµa yµb − pquab
!
!
yµa yµb tµ − pqtab
!!
.
(A2)
11 Finally, we obtain 1 log EX [Z n (λ + iε|X)] N α 1 ˜ s + 1 TrQw Q ˜ w + α TrQu Q ˜u = − TrQs Qt + TrQs Q 2 2 2 2 E D α ˜ w + sQ ˜ s ˜ t − 1 log det (λ + iε)In + Q + TrQt Q 2 s E 2 αD ˜ ˜ − log det In − Qu − tQt , (A3) 2 t lim
N →∞
Rp×p and the orthogonal matrix U ∈ Rp×p , we use the following iterations; s χs = , (B7) λ + iε + αsχt s t , (B8) χt = tχs − 1 t then,
where Extr is abbreviated here.
Appendix B: Algorithms based on replica analysis and belief propagation
We summary the both algorithms for resolving the eigenvalue distribution ρ(λ) in three cases and use them in order to derive the eigenvalue distribution in numerical experiments.
1.
2.
a. Algorithm for Case (1) In order to assess ρ(λ) when EX [xiµ xjν ] = si δij δµν , we use the following iteration; + * s , (B1) χs = λ + iε + χsαs −1
,
(B9)
s
(B10)
In order to assess ρ(λ),
1 , λ + iε + χ ˜wk p 1 X 2 x χuµ , = N µ=1 kµ
χwk =
(B11)
χ ˜wk
(B12)
χuµ = χ ˜uµ =
s
Algorithm based on belief propagation
d. Algorithm for three cases we use the following iterations;
Algorithms based on replica analysis
then,
1 λ + iε + αsχt 1 ρ(λ) = − Im lim χw . ε→+0 π χw =
1 , χ ˜uµ − 1
(B13)
N 1 X 2 xkµ χwk , N
(B14)
k=1
then, χw =
*
1 λ + iε +
αs χs −1
+
,
(B2)
χw =
s
1 ρ(λ) = − Im lim χw . ε→+0 π
(B3)
(B4) (B5)
then,
1 ρ(λ) = − Im lim χw . ε→+0 π
From [15, 17], two quantities in portfolio optimization problem are derived by replica analysis, as follows: ( 1
(B6)
c. Algorithm for Case (3) In order to assess ρ(λ) when EX [xiµ xjν ] = mij θµν ; M = {mij } = W SW T ∈ RN ×N is composed by the diagonal matrix S = diag {s1 , · · · , sN } ∈ RN ×N and the orthogonal matrix W ∈ RN ×N and Θ = {θµν } = U T U T ∈ Rp×p is composed by the diagonal matrix T = diag {t1 , · · · , tp } ∈
(B16)
Appendix C: Two quantities in the portfolio optimization problem
ε=
1 ρ(λ) = − Im lim χw . ε→+0 π
(B15)
k=1
b. Algorithm for Case (2) In order to assess ρ(λ) when EX [xiµ xjν ] = tµ δij δµν , we use the following iterations; 1 , χw = λ + iε + χ˜w t , χ ˜w = α tχw − 1 t
N 1 X χwk , N
qw =
2hλ−1 iλ α−1 2hs−1 is
−2 hλ−1 i2λ hλ
,
(C1)
iλ
s−2 h −1 i2s + hs i s
1 α−1
,
(C2)
where ǫ and qw are from [15, 17] and the notation Z ∞ hf (λ)iλ = dλρ(λ)f (λ). (C3) −∞
12 From these, we can then find
−1
−1 s s = λ , λ α−1
−2
−1 2
−2 s s s s + , = λ 3 λ (α − 1) (α − 1)2
(C4) (C5)
which are consistent with the results of Eq. (124) and Eq. (127).
[1] M. L. Mehta, Random Matrices, (Academic Press, 2004). [2] A. M. Tulino,S. Verd´ u, Random Matrix Theory and Wireless Communications, (Now Publishers, 2004). [3] Z. Bai, J. W. Silverstein, Spectral Analysis of Large Dimensional Random Matrices (Springer, 2009). [4] T. Guhr, A. M¨ uller-Groeling, A. Weidenm¨ uller, Phys. Rep. 299, 189, (1998). [5] J. Kwapie´ n, S. Dro˙zd˙z, Phys. Rep. 515, 115, (2012). [6] M. A. Marˇcenko, L. A. Pastur, Math. USSR-Sb., 1, 457 (1967). [7] J. W. Silverstein, S. I. Choi, J. Multi. Ana., 54, 295 (1995). [8] J. W. Silverstein, J. Multi. Ana., 55, 331 (1995). [9] A. M. Sengupta, P. P. Mitra, Phys. Rev. E, 60, 3389 (1999). [10] Z. Burda, A. G¨ orlich, A. Jarosz, J. Jurkiewicz, Phys. A, 343, 295 (2004). [11] Z. Burda, J. Jurkiewicz, B. Waclaw, Phys. Rev. E, 71, 026111 (2005). [12] C. Recher, M. Kieburg, T. Guhr, Phys. Rev. Let., 105, 244101 (2010). [13] H. Markowitz, Portfolio Selection: Efficient Diversification of Investments (J. Wiley and Sons, New York,
1959). [14] R. Wakai, T. Shinzato, Y. Shimazaki, J. Japan Indust. Mana. Associ., 65, 17 (2014). [15] T. Shinzato, PLoS One, 10, e0133846 (2015). [16] T. Shinzato, M. Yasuda, PLoS One, 10, e0134968 (2015). [17] T. Shinzato, to be submitted. (2016). [18] J. Kwapie´ n, S. Dro˙zd˙z, P. O´swi¸ecimka, Physica A, 359, 589 (2006). [19] S. Dro˙zd˙z, J. Kwapie´ n, P. O´swi¸ecimka, Acta Phys. Polo. B, 38, 4027 (2007). [20] R. A. Fishar, Biometrika, 10, 507 (1912). [21] J. Wishart, Biometrika, 20A, 32, (1928). [22] A. Kamenev, M. M´ezard, J. Phys. A, 32, 4373 (1999). [23] S. F. Edwards, R. C. Jones, J. Phys. A, 9, 1595 (1976). [24] G. S. Dhesi, R. C. Jones, J. Phys. A, 23, 5577 (1990). [25] H. Nishimori, Statistical Physics of Spin Glasses and Information Processing (Oxford University Press, 2001). [26] Y. Kabashima, J. Phys. Soc. Jpn. 74, 2133 (2005). [27] T. Roger, I. P. Castillo, R. K¨ uhn, K. Takeda, Phys. Rev. E, 78, 0131116. (2008). [28] W. H. Press, S. A. Teukolsky, W. T. Vetterling, B. P. Flannery, Numerical Recipes 3rd Edition: The Art of Scientific Computing (Cambridge University Press, 2007).
13
ρ (λ)
(1,a) (smin,smax)=(1,5)
0.08 0.06 0.04 0.02 0
replica analysis belief propagation Householder MP law
0
5 10 15 20 25 30 35 λ
ρ (λ)
(1,b) (smin,smax)=(2,4)
0.08 0.06 0.04 0.02 0
replica analysis belief propagation Householder MP law
0
5 10 15 20 25 30 35 λ
ρ (λ)
(1,c) (smin,smax)=(2.5,3.5)
0.08 0.06 0.04 0.02 0
replica analysis belief propagation Householder MP law
0
5 10 15 20 25 30 35 λ
FIG. 1. Comparison of the asymptotic eigenvalue distribution derived with analysis and by belief propagation for cases (1,a), (1,b) and (1,c), with α = p/N = 4. The horizontal axis shows the eigenvalues, λ, and the vertical axis shows the asymptotic eigenvalue distribution, ρ(λ). The solid line (orange) shows the results of replica analysis, asterisks with error bars (blue) show the results of belief propagation, and boxes with error bars (green) show the results of the Householder method; the matrix size is N = 500 with 100 samples. (1,a) λmin ≃ 1.950 and λmax ≃ 32.487. (1,b) λmin ≃ 2.768 and λmax ≃ 28.762. (1,c) λmin ≃ 2.944 and λmax ≃ 27.504. As compared with i.i.d. case, the dashed line (purple) shows the results of Marˇcenko-Pastur law in Eq. max (115) with v = smin +s = 3. 2
14
ρ (λ)
(2,a) (tmin,tmax)=(1,5)
0.08 0.06 0.04 0.02 0
replica analysis belief propagation Householder MP law
0
5 10 15 20 25 30 35 λ
ρ (λ)
(2,b) (tmin,tmax)=(2,4)
0.08 0.06 0.04 0.02 0
replica analysis belief propagation Householder MP law
0
5 10 15 20 25 30 35 λ
ρ (λ)
(2,c) (tmin,tmax)=(2.5,3.5)
0.08 0.06 0.04 0.02 0
replica analysis belief propagation Householder MP law
0
5 10 15 20 25 30 35 λ
FIG. 2. Comparison of the asymptotic eigenvalue distribution derived with analysis and by belief propagation for cases (2,a), (2,b), and (2,c), with α = p/N = 4. The horizontal axis shows the eigenvalues, λ, and the vertical axis shows the asymptotic eigenvalue distribution, ρ(λ). The solid line (orange) shows the results of replica analysis, asterisks with error bars (blue) show the results of belief propagation, and boxes with error bars (green) show the results of the Householder method; the matrix size is N = 500 with 100 samples. The numerical setting is similar to that of Fig. 1. (2,a) λmin ≃ 2.763 and λmax ≃ 28.765. (2,b) λmin ≃ 2.944 and λmax ≃ 27.489. (2,c) λmin ≃ 2.986 and λmax ≃ 27.129. As compared with i.i.d. case, the dashed line (purple) shows the results of max Marˇcenko-Pastur law in Eq. (115) with v = tmin +t = 3. 2
15
0.1
(3) (smin,smax)=(1,5), (tmin,tmax)=(0,2) replica analysis belief propagation
ρ (λ)
0.08
Householder MP law
0.06 0.04 0.02 0 0
5
10 15 20 25 30 35 40 λ
FIG. 3. Comparison of the asymptotic eigenvalue distributions derived by replica analysis and belief propagation for case (3). The numerical setting is similar to that of Fig. 1. (3) λmin ≃ 1.606 and λmax ≃ 35.713. As compared with i.i.d. case, the dashed line (purple) shows the results of Marˇcenkomax tmin +tmax = 3. Pastur law in Eq. (115) with v = smin +s 2 2
