1
Shift-enabled condition is necessary even for symmetric shift matrices
arXiv:1810.12677v1 [eess.SP] 30 Oct 2018
Liyan Chen, Samuel Cheng∗ , Senior Member, IEEE, Kanghang He, Lina Stankovic, Senior Member, IEEE, and Vladimir Stankovic, Senior Member, IEEE
Abstract—In a 2013 paper by Sandryhaila and Moura, the authors introduced a condition (herein we will call it shift-enabled condition) that any shift invariant filter can be represented by the shift matrix if the condition is satisfied. In the same, the authors also argued that shift-enabled condition can be ignored as any non-shift-enabled matrix can be converted to a shift-enabled one. In our prior work, we proved that such conversion in general may not hold for a directed graph with non-symmetric shift matrix. This letter will focus on undirected graphs where shift matrix is generally symmetric. Though the shift matrix can be converted to satisfy shift-enabled condition, the converted matrix is not associated with the original graph, making the conversion moot. Finally, some potential methods which preserving main graph topologies to convert graph shift matrices will be introduced. Note that these methods also do not hold for all matrices and further researches on shift enabled conditions are needed. Index Terms—graph signal processing, shift-enabled graphs, shift-invariant filter, perturbation.
I. I NTRODUCTION Graph signal processing (GSP) extends classical digital signal processing (DSP) and provides a prospective solution to numerous real-world problems that involve data defined on topologically complicated domains. However, there are several challenges in extending classic DSP to signals on graphs, particularly related to the design and application of filters. As in GSP, if one ignores the graph structure and puts all graph signal samples on to a vector, a linear graph filter can simply be written as the multiplication of a square matrix over the signal vector. Each output at a vertex will be a linear combination of inputs at all vertices. However, we may have billions or even trillions of vertices in practice and thus the naive implementation using such simple matrix multiplication will be computationally unaffordable. Actually, taking into account the graph structure of the data, an output often only depends on the inputs in its neighborhoods. Therefore, the direct matrix multiplication above may not be infeasible but also unnecessary. L. Chen is with the Department of Computer Science and Technology, Tongji University, Shanghai, 201804 China and Key Laboratory of Oceanographic Big Data Mining & Application of Zhejiang Province, Zhejiang Ocean University, Zhoushan, Zhejiang 316022, China (e-mail:
[email protected]). S. Cheng is with the Department of Computer Science and Technology, Tongji University, Shanghai, 201804 China and the School of Electrical and Computer Engineering, University of Oklahoma, OK 74105, USA (email:
[email protected]). K. He, V. Stankovic, and L. Stankovic are with Department of Electronic and Electrical Engineering, University of Strathclye, Glasgow, G1 1XW U.K. (e-mail:{vladimir.stankovic, lina.stankovic, kanghang.he}@strath.ac.uk). ∗ Corresponding author.
Of particular interest to this study is whether we can fully use the structure information of the graph to design the graph linear filter. Consider a generic matrix S associated with the graph such that Si,j 6= 0 if and only if there is an edge between vertex i and j. A signal filtered by such S can be computed efficiently under any data structure implementation since an output only involves inputs of its immediate neighborhood. For example, if the graph is a sensor network, filtering by S merely mixing the signal of a sensor with those of other nearest sensors. Moreover, filtering a signal by S k can be achieved by simply repeating the above step by K times. As a consequence, inherently it is advantageous to always decompose filter in a form of polynomial of such graph matrix S. This matrix can be considered as the generator of a filter. But we will call it the “shift” operator or “shift” matrix as it is more commonly referred to in the literatures [1]–[3]. It is known to be a “shift” since it is analogous to z −1 operator in Z−transform of classical 1-D DSP. In classical 1-D DSP, any linear time-invariant or shiftinvariant filter that commutes with time shift operator z −1 can be represented as a polynomial of z −1 . Meanwhile, if a linear filter can be represented as a polynomial of z −1 , the filter must be shift-invariant. Therefore, for linear classical DSP, shiftinvariant and polynomial representable are equivalent. However, the situation is much less obvious when we generalize to the case of GSP. That is, unlike classical DSP, being shiftinvariant (shift matrix commuting with the target filter) does not automatically imply that a polynomial representation of the filter exists. A. Contributions In [1], the authors showed that any filter commuting with shift matrix S can be represented as a polynomial in S provided that the characteristic and minimal polynomial of the shift matrix are equal (in this following, we will refer the condition as shift-enabled condition, also see Definition 1). However, the authors also immediately disregarded the latter condition arguing that one might convert any shift matrix that does not satisfy the condition into one that does. Based on this conclusion, most researchers always assume that the shift-enabled condition holds or simply ignoring the condition completely. However, it was proved in [4] through a counterexample that such conversion may not hold for a directed graph with asymmetric shift matrix. In this letter, we focus on the undirected graph and illustrate with examples that when the
2
symmetric shift matrix of an undirected graph is non-shiftenabled, the converted graph may not describe the same graph as the original graph, even if the shift-invariant filter can be represented as the converted shift matrix. This further emphasizes that, no matter what the structure of the graph is, the shift-enabled condition is necessary for any filter commutative with shift matrix to be representable as the polynomial in the shift matrix. The outline of the letter is as follows. Section II starts the basic concepts and key properties on shift-enabled graph. Section III provides counterexamples to prove that shiftenabled condition is essential to the symmetric graph. Section IV gives some conclusions. II. BASIC CONCEPTS AND PROPERTIES ON SHIFT- ENABLED GRAPH
In this section, we briefly review the concepts and properties of shift-enabled graph in GSP that are relevant to our letter. For more detail, see [1]–[3], [5]. GSP studies signals basing on graph G = (V, A), where V = {v0 , v1 , · · · , vn−1 } is a set of vertices and A ∈ Cn×n is the adjacency matrix of the graph which determined by the structure of the graph. Each data element corresponding to vertex forms a graph signal which usually is written as a vector x = (x0 , x1 , · · · , xn−1 )T . In particular, when G is a directed circular graph, ! the corresponding adjacency matrix 0 0 ··· 1 0 ···
0 1 0 0
.. .. . . . . .. , then Ac x = (xn−1 , x0 · · · , xn−2 )T . . . .. 0 0 ··· 1 0 which shift each signal to the next vertex. So A is called shift operator or shift matrix that is similar with time shift operator z −1 in DSP. In practice, shift matrix also can be replaced by other matrices which reflect the structure of the graph, such as the Laplacian matrix, the normalized Laplacian matrix, the probability transition matrix, and so on. In the remaining letter, we use S to denote the general shift matrix. In classical 1-D DSP, a shift-invariant filter F has a Ztransform (polynomial representation in z −1 ), that is Ac =
F (z −1 ) =
+∞ X
hk z −k ,
−∞
where hk is coefficients. Moreover, the filtered output of a shifted input should equal to the shifted filtered output of the original input. In other words, the shift operation and the filter should commute. That is, F z −1 = z −1 F . Extending this to GSP, we can thus also define a shift-invariant filters as one that commutes with the shift matrix, i.e., HS = SH. However, unlike the classical case, a shift-invariant filter does not automatically have a polynomial representation in terms of the shift-operator. Yet, H can be represented as a polynomial in S if the shift matrix S satisfied the following condition. Definition 1 (Shift-enabled graph [4]). A graph G is shiftenabled if its corresponding shift matrix S satisfies pS (λ) = mS (λ), where pS (λ) and mS (λ) are the minimum polynomial and the characteristic polynomials of S, respectively. We also say that S is shift-enabled when the above condition is
satisfied. Otherwise, S and the corresponding graph are nonshift-enabled. For shift-enabled graphs, we have the following important theorem which is the basis of linear filter design. Theorem 1. The shift matrix S is shift-enabled if and only if every matrix H commuting with S is a polynomial in S [1]. Note that the theorem actually implies that as long as the shift matrix S does not satisfy the shift-enabled condition (i.e., mS (λ) 6= pS (λ)), there will always be some shift-invariant filters (and thus some filters) that cannot be represented as a polynomial of S. Ref [1] deemphasized the shift-enabled condition by suggesting that we may work around it with the following theorem. However, as we demonstrate below, the shift-enabled condition described in Definition 1 is really essential and cannot be ignored. Theorem 2 (Theorem 2 in [1]). For any shift matrix S, there exists a converted matrix S˜ and matrix polynomial r(·), such ˜ and mS (λ) = pS (λ). that S = r(S) While the above theorem is accurate, it does not take into account that the target filter H may not be shift-invariant with the converted matrix any more. In particular, S is not symmetric for a directed graph and thus generally is not jointly diagonalized with H. Consequently, one can show that generally there is no converted shift-enabled S˜ that can maintain shift-invariance with the target filter when the graph is directed and S is asymmetric [4]. However, the conversion method suggested in [1] does hold for undirected graphs when H can be jointly diagonalized with S. Yet, as we will show in the following, S˜ may not describe the same graph as the original S. This makes the whole conversion process moot. Hence, the shift-enabled condition is necessary regardless the graph is directed or not (shift matrix is unsymmetric or not). III. T HE NECESSITY OF S HIFT- ENABLED CONDITION FOR SYMMETRIC SHIFT MATRICES
Before continuing with a concrete example, let us first review the conversion process described in [1]. As mentioned earlier, even though it does not hold for arbitrary graph matrices. The conversion process can be applied to symmetric graph matrices. According to Lemma 3, there must exist an invertible matrix T that simultaneously diagonalize two symmetric and commuting matrices S and H. That is, S = T ΛS T −1 and H = T ΛH T −1 , where ΛS and ΛH are composed by the eigenvalues of S and H. Then, a new matrix Λperturb with distinct diagonal elements can be generated by slightly perturbing the multiple values of ΛS . The new shift matrix is calculated as S˜ = T Λperturb T −1 . According to Lemma 2 and Lemma 3, the restructured shift matrix S˜ satisfies pS˜ (λ) = mS˜ (λ) and ˜ Hence, from Theorem 1, H is a polynomial in S. ˜ H S˜ = SH. However, it is not sufficient just to have H to be represented ˜ A natural and basic conas a polynomial of any arbitrary S. ˜ straint is that the converted S should keep the same topological structure as S of the graph, which is essential in virtually
3
4
5
1) Extension of H to a class of filters: Note that we can extended H to the following class of filters that all cannot be represented as polynomials of S:
4
1
2
5
1
2
H = {αH + q(S)|α ∈ R, q(S) is a polynomial of S}. (1) 3
3
(a)
(b)
(c)
Fig. 1: Graph topology of examples. (a) represents original graph with shift matrix S. (b) represents converted shift matrix S˜ which loosely describes the same graph as S. (c) is a cycle graph with shift matrix S 0 .
all GSP applications, such as filter design [6], sampling [7], denoising [8], and classification [9]. In a nutshell, two graph matrices describe the same graph if the conversion from one to another preserves the graph topological structure. The precise definition is specified as follows. Definition 2. Shift matrices S and S˜ strictly describe the same graph if 1) Si,j 6= 0 if and only if S˜i,j 6= 0 for any i and j, and 2) S˜ is symmetric if and only if S is symmetric. And we will say S and S˜ loosely describe the same graph if the first condition is relaxed to 1’) Si,j 6= 0 if and only if S˜i,j 6= 0 only for i 6= j. That is, we allow some i where only Si,i or S˜i,i equal to 0. Given this additional constraint that S˜ and S should describe the same graph, we can show that it is impossible to guarantee all following three conditions to be satisfied: ˜ is shift-enabled (i.e., p ˜ (λ) = m ˜ (λ)). • S S S ˜ (i.e., H S˜ = SH). ˜ • H is shift-invariant on S ˜ and S strictly or loosely describe the same graph. • S
A. A counter-example that S˜ can loosely but not strictly describe the original graph Let us start with a non-shift-enabled graph as shown in Fig. 1(a). ! The shift matrix of the undirected graph is S = 01111 1 1 1 1
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
. It is clear that pS (λ) = λ3 (λ − 2) (λ + 2) 6=
λ (λ − 2) (λ + 2) = mS (λ) and hence S is not shift-enabled. Since shift-enabled condition is not just sufficient but also necessary [4], there must exist a shift-invariant filter not representable as a polynomial of S. Indeed, as shown below that ! 0 0 0 00 H=
0 0 0 0
1 −1 0 0
−1 1 0 0
0 0 0 0
0 0 0 0
is such a filter. It can be readily verified
that HS = 0 = SH and thus the filter is shift-invariant. Yet, it is impossible to find polynomial representation of H in terms n n ∗ of S. Note that S2,3 = S2,4 for all n ∈ N. Thus for any polynomial h(S), we must have h(S)2,3 = h(S)2,4 . But since H2,3 = −1 6= 0 = H2,4 , H 6= h (S) for any polynomial function h(·). ∗ Note
k denotes the (i, j)-element of matrix S k . that Si,j
Since apparently q(S)S = Sq(S) for any polynomial q(S) and HS = SH as discussed above, any filter αH + q(S) ∈ H commutes with S as well. Thus any filter in H is shiftinvariant. However, since H is not representable as a polynomial of S, as discussed above, so does αH + q(S). From the examples presented above, we note that when the condition shift-enabled condition is violated, we may find not just a finite but an infinite number of shift-invariant filters that are not representable as polynomials of S. 2) Shift-enabled S˜ that strictly describes the original graph does not exist: First, let us restrict the converted shift matrix S˜ to strictly describe the same graph as S. Thus S˜ could be written as ˜ ˜ ˜ ˜ 0
S˜1,2 ˜ S˜ = S˜1,3 S1,4 ˜1,5 S
S1,2 S1,3 S1,4 S1,5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
(2)
with non-zero S˜1,2 , S˜1,3 , S˜1,4 and S˜1,5 . We can readily verify 2 2 −S˜13 − that the characteristic polynomial pS˜ (λ) = λ3 (λ2 −S˜12 2 2 ˜ ˜ ˜ S14 − S15 ) and 0 is the triple eigenvalues of S. According to Lemma 2, a shift-enabled graph matrix has to have unique eigenvalues and thus S˜ is not shift-enabled. Therefore, all graphs which have the same structure as Figure 1(a) are nonshift-enabled. 3) Shift-enabled S˜ that loosely describes the original graph exists: Now, let us relax our S˜ so that it may just loosely describe the original graph. In other words, we allow the diagonal element to be non-zero. In applications where diffusion or state transition matrices are treated as shift matrices, the diagonal elements can be interpreted as the returning probabilities of the current state to itself. Thus, the converted shift matrix S˜ can be written as ˜ ˜ ˜ ˜ ˜ S1,1 S1,2 S1,3 S1,4 S1,5 ˜2,2 0 S 0 0 ˜3,3 0 0 S 0 ˜4,4 0 S1,4 0 0 S ˜1,5 0 ˜5,5 S 0 0 S
S˜1,2 ˜ S˜ = S˜1,3
.
(3)
Using the symbolic math toolbox in MATLAB, many solutions that satisfy shift-enabled and shift-invariant ! conditions can be found. For instance, S˜ =
0 1 1 1 1
1 1 0 0 0
1 0 1 0 0
1 0 0 0 0
1 0 0 0 0
is one such
solution, where the original graph structure is only slightly modified as shown in Figure 1(b). One can verify that the eigenvalues, (−1.8136, 0, 0.4707, 1, 2.3429), of S˜ are distinct and thus S˜ is shift-enabled. Moreover, as one can also readily ˜ verify that H S˜ = SH. By Theorem 1, the above two ˜ conditions ensure that H is a polynomial in S. B. A counter example that the converted shift matrix can neither strictly nor loosely describe the original graph Note that there are situations where no shift-enabled S˜ exists even after we relax the graph structure constraint as in the
4
0 1 0 1 earlier example. Consider shift matrix S 0 = 10 01 10 01 as 10 0 10 0 −1 1 0 −1 shown in Figure 1(c) and target filter H 0 = −1 1 10 00 . 1
0
0 −1
It can be easily seen that the eigenvalues, (0, 0, 2, −2), of S 0 have duplicate values. Thus S 0 is non-shift-enabled according to Lemma 2. So we do expect that there exists shift-invariant filter not representable by S 0 . Indeed, we can easily show that H 0 is such a filter. First, note that H 0 S 0 = S 0 H 0 and thus H 0 is shift-invariant under S 0 . Furthermore, note that (S 0 )n1,2 = (S 0 )n1,4 for all n ∈ N, and so h(S 0 )1,2 = h(S 0 )1,4 for any polynomial h(S 0 ). But 0 0 since H1,2 = 0 6= 1 = H1,4 , H 0 6= h (S 0 ) for any polynomial function h(·). Next, let us prove that it is impossible to find a converted shift matrix S˜0 which is shift-enabled and commutes with H 0 by only changing the weights of nonzero and diagonal elements. Consider a general symmetric matrix S˜0 S˜0 0 S˜0 1,1
1,2
1,4
S˜0 S˜0 = 01,2
S˜0 2,2 S˜0 2,3 0
S˜0 2,3 0 S˜0 3,3 S˜0 3,4 S˜0 3,4 S˜0 4,4
S˜0 1,4
(4)
which has arbitrary weights on nonzero and diagonal elements. That is, S˜0 loosely describes the same graph as S 0 . For H 0 = h(S˜0 ) clearly implies that H 0 commutes with 0 ˜ S , namely, H 0 S˜0 = S˜0 H 0 is a necessary condition for H 0 = h(S˜0 ). It follows from H 0 S˜0 = S˜0 H 0 that S˜0 1,1 = S˜0 2,2 = S˜0 3,3 = S˜0 4,4 and S˜0 1,2 = S˜0 1,4 = S˜0 2,3 = S˜0 3,4 , i.e., S˜0 S˜0 0 S˜0 1,1
1,2
1,2
S˜0 S˜0 = 01,2
S˜0 1,1 S˜0 1,2 0
S˜0 1,2 0 S˜0 1,1 S˜0 1,2 S˜0 1,2 S˜0 1,1
S˜0 1,2
.
(5)
Following Cayley-Hamilton Theorem, if H 0 is a polynomial 2 3 in S˜0 , then H 0 = h(S˜0 ) = h0 I + h1 S˜0 + h2 S˜0 + h3 S˜0 , where I as the identity matrix. In fact, it is easy to figure out that (S˜0 )k1,2 = (S˜0 )k1,4 for k = 0, 1, 2, 3. Hence, 0 0 h(S˜0 )1,2 = h(S˜0 )1,4 which contradicts with H1,2 6= H1,4 . 0 Thus, for this example, the filter H cannot be represented as the polynomial in the converted shift matrix S˜0 which even just loosely describes the original graph. IV. C ONCLUSION For a non-shift-enabled graph, even if we can easily “transform” the symmetric shift matrix S into one that satisfies the shift-enabled condition, the new S˜ is always irrelevant since it describes a completely different graph from S. Note that, if S˜ is not a general graph matrix, S˜ is a disqualified “shift” operator in practice. The operator S˜ on a graph signal may involve mixing inputs far beyond its neighborhood and become impractical for huge graphs. Combining the necessity of shift-enabled condition for directed graph [4], shift-enabled condition is essential for any structure graph. A PPENDIX A It is easily determined whether a graph is shift-enabled by the following lemmas.
Lemma 1. The shift matrix S is shift-enabled if and only if each Jordan block in the Jordan canonical form of S is associated with distinct eigenvalue (see Proposition 6.6.2 in [10]). Since the Jordan canonical form of real symmetric matrix is a diagonal matrix, for which each diagonal element is a Jordan block, we obtain the next lemma from Lemma 1. Lemma 2. If shift matrix S is a real symmetric matrix, then S is shift-enabled, if and only if all eigenvalues of S are distinct. Lemma 2 indicates undirected graph is shift-enabled if and only if its eigenvalues are all distinct [11]. As both shift matrix S and filter matrix H are symmetric, we can obtain the following lemma. Lemma 3. If shift matrix S and filter matrix H are diagonalizable (this condition always holds for symmetric matrix) then S and H are simultaneously diagonalizable (by an invertible matrix) if and only if HS = SH (see Theorem 1.3.12 in [12]). ACKNOWLEDGMENT The project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement no. 734331. R EFERENCES [1] A. Sandryhaila and J. M. Moura, “Discrete signal processing on graphs,” IEEE transactions on signal processing, vol. 61, no. 7, pp. 1644–1656, 2013. [2] ——, “Discrete signal processing on graphs: Frequency analysis.” IEEE Trans. Signal Processing, vol. 62, no. 12, pp. 3042–3054, 2014. [3] ——, “Big data analysis with signal processing on graphs: Representation and processing of massive data sets with irregular structure,” IEEE Signal Processing Magazine, vol. 31, no. 5, pp. 80–90, 2014. [4] V. S. L. S. Liyan Chen, Samuel Cheng, “Shift-enabled graphs: Graphs where shift-invariant filters are representable as polynomials of shift operations,” IEEE Signal Processing Letters, vol. 25, no. 9, pp. 67–128, 2018. [5] D. I. Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst, “The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains,” IEEE Signal Processing Magazine, vol. 30, no. 3, pp. 83–98, 2013. [6] D. K. Hammond, P. Vandergheynst, and R. Gribonval, “Wavelets on graphs via spectral graph theory,” Applied and Computational Harmonic Analysis, vol. 30, no. 2, pp. 129–150, 2011. [7] A. Gadde, A. Anis, and A. Ortega, “Active semi-supervised learning using sampling theory for graph signals,” in Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2014, pp. 492–501. [8] C. Yang, G. Cheung, and V. Stankovic, “Estimating heart rate and rhythm via 3d motion tracking in depth video,” IEEE Transactions on Multimedia, vol. 19, no. 7, pp. 1625–1636, 2017. [9] H. Kanghang, L. Stankovic, J. Liao, and V. Stankovic, “Non-intrusive load disaggregation using graph signal processing,” IEEE Transactions on Smart Grid, vol. 9, no. 3, pp. 1739–1747, 2018. [10] P. Lancaster and M. Tismenetsky, The theory of matrices: with applications. Elsevier, 1985. [11] A. Ortega, P. Frossard, J. Kovaˇcevi´c, J. M. Moura, and P. Vandergheynst, “Graph signal processing: Overview, challenges, and applications,” Proceedings of the IEEE, vol. 106, no. 5, pp. 808–828, 2018. [12] R. A. Horn and C. R. Johnson, Matrix analysis. Cambridge university press, 2012.
