Representation of Convolution Systems on Finite

Preprint: Proceedings 13th International Conference on Computer Aided Systems Theory EUROCAST 2011

Representation of Convolution Systems on Finite Groups by Heterogeneous Decision Diagrams 1

1

Stanislav Stankovi´c, 2 Radomir S. Stankovi´c 1 Jaakko T. Astola, 3 Claudio Moraga

Dept. of Signal Processing, Tampere University of Technology, Tampere, Finland 2 Dept. of Computer Science, Faculty of Electronics, Niˇs, Serbia 3 European Centre for Soft Computing, 33600 Mieres, Spain & Technical University of Dortmund, 44221 Dortmund, Germany

Abstract. The outputs of linear shift-invariant systems are usually defined in terms of the convolution of input signals with the impulse response functions characterizing the systems. In many areas, as for instance, electrical engineering, digital signal and image processing, statistics, physic, optics, etc., convolution systems defined on finite groups are used. Such systems can be modeled and represented by convolution matrices. The problem is that due to the complexity of systems, dealing with large matrices is required. In this paper, we discuss representation of convolution systems on finite groups by Heterogeneous decision diagrams (HDDs). Such representations permit compact representations of convolution systems, and thanks to that, efficient manipulations and computations related to investigation of features and applications of such systems. Keywords: Convolution, Finite groups, Polynomial expressions, Spectral representations.

Signals representing information in communication and control systems demand in processing the use of strongly time-invariant systems, and shift-invariant and rotational-invariant systems in the case of speech and images, respectively. This means mathematically that the groups of real numbers R and complex numbers C are the natural domains for the definition of signals. However, in some areas of contemporary engineering practice, as for instance, in computer engineering, consideration of systems defined on various other groups may be required [4]. Conversely, methods developed for such (generalized) systems may provide advantages in solving some particular tasks in classical system theory and applications. Due to that, generalizations of systems theory to systems described by signals on groups different from R or C are a subject of considerable research efforts in the last few decades. For instance, the dyadic systems, i.e., systems defined on dyadic groups, have been intensively studied by F. Pichler in a series of papers and by several other authors; see [3] for a bibliography up to 1989. For more recent result, we refer to [2], [12], [15].

2

S. Stankovi´c, R.S. Stankovi´c, J.T. Astola, C. Moraga

In this paper, we consider linear convolution systems whose input and output signals are deterministic signals on finite groups. If a group G is decomposable into the direct product of some subgroups, then the input of a multi-input system S on G can be modeled by a multi-variable function f (x1 , . . . , xn ), where n is number of inputs in S. The variables xi do not necessarily take values in the same sets. Thus, the function f can be viewed as a function defined on a decomposable group G = ×ni=1 Gi , where × denotes the direct product and xi ∈ Gi . Modeling of systems in terms of the convolution product on finite groups can be performed in terms of convolution matrices. Such representations of systems usually require dealing with large matrices. If a matrix can be partitioned in a set of matrices of different radices, it can be efficiently represented using a Heterogeneous decision diagrams (HDDs) [5], [14]. Since convolution matrices on finite decomposable groups inherently have such a property, they can be compactly represented by HDDs. This property is determined by the structure of the underlying domain group. Thanks to this observation, in this paper, we propose a method to represent convolution matrices of systems on finite groups by HDDs. For construction of HDDs we make use of the XML based framework for the representation of decision diagrams [13].

1

Convolution Systems on Finite Groups

Here we will fix the notation that will be used later in the paper. Let G be a finite, not necessarily Abelian, group of order g = |G|. We associate (permanently and bijectively) with each group element a non-negative integer from the set {0, 1, . . . , g − 1}, and 0 is associated with the group identity. Thus, each group element will be identified with the fixed non-negative integer associated with it and with no other element. We assume that G can be represented as a direct product of subgroups G1 , . . . , Gn of orders g1 = |G1 |, . . . , gn = |Gn |, respectively, i.e., n G = ×ni=1 Gi , g = Πi=1 gi , g1 ≤ g2 ≤ . . . ≤ gn .

(1)

The convention adopted above for the notation of group elements applies to the subgroups Gi as well. Provided that the notational bijections of the subgroups and of G are consistently chosen, each x ∈ G can be uniquely represented as  n n  Πj=i+1 gj , i = 1, . . . , n − 1 x= ai xi , xi ∈ Gi , x ∈ G, ai = (2) 1, i = n, i=1

where gj = |Gj | is the order of Gj , and 0 ≤ xi < gi , i = 1, . . . , n. The group operation ◦ of G can be expressed in terms of the group operations ◦ of i the subgroups Gi , i = 1, . . . , n by: ◦

◦

◦

x ◦ y = (x1 1 y1 , x2 2 y2 , . . . xn n yn ), x, y ∈ G, xi , yi ∈ Gi .

(3)

Convolution Systems on Finite Groups and QMDDs

3

Denote by P the complex field or a finite field and by P (G) the space of functions f mapping G into P , i.e., f : G → P . Due to the assumption (1) and the relation (2), each function f ∈ P (G) can be considered as an n-variable function f (x1 , . . . , xn ), xi ∈ Gi . Definition 1 A scalar linear system S over a finite not necessarily Abelian group G is defined as a quadruple (P (G), P (G), h, ∗), where the input-output relation ∗ is the convolution product on G,  h(x)f (τ ◦ x−1 ), ∀τ ∈ G, (4) y = h ∗ f, f, h, y ∈ P (G), i.e., y(τ ) = x∈G

where ◦ is the group operation of G. An ordered pair (f, y) ∈ P (G) × P (G) is exactly then an input-output pair of S if f and y fulfill equation (4). The function h ∈ P (G) is the impulse response of S. It is easy to show that the system S is invariant against the translation of the input function. By that we mean that if y is the output to f , then T τ y is the output to T τ f , for all τ ∈ G, where T is the translation on G. Therefore, we denote the system S as a linear translation invariant (LTI) system. Example 1 Let S3 = (0, (132), (123), (12), (13), (23), ◦) be the symmetric group of permutations of order 3. According to the convention adopted in this paper, the group elements of S3 will be denoted by 0,1,2,3,4,5, respectively. A convolution system on the group S3 whose impulse response is h(x), x ∈ S3 is defined by the convolution matrix ⎤ ⎡ h(0) h(1) h(2) h(3) h(4) h(5) ⎢ h(1) h(2) h(0) h(5) h(3) h(4) ⎥ ⎥ ⎢ ⎢ h(2) h(0) h(1) h(4) h(5) h(3) ⎥ ⎥. ⎢ CS3 = ⎢ ⎥ ⎢ h(3) h(4) h(5) h(0) h(1) h(2) ⎥ ⎣ h(4) h(5) h(3) h(2) h(0) h(1) ⎦ h(5) h(3) h(4) h(1) h(2) h(0) If the input f is specified by the vector F = [f (0), f (1), f (2), f (3), f (4), f (5)]T , then the output of the system is the vector Y = [y(0), y(1), y(2), y(3), y(4), y(5)]T defined as Y = CS3 · F.

2

Representation of Matrices by Decision Diagrams

Quantum multiple-valued decision diagrams (QMDD) were introduced in [6] as a data structure to represent (pn × pn ) matrices through their decomposition into p2 submatrices of size pn−1 × pn−1 , with each non-terminal vertex in a QMDD specifying a decomposition of the given matrix. The non-terminal nodes necessarily have p2 outgoing edges with each edge assigned is a multiplicative attribute w. In this way, an edge points to the matrix which is the submatrix rooted by the vertex it points to multiplied by w.

4

S. Stankovi´c, R.S. Stankovi´c, J.T. Astola, C. Moraga

If p = 2, i.e., when the matrix to be represented is partitioned into (2 × 2) submatrices, a QMDD has nodes with four outgoing edges, which reduces the dept compared with BDDs and MTBDD by a half, resulting in the corresponding speed up in matrix computations over QMDDs. The heterogeneous decision diagrams (HDD) [5] are a generalization of QMDD derived by allowing a different number of outgoing edges for nodes at different levels in the diagram. It means that the matrix has a recursive structure consisting of submatrices of different dimensions at different levels of the decomposition of the matrix. A level consists of nodes to which the same decision variable is assigned. In HDDs, in addition to decision variables, a radix ri , showing the number of outgoing edges, is assigned to each non-terminal node. This is necessary in order to unambiguously specify the way a matrix is partitioned. Thus, HDDs can represent matrices that can be divided into blocks (submatrices) of different sizes. Therefore, HDDs can be used to represent convolution matrices of decomposable groups of the form (1) with subgroups of different sizes. A further generalization is proposed in [14] to represent rectangular matrices. Representation of matrices and elementary calculations, including matrix addition, multiplication Kronecker product, and Cartesian product, over HDDs and their generalizations are supported by the corresponding programming packages as reported in [5], [6], [14].

3

Representation of Convolution Matrices by HDDs

Convolution matrices on groups of the form (1) have a structure that is very suitable for representation by HDDs. This structure expresses as the appearance of identical blocks the sizes of which correspond to the orders of the subgroups |Gi |, i = 1, . . . , n. Block are nested into each other which corresponds to the hierarchy of levels in decision diagrams. Nodes at each level represent submatrices (blocks) in the convolution matrix. Constant nodes represent values of entries in the convolution matrix. To read a particular entry c(j, k) in the convolution matrix, we follow the corresponding path from the root node to the constant node showing the value c(j, k). Since the entries with the same values repeat in the convolution matrix, the number of paths to reach a constant node is equal to the number of appearances of that entry. In a decision diagram, paths consist of edges, and can be uniquely specified by ordered sets of labels at the edges a path consists of. These labels are determined as follows. Definition 2 If nodes at a level i represent an (r × r) submatix consisting of (q × q) blocks, the label at the edge in the path pointing to the node c(j, k) is determined as li = j/h + q · k/h + 1, where h = r/2, and x is the largest integer smaller than x. The following examples illustrate representation of convolution matrices by HDDs.

Convolution Systems on Finite Groups and QMDDs

3.1

5

Dyadic systems

Systems defined on dyadic groups have been introduced and greatly investigated in early seventies in the context of hight interest in the discrete Walsh functions and their applications. See, for example, [2], [7], [9], [10], [11], [12]. Example 2 For n = 3, the finite dyadic group of order 2n is defined as G = C23 , where C2 = ({0, 1}, ⊕), and ⊕ denotes the addition modulo 2, (logic EXOR). A convolution system on C23 is specified by the convolution matrix ⎤ ⎡ h(0) h(1) h(2) h(3) h(4) h(5) h(6) h(7) ⎢ h(1) h(0) h(3) h(2) h(5) h(4) h(7) h(6) ⎥ ⎥ ⎢ ⎢ h(2) h(3) h(0) h(1) h(6) h(7) h(4) h(5) ⎥ ⎥ ⎢ ⎢ h(3) h(2) h(1) h(0) h(7) h(6) h(5) h(4) ⎥ ⎥. ⎢ CC23 = ⎢ ⎥ ⎢ h(4) h(5) h(6) h(7) h(0) h(1) h(2) h(3) ⎥ ⎢ h(5) h(4) h(7) h(6) h(1) h(0) h(3) h(2) ⎥ ⎥ ⎢ ⎣ h(6) h(7) h(4) h(5) h(2) h(3) h(0) h(1) ⎦ h(7) h(6) h(5) h(4) h(3) h(2) h(1) h(0) This matrix structure that  has a recursive



can be expressed as ab AB CD , where a = , b= , where CC23 = ba BA DC

A=

 h(0) h(1) , h(1) h(0)

B=

 h(2) h(3) , h(3) h(2)

C=

 h(4) h(5) , h(5) h(4)

D=

 h(6) h(7) , h(7) h(6)

which allows to derive a compact representation for CC23 . Fig. 1 shows the HDD for CC23 . This HDD represents the matrix CC23 by capturing the structure of it in the following way. The root node has four outgoing edges labeled by 1, 2, 3, and 4. Two of them (1 and 4) point to the submatrix a, while the other two (2 and 3) point to the submatrix b. These submatrices are represented by subdiagrams rooted at the nodes labeled by a and b. Each of these nodes has four outgoing edges. Again pairs of them point to the same submatrices A, B, C, and D, in the left and the right subtrees, respectively. These submatrices are represented by subtrees rooted in the nodes labeled by A,B,C, and D. These nodes have four outgoing edges, the pairs of which point to the same values h(i), i = 0, 1, . . . , 7, that are entries of the matrix CC23 and, therefore, are represented by the values assigned to constant nodes. Consider, for example, the element c(3, 5) ∈ CC23 to illustrate the way of reading particular entries in the convolution matrices by traversing the corresponding paths in the given HDD. At the topmost level of the diagram in Fig. 1, the (8×8) convolution matrix is represented by a (2×2) symbolic matrix, and we obtain the appropriate edge label as l0 = j/h + q · k/h + 1, where x is the largest integer smaller than x, and q = 2 and h = 8/2 = 4. For j = 3 and k = 5, we obtain l1 = 3/4 + 2 · 5/4 = 0 + 2 + 1 = 3.

6

S. Stankovi´c, R.S. Stankovi´c, J.T. Astola, C. Moraga

We repeat the same process for indices ji , ki for each following level of HDD, where ji is a reminder of integer division of ji+1 /hi+1 − 1. For the second level in this particular HDD, j2 = 2, k2 = 0. Since at this level, (4 × 4) matrix a is represented by a (2 × 2) matrix, q2 = 2 and h2 = 4/2 = 2. Therefore, l2 = 2/2 + 2 · 0/2 = 1 + 0 + 1 = 2. Finally, in the similar manner, at the third level of the HDD we have, j3 = 0, k3 = 0, q3 = 2, h3 = 1. Thus, l3 = 0 + 0 + 1 = 1. CC23 3

1 2

4

a 1

4

2

3

1

h(1)

h(2)

B

A 1

4

2

h(0)

4

b

1 4

3

2

3

D

C 2

3

1

h(3)

h(4)

4

2

1

3 h(5)

4

h(6)

2

3 h(7)

Fig. 1. HDD for the convolution matrix of dyadic systems for n = 3.

◦

Example 3 Let G3×6 be the direct product of the group Z3 = (0, 1, 2, 3) of integers less than 3 with modulo 3 addition as the group operation, and the symmetric group of permutations of order 3, S3 described in Example 1. Hence, G3×6 consists of pairs (h1 , h2 ) = g ∈ G3×6 where h1 ∈ Z3 and h2 ∈ S3 . The group operation ◦ of G3×6 is specified as follows: for (h1 , h2 ) = g ∈ G3×6 ◦

◦

and (h1 , h2 ) = g  ∈ G3×6 we have (h1 3 h1 , h2 3 h2 ) = g ◦ g  ∈ G3×6 . For such decomposition the convolution matrix on G3×6 can be recursively represented as ⎡ ⎤ ab c C3×6 = ⎣ b c a ⎦ , cab where

and



 AB a= , BA ⎡

⎤ 012 A = ⎣1 2 0⎦, 201 ⎡

⎤ 9 10 11 D = ⎣ 10 11 9 ⎦ , 11 9 10



 CD b= , DC

 EF c= , FE

⎡

⎡

⎤ 345 B = ⎣4 5 3⎦, 534 ⎡

⎤ 12 13 14 E = ⎣ 13 14 12 ⎦ , 14 12 13



⎤ 678 C = ⎣7 8 6⎦, 867 ⎡

⎤ 15 16 17 F = ⎣ 16 17 15 ⎦ . 17 15 16

Convolution Systems on Finite Groups and QMDDs

7

Fig. 2 shows the HDD for the matrix C3×6 . In this HDD, the first level corresponds to the matrix with entries a, b, and c. The second level corresponds to the blocks A, B, C, D, E, and F, while the third level represents these blocks with their entries shown in constant nodes.

C3x6 1

1 3

a

4

0

8

2 49

1

1 6

9

2

3

8

b

1 3

B 5 3

2 49

4

3

249

2

A 1 6

6 8

2

4

5

6

8

2 49

7

1 6

9 5 3

8

9

8

c

1 3

D

C 1 6

7 5 3

7 5

249

10

2

4

E 9 5 3

11

1 6

12

8

2 49

13

F 7 5 3

14

1 6

15

8

2 49

16

9 5 3

17

Fig. 2. HDD for the convolution matrix C3×6 .

4

Experimental Results

We performed a series of experiments exploring complexity of HDDs for convolution with the specified impulse responses viewed as systems on different groups. As system responses, we have taken integer valued switching functions from MCNC set of benchmark functions. These functions are treated as impulse responses on groups of type C2n , where n is the number of input variables of the given switching function. Table 1 shows the sizes (number of nodes) of their corresponding HDDs.

Function x1 x5 x4 ⊕ x5 ¯5 x4 ⊕ x Add2 MulL2 bw maj5 rd53 rd531 squar53

Table 1. Sizes of HDDs on different groups. n Group Number of nodes Function n Group Number of nodes 5 C25 5 xor5 5 C25 5 5 5 C2 5 5xp1 7 C27 127 5 C25 5 con1 7 C27 16 5 5 C2 5 rd73 7 C27 26 14 rd84 8 C28 34 4 C24 8 sqrt8 8 C28 70 4 C24 25 misex1 8 C28 47 5 C25 5 9sym 9 C29 9 5 C25 13 clip 9 C29 180 5 C25 5 ex1010 10 C210 561 5 C25 5 sao2 10 C210 114 5 C25

8

S. Stankovi´c, R.S. Stankovi´c, J.T. Astola, C. Moraga

As excepted, linear single output switching functions such as x1 , x5 , x4 ⊕ x5 , x4 ⊕ x ¯5 , where n = 5 viewed as systems on C25 exhibit a strong regularity in their corresponding convolution matrices which results in a very compact HDD representations.

References 1. Clarke, E.M., Fujita, M., Zhao, X., ”Multi-terminal binary decision diagrams and hybrid decision diagrams”, in T. Sasao, M. Fujita, (eds.), Representations of Discrete Functions, Kluwer Academic Publishers, 1996, 93-108. 2. Endow, Y., ”Walsh harmonizable processes in linear system theory”, Cybernetics and Systems, 1996, 489-512. 3. Gibbs, J.E., Stankovi´c, R.S., ”Why IWGD-89? a look at the bibliography of Gibbs derivatives”, in Theory and Applications of Gibbs Derivatives, P. L. Butzer and R. S. Stankovi´c, (Eds.), Belgrade, Serbia, Matematiˇcki institut, 1990, xi-xxiv. 4. Karpovsky, M.G., Trachtenberg, E.A., ”Some optimization problems for convolution systems over finite groups”, Inf. and Control, Vol. 34, 1977, 227-247. 5. Miller, D.M., Stankovi´c, R.S., ”A Heterogeneous decision diagram package”, in R. Moreno-Diaz, F. Pichler, Q. Arenchibia, (Eds.), EUROCAST 2009, LNCS 5717, Springer-Verlag Berlin Heidelberg 2009, 540547. 6. Miller, D., Thornton, M., ”QMDD: A decision diagram structure for reversible and quantum circuits”, Proc. Int. Symp. on Multiple-Valued Logic (CD), 6 pages, 2006. 7. Moraga, C., ”Introduction to linear p-adic systems”, Cybernetics and Systems Research 2, (Ed.) R. Trappl, North-Holland, 1984. 8. Pearl, J., ”Optimal dyadic models of time-invariant systems”, IEEE Trans. Computers, Vol. C-24, 1975. 9. Pichler, F.R., ”Some aspects of a theory of correlation with respect to Walsh harmonic analysis”, Univ. Maryland Tech. Res. Rept R-70-11, (1970). 10. Pichler, F., ”On state space description of linear dyadic invariant systems”, IEEE Trans. on Electromagnetic Compatibility, Vol. EMC-13, 1971, 166. 11. Pichler, F.R., ”Dyadische Faltungsoperatoren zur Beschreibung linearer Systeme”, ¨ Akad. Wiss., Math.-naturwiss. KL, Abt. II, 180, Heft 1-3, 1971, 69-87. Siteber. Ost. 12. Pichler, F., ”Construction of dissipative dynamical system using Gibbs derivatives”, in R.S. Stankovi´c, (ed.), Walsh and Dyadic Analysis, Elektronski fakultet, Niˇs, Serbia, 2008, 31-36, (ISBN 978-86-85195-47-1). 13. Stankovi´c, S., Astola, J., ”XML Framework for various types of decision diagrams for discrete functions”, IEICE - Transactions on Information and Systems, Vol. E90-D, No. 11, 2007, 1731-1740. 14. Stankovi´c, S., Astola, J.T., Miller, D.M., Stankovi´c, R.S., ”Heterogeneous decision diagrams for applications in Harmonic Analysis on finite non-Abelian groups”, Proc. 40th Int. Symp. on Multiple-Valued Logic, Barcelona, Spain, May 26-18, 2010, 307-312, IEEE-CS-Press, 2010. 15. Stankovi´c, R.S. Stankovi´c, M.S., Moraga, C., ”Remarks on systems and differential operators on groups”, Facta Universitatis (Niˇs), Ser. Elec. Energ., Vol. 18, No. 3, 2005, 531-545.