remarks on history of fft and related algorithms - CiteSeerX

REMARKS ON HISTORY OF FFT AND RELATED ALGORITHMS Radomir S. Stanković, Miomir S. Stanković1 , Karen Egiazarian2 , Leonid P. Yaroslavsky3 Dept. of Computer Science, Faculty of Electronics, 18 000 Niˇs, Serbia 1 Dept. of Mathematics, Faculty of Occupational Safety, Niˇs, Serbia 2 Tampere International Center for Signal Processing Tampere University of Technology, Tampere, Finland 3 Dept. of Interdisciplinary Studies, Faculty of Engineering Tel Aviv University, Tel Aviv 69978, Israel ABSTRACT This paper reviews history of work in development of FFT and FFT-like algorithms for calculation of the Discrete Fourier Transform (DFT), Walsh transform and Haar transform. 1

INTRODUCTION

Considerations related to efficient calculation of the Fourier series have been published by eminent authors [48], [49], [50], and the same problem has been considered and successfully solved much earlier, including Gauss as it is documented in [29], however, these results did not attract the due attention for the lack at that time of the corresponding hardware required for processing of large amounts of data. Cooley-Tukey algorithm [18], and related SandeTukey algorithm [25], [42], also denoted as algorithms with decimation in time and frequency, respectively, start the new era of fast calculation methods in Fourier analysis. Although in the time of publication in 1965, the Cooley-Tukey algorithm was accepted as completely new, in an answer to [18], Rudnick [47] described his algorithm for DFT of the length N = 2n with the number of operations proportional to N log2 N presented in [19]. This algorithm uses a method based on the work by Runge and Konig [48], [49], [50], that exploits the symmetry of sinus and cosinus functions, and a 2N dimensional transform converts into two N -dimensional transforms with a bit more than N operations. A continuation of this approach by splitting into half the related sequences of the length = 2m leads to m = log2 N fields where each field contains twice more sequences of twice smaller length. The final field will contain N/2 sequences each sequence with two elements. It is obvious that the number of operations in thus derived algorithm is proportional to N log2 N . Cooley-Tukey algorithm can be viewed as a generalization of this method to the case of an arbitrary nonprime N that is not restricted to powers of 2. For N = r1 r2 , r1 6= r2 , the algorithm reduces to the representation of the input sequence as an array of r1

columns and r2 rows and the calculation of two dimensional transforms with additional multiplication by twiddle factors between implementation of transforms with respect to particular dimensions. It is clear that for N = 2m the algorithm reduces to the previously described method of splitting the input sequences into subsequences with the half length. In [61], Thomas introduced an algorithm for calculation of Fourier series through a generalization of results by Stumpff [59], the efficiency of which is based on the replacement of one-dimensional transform by multidimensional transforms. An algorithm by Good [27] derived from a work devoted to the analysis and description of experiments [72], can be viewed as a generalization of the algorithm by Thomas. Good has shown that for a non-prime N = r1 r2 · · · rm with mutually prime factors, the single dimensional Fourier transform of the length N can be replaced by an m-dimensional transform of arrays of dimensions r1 × r2 × · · · × rm . However, it is incorrect to consider this algorithm equivalent to the Cooley-Tukey FFT algorithm. Differences can be summarized as follows [28] 1. In the Good algorithm factors of N must be mutually prime numbers. 2. In this case, steps are purely multidimensional transform without twiddle factors. 3. The correspondences between single and multiple dimensional indexing are completely different. 2

MATRIX REPRESENTATION OF FFT

Good’s work [27] is important in one more respect. It apparently was the first one where matrix representation of transforms and basic principles of factorization of the transform matrix were introduced. It was shown that an m-dimensional transform can be generated as the Kronecker product R1 ⊗ R2 ⊗ · · · ⊗ Rm of m 1-D square transform matrices {R1 , R2 , · · · , Rm } with rows and columns ordered lexicographically and that the Kronecker product of m matrices can be converted into the product of the same number of sparse matrices:

3 R1 ⊗ · · · ⊗ Rm =

m Y ¡ ¢ ISk+1 ·Sk+2 ····Sm ⊗ Rk ⊗ IS1 ·S2 ····Sk−1 k=1

where IS is the identity (S × S) matrix and Sk is the size of the k-th matrix Rk . In [60], Theilheimer has shown that DFT transform matrix F for N = r1 r2 can be represented as F = PQR, where R and Q are suitably defined block and block diagonal matrices, respectively, and P is an appropriately defined permutation matrix. This representation is the matrix form of the Sande-Tukey algorithm [25]. Since F and Q are symmetric matrices, a relation that corresponds to the Cooley-Tukey algorithm is given by F = RT QPT , where T denotes the transposition of a matrix. In [26], Glassman presented another factorization of the DFT matrix for N = r1 · · · rm into the product of sparse matrices F = F1 · · · Fm with suitably defined elements of each Fi , that leads to the so-called algorithms of fixed geometry. Some other factorizations of the DFT matrix can be found in the literature. For instance, the factorization in [14] and [40] are suitable for parallel realizations. In [70], [71] it was shown how different versions of FFT algorithms, including pruned ones, can be analytically obtained using matrix factorization technique and, on the example of ”Quantized DFT”, how new fast transforms can be generated. The factorization by Good of Kronecker product representable matrices matrices can be extended to matrices derived by the generalized Kronecker product [23]. This factorization is useful for extension to Kronecker product representable transform matrices in different algebraic structures. An extension to Fourier and Fourier like matrices on finite Abelian groups has been given in [6]. Generalizations to finite non-Abelian groups are also provided by suitably defined matrix operations [55]. In [68], [69] matrix representation was extended to the representation of entire family of fast transforms such as Walsh Transform, Haar Transform, Slant Transform and alike [1], [4]. It was shown that, using a notion of ”layered Kronecker matrices”, or row wise concatenated matrices that are Kronecker product of their constituent matrices, one can represent all known fast transforms in a unified way matrices, £ using, ¤ £as constituent ¤ 1 0 0 1 elementary matrices , , their combina£ ¤ £ ¤ tion 1 1 and 1 −1 , scalar multipliers, and operations of Kronecker matrix product and direct matrix sum. Good’s factorization theorem was extended to layered Kronecker matrices to enable analytical derivation of transform fast algorithms in their different modifications, including fast permutation algorithms and fast algorithms for matrix transposition [64].

RADER-BRENNER ALGORITHM

Calculation of DFT by FFT requires multiplication with complex numbers. It is shown in [44] that this can be avoided by a simple modification of the FFT algorithm with decimation in time. This operation is replaced by the multiplication with numbers where real or imaginary parts are different from zero but not both. The algorithm is called the Rader-Brenner algorithm [38]. Compared with FFT of radix 2 and base 4, the RaderBrenner algorithm requires fewer number of multiplications, but at about 10% more additions. A disadvantage of the algorithm is that although the weight coefficients wk have the amplitude equal to 1, the values of cosec(2πk/N ) can be large, and due to that, a relatively small calculation error produces a considerable error in the final result. In [15], a modification of the basic Rader-Brenner algorithm is presented that permits avoiding of this disadvantage. 4

WINOGRAD ALGORITHM

In [9], [10], calculation of DFT is reduced to the cyclic convolution by a development of algorithms for linear filtering, which permits releasing of the request that N must be a non-prime number of the form N = r1 · · · rm . Fundamentals of this class of algorithms are due to Rader [43], who had shown that when N is a prime number, a reordering of indices converts calculation of DFT into calculation of a cyclic convolution of the length N − 1, which can be performed by the application of the convolution theorem with application of the FFT algorithms. For reordering of indices, it is used a mapping that converts the multiplication of indices modulo N into addition of indices modulo N − 1. Winograd [67] proposed a generalization of the idea by Rader to reduce DFT to convolution and introduced a class of the so-called Winograd algorithms for DFT exploiting fast algorithms for calculation of cyclic convolutions with the minimum number of multiplications. His method exploits the property that calculation of the convolution {y(n)} of a sequence {x(n)} of the length L and a sequence {h(n)} of the length M is equivalent to the determination of the coefficients of a polynomial Y (r) that is the result of the multiplication of two polynomials H(r) and X(r) assigned in a suitable way to {x(n)}, {h(n)}, and {y(n)}. The Cook-Toom algorithm from theory of data structures is an efficient way to accomplish this task. The algorithm exploits the property that every polynomial of order N by the application of the Lagrange interpolation formulae can be exactly represented through its values at N + 1 points. A generalization of the Cook-Toom algorithm is proposed in [37]. Winograd had shown in [66] that the minimum number of multiplications required to calculate the expres-

sion X(r)H(r) modulo P (r) is 2N − k where k is the number of irreducible factors Pq (r) of the polynomial P (r) over a field F. The multiplication by fixed elements in F is not counted. If the coefficients of the polynomials X(r), H(r), and Y (r) are represented as vectors of N elements x = [x(0), · · · , x(N − 1)]T , h = [h(0), · · · , h(N − 1)]T , and y = [y(0), . . . , y(N − 1)]T , then every algorithm with the minimum number of multiplications has the form y = C(Bh · Ax), where · denotes the componentwise (Hadamard) product of vectors. The matrices A and B are of orders ((2N − k) × N ) and C is of order (N × (2N − k)) with elements in F. Calculation of the cyclic convolution by the convolution theorem for DFT is a special case of the algorithm. In this case, matrices A and B perform DFT, and C the inverse DFT. A nice feature is that in many practical approaches for a suitable choice of the field F, the elements of matrices A and B can be restricted to elements in the set {−1, 0, 1} and elements of C are rational numbers. 5

PRIME FACTOR ALGORITHMS

In prime factor algorithms (PFA) [31], the single dimensional DFT is converted by the Chinese remainder theorem into the multidimensional DFT by the Good algorithm, and then transforms of small dimensions are calculated by the corresponding Winograd algorithms. Therefore, these algorithms are referred to as the GoodWinograd (GW) algorithms [39]. Characteristics of PFA algorithms have been improved by some suitable modifications. A particular reordering of indices suggested in [11] converts PFA into a form that enables in-place calculations. The reordering can be efficiently realized by the Chinese remainder theorem. A disadvantage is that the structure of each module depends on the length of the transform N , so that depending on this value, a certain reordering of elements for each module should be performed. In [11], modules for specified values of N are generalized. Alternative is to include a set of control instructions to ensure these requirements [46]. Winograd-Fourier algorithms (WFTA) are based on a different convolution of basic algorithms of small length [53] by the Chinese remainder theorem and Kronecker product representable matrices. In [32], an algorithm is proposed to calculate the multidimensional Fourier transform by the fast Radon transform. 6

ALGORITHMS FOR WALSH AND HAAR TRANSFORMS

The term Kronecker product representable transforms denotes spectral transforms whose transform matrix can

be represented as the Kronecker product of some basic matrices. The Walsh-Hadamard transform, the ReedMuller and the arithmetic transform are examples of such spectral transforms for N = 2n . The VilenkinChrestenson transform, the Galois field transforms, and the Reed-Muller-Fourier transforms are extensions of Kronecker product representable transforms for N = pk , k ∈ N , N -the set of natural numbers. As it was already mentioned, if the transform matrix can be split into leyers that are Kronecker product representable, the transforms are denoted as the leyer Kronecker product representable. The Haar transform and its various generalizations are examples of the leyer Kronecker product representable transforms [70], [71]. From a historic point of view, the algorithm by Yates [72] derived in considerations related to the determination of variance in particular classes of experiments is the first algorithm of the fast Walsh transform. In discussions of fast algorithms for the Walsh transform, the algorithms for different orderings of the Walsh transform matrix should be distinguished. In [65], Whelchel and Guinn proposed a fast algorithm based on the recurrence relation for the Walsh matrix. The algorithm can be derived from this recurrence relation, and also through the Good-Thomas factorization of the Walsh matrix. The algorithm by Shanks [52] derived as a method between the factorization of the Walsh matrix and the Cooley-Tukey algorithm for DFT is equivalent to the Good-Thomas algorithm as has been pointed in [35]. Algorithms with calculation in-place and a reordering of input or output data are discussed in [7], [8], [13]. The application of a generalization of the Kronecker product and Chinese remainder theorem permitted a systematization of various algorithms for different orderings of the Walsh matrix [22]. A fast algorithm for direct generation of the sequency ordered Walsh spectrum without reordering has been presented in [41]. An alternative version of this algorithm that does not start with calculations over neighboring elements is defined by Ulman [63]. This algorithm is suitable for calculation of the so-called Rtransform [45] which is invariant to the cyclic shift. An algorithm for sequency ordered Walsh spectrum has been proposed by Manz [36] as a counterpart of the Cooley-Tukey algorithm. In this algorithm, the input data are in the bit-reverse order. In every second group of data, starting from the second step of the algorithm, operations of additions and subtractions are permuted. An algorithm complemented to that of Manz in the sense that the input data are in the initial order, while the output spectral coefficients are in sequency ordered is proposed in [34]. The algorithms is particularly convenient in such applications as, for example, Walsh spectral analysis of stochastic data. The Haar transform can be calculated by the modification of the Cooley-Tukey algorithm, with the modi-

fication related to exploiting zero elements in the Haar matrix [2], [3]. An algorithm different from the Cooley-Tukey algorithm has been proposed for the Haar transform in [5]. The algorithm exploits factorization into diagonal and block diagonal factor matrices. As noted above, various matrix representations of these algorithms can be derived by the methods described in [68], [69]. 7

DECISION DIAGRAM METHODS

In the last decade, methods have been proposed to calculate spectral transforms through decision diagrams (DDs). These methods were introduced in [16], and [17] by the examples of the Walsh and the Reed-Muller transforms. The algorithm exploits the recurrence relation for the Walsh and the Reed-Muller transforms. Calculations of Walsh transform and related transforms by referring to the same recurrence relations but over some other types of DDs as for example, EdgeValued BDDs, is possible and was considered in several publications, for instance in [33], [51]. In these papers, algorithms are derived referring to the recurrence relation without noticing that this relation originates in the Kronecker product structure of the related transform matrices. If this is noticed, the algorithms can be viewed as implementations of FFT over DDs instead over vectors, see for example [57]. The observation that in DD-methods for Kroncker product representable spectral transforms we are actually performing FFT over DDs can be formulated in the simplest way as follows [56]. At each non terminal node and the cross point we perform the calculations defined by the basic transform matrix that is use din definition of the spectral transform considered. This statement holds generally for all Kronecker product representable transforms on finite not necessarily Abelian groups. However, it can be extended as well to layer Kronecker product representable transforms. For example, in calculation of the Haar spectrum, it is enough to perform the basic Haar matrix, which is actually the basic Walsh matrix, in the leftmost nodes at each level in the decision diagram for the function processed, providing that the labels at the edges of nodes are consistently assigned over levels. This can be seen from either definition of the Haar matrix or the structure of the FFT-like algorithm for the Haar transform [58]. This interpretation explains an approach that was exploited in formulation of an algorithm for calculation of the Haar transform through decision diagrams [54]. There are other algorithms to calculate Haar spectrum trough decision diagrams, for instance, [12], [20], [62]. However, these algorithms are based on the recurrence relations for the Haar transform similar to the

decision diagram methods for the Walsh and the ReedMuller transforms. Since these algorithms do not exploit the discussed relationships between the properties of the Haar functions and decision diagrams, this reduce their efficiency in terms of time. Due to the above mentioned observation, an efficient method to calculate pairs of Walsh spectral coefficients has been proposed in [30]. For the calculation of particular Walsh coefficients separately at the time, in [24] is is suggested a method that reduces to the multiplication of the decision diagram for the Walsh function in respect to which the spectral coefficient is required with the diagram for the function processed. In [21], the opposite task of synthesis of decision diagrams from the Walsh spectrum was considered. In FFT and related algorithms, functions whose spectra should be calculated are represented by vectors of function values. It follows that the space and time complexity of such algorithms is equal for all functions represented by vectors of a given length. In graphical representations of discrete functions, vectors of function values correspond to the decision trees. Decision diagrams are derived by reducing decision trees. The reduction is possible when the functions represented have some peculiar properties as for example, symmetry, decomposability with respect to a subset of variables, many equal or constant subvectors, etc. Due to that, decision diagrams may provide more compact representations than other methods of representing discrete functions. It follows that decision diagram methods permit to calculate efficiently in terms of space and time spectra of functions represented by huge vectors when FFT and related algorithms with calculations over vectors cannot be performed at the available hardware. References [1] Agaian, S., Astola, J., Egiazarian, K., Binary Polynomial Transforms and Nonlinear Digital Filters, Marcel Dekker, 1995. [2] Ahmed, N., Natarajan, T., ”Cooley-Tukey type algorithm for the Haar transform”, Elect. Lett., Vol. 9, No. 12, 276-278, June 1973. [3] Ahmed, N., Natarajan, T., Rao, K.R., ”Some considerations of the Modified Walsh-Hadamard and Haar transforms”, Proc., Symp. Applications of Walsh Functions, 1973, 91-95. [4] Ahmed, N., Rao, K. R., Orthogonal Transforms for Digital Signal Processing, Springer-Verlag, Berlin, 1975. [5] Andrews, H.C., Caspari, K.L., ”A generalized technique for spectral analysis”, IEEE Trans. on Compuuters, Vol. C-19, No. l, Jan 1970, 16-25. [6] Aple, G., Wintz, D., ”Calculation of Fourier transform on finite Abelian groups”, IEEE Trans., Vol. IT-16, 1970, 233-234. [7] Berauer, G., ”Fast in-place computation of the discrete Walsh transform in sequence order”, Proc. Applications of Walsh Functions, Washington, D.C., 1972, 272-275.

[8] Bhagavan, B.K., Polge, R.J., ”Sequencing the Hadamard transform”, IEEE Trans., Vol. AU-21, Oct. 1973, 472-473. [9] Bluestein, L.I., ”A linear filtering approach to the computation of the discrete Fourier transform”, Northeast Electronics Research and Engineering Meeting Rec., 1968, 218-219. [10] Bluestein, L.I., ”A linear filtering approach to the computation of the discrete Fourier transform”, IEEE Trans., Vol. AU-18, 1970, 451-455. [11] Burrus, C.S., Eschenbacher, P.W., ”An In-place, Inorder prime factor FFT algorithm”, IEEE Trans. Acoust., Speech, Signal Processing, Vol. ASSP-29, No. 4, Aug. 1981, 806-816. [12] Chang, C.H., Falkowski, B.J., ”Haar spectra based entropy approach to quasi-minimization of FBDDs”, IEE Proc. Comput. Digit. Tech., Vol. 146, No. 1, 1999, 4149. [13] Cheng, D.K., Liu, J.J., ”An algorithm for sequency ordering of Hadamard functions”, IEEE Trans.Comput., March 1977, 308-309. [14] Chorinthios, M.J., ”The design of a class of fast Fourier transform computers”, IEEE Trans. on Computers, Vol. C-20, No. 6, June 1971, 617-623. [15] Cho, K.M., Temes, G.C., ”Real-factor FFT algorithms”, IEEE Intern. Conf. Acoust, Speech, Signal Processing, 1978, 634-637. [16] Clarke, E.M., McMillan, K.L., Zhao, X., Fujita, M., Yang, J., “Spectral transforms for large Boolean functions with application to technology mapping”, DAC1993, June 1993. [17] Clarke, E.M., Mc Millan, K.L., Zhao, X., Fujita, M., “Spectral transforms for extremely large Boolean functions”, in Kebschull, U., Schubert, E., Rosenstiel, W., Eds., Proc. IFIP WG 10.5 Workshop on Applications of the Reed-Muller Expression in Circuit Design, Hamburg, Germany, Reed-Muller Workshop, RM-1993, September 16-17, 1993, 86-90. [18] Cooley, J.W., Tukey, J.W., ”An algorithm for the machine calculation of complex Fourier series”, Math. of Comput., Vol. 19, April 1965, 297- 301. [19] Danielson, G.C., Lanczos, C., ”Some improvements in practical Fourier analysis and their application to Xray scattering from liquids”, J. Franklin Inst., Vol. 233, April 1942, 365-380, and 435-452. [20] Falkowski, B.J., Chang, C.H., ”Forward and inverse transformations between Haar spectra and ordered binary decision diagrams of Boolean functions”, IEEE Trans. on Computers, Vol. 46, No. 11, 1997, 1272-1279. [21] Falkowski, B.J., Chang, C.H., ”Efficient algorithms for the calcualtion of Walsh spectrum from OBDD and synthesis os OBDD form Walsh spectrum”, proc. 37th Midwes Symp. Circuits and Systems, August 3-5, 1994, Vol. 1, 393-396. [22] Fino, B.J., Algazi, V.R., ”Unified matrix treatment of the fast Walsh-Hadamard transform”, IEEE Trans. on Computers, Vol. C-25, Nov. 1976, 1142-1146. [23] Fino, B.J., Algazi, V.R., ”A unified treatement of discrete fast unitary transforms”, SIAM J. on Comput., Vol. 6, Dec. 1977, 700-717.

[24] Fujita, M., Yang, J.C.-H., Clarke, E.M., Zhao, X., McGeer, P., ”Fast spectrum computation for logic functions usign binary decision diagrams”, ISCAS-94, 1994, 275-278. [25] Gentleman, W.M., Sande, G., ”Fast Fourier transforms for fun and profit”, Fall Joint Computer Conf. AFIPS Proc., Vol. 29, Washington D.C., Spartan, 1966, 563578. [26] Glassman, J.A., ”A generalisation of the Fast Fourier Transform”, IEEE Trans. on Computers, Vol. C-19, No. 2, Feb. 1970, 105-111. [27] Good, I.J., ”The interaction algorithm and practical Fourier analysis”, J. Roy. Statist. Soc., ser. B, Vol. 20, 1958, 361-372, Addendum, Vol. 22, 1960, 372-375. [28] Good, I.J., ”The relationship between two Fast Fourier Transforms”, IEEE Trans. on Computers, Vol. C-20, No. 3, March 1970, 310-317. [29] Heideman, M.T., Johnson, D.H., Burrus, C.S., ”Gauss and the history of the fast Fourier transform”, Archiv for History of Exact Science, Vol. 34, No. 3, 1985, 265277, also in: IEEE ASSP Magazine, Vol. 1, No. 4, 1984, 14-21. [30] Janković, D., Stanković, R.S., Drechsler, R., ”Decision diagram method for calculation of pruned Walsh transform”, IEEE Trans. on Computers, Vol. 50, No. 2, 2001, 147-157. [31] Kolba, D.P., Parks, T.W., ”A prime factor FFT algorithm using high speed convolution”, IEEE Trans. Acoust., Speech, Signal Processing, Vol. ASSP-25, Aug. 1977, 281-294. [32] Labunets, V.G., ”Fast multidimensional Fourier transform based on the fast Radon transform”, Digital Methods in Control, Radiolocation and Communications, Uralsk Polytechnich Institute (UPI) by the name of S.M. Kirov Publishing, 1986, 152-162. [33] Lai, Y.F., Pedram, M., Vrudhula, S.B.K., “EVBDDbased algorithms for integer linear programming, spectral transformation, and functional decomposition”, IEEE Trans. Computer-Aided Design of Integrated Circuits and Systems, Vol. 13, No. 8, 1994, 959-975. [34] Larsen, H., ”An algorithm to compute the sequency ordered Walsh transform”, IEEE Trans., Vol. ASSP, Aug. 1976, 335-336. [35] Lechner, R.J., ”Comment on Computation of the Fast Walsh-Fourier transform”, IEEE Trans. on Computers, Vol. C-19, Feb. 1970, l74. [36] Manz, J.W., ”A sequency ordered fast Walsh transform”, IEEE Trans., Vol. AU-20, No. 3, 1972, 204-205. [37] McClellan, J.H., Rader, C.M., Number Theory in Digital Signal Processing, Prentice-Hall, Inc., Englewood Cliffs, New Jersey, 1979. [38] Nussbaumer, H.J., Fast Fourier Transform and Convolution Algorithms, Springer-Verlag, Berlin, Heidelberg, New York, 1981. [39] Patterson, R.W., McCllelan, J.H., ”Fixed-point error analysis of Winograd-Fourier transform algorithms”, IEEE Trans. Acoust., Speech, Signal Processing, Vol. ASSP-26, No. 5, Oct. 1978, 447 -455. [40] Pease, M.C., ”An adaptation of the fast Fourier transform for parallel processing”, J. Ass. Comput. Mach., Vol. 15, April 1968, 252-264.

[41] Pratt, W.K., Kane, J., Andrews, H.C., ”Hadamard transform image coding”, Proc. IEEE, 57, 1969, 58-68. [42] Rabiner, L.R., Gold, B., Theory and Applications of Digital Signal Processing, Englewood Cliffs, N.J., Prentice-Hall, 1975. [43] Rader, C.M., ”Discrete Fourier transforms when the number of data samples is prime”, Proc. IEEE, Vol. 5, No. 6, June 1968, 1107-1108. [44] Rader,C.M., Brenner, N.M., ”A new principle for fast Fourier transformation”, IEEE Trans. Acoust. Speech. Signal Processing, Vol. ASSP-24, June 1976, 264-266. [45] Reitboeck, H., Brody, T.P., ”A transformation with invariance under cyclic permutation for applications in pattern recognition”, Westinghouse Research Labs., Scientific Paper 68-1F1-ADAPT-P1, Sept. 17, 1968. [46] Rothweiler, J.H., ”Implementation of the in-order prime factor transform for variable sizes”, IEEE Trans. Acoust., Speech, Signal Processing, Vol. ASSP-30, No. 1, Feb. 1982, 105-107. [47] Rudnick, P., ”Note on the calculation of Fourier series”, Math. of Comput., Vol. 20, July 1966, 429-430. [48] Runge, C., ”Uber die Zerlegung empirisch periodischer Funktionen in Sinnus-wellen”, Zeit. f¨ ur Math, und Physik, Vol. 48, 443-456, 1903. [49] Runge, C., Zeit f¨ ur Math, und Physik, Vol. 53, 1-17, 1905. [50] Runge, C., Konig, H., ”Die Grundlehren der Mathematischen Wissenschaften”, Vorlesungen u ¨ber Numerisches Rechnen, Vol. 11 Berlin, Julius Springer, 1924. [51] Sasao, T., Fujita, M.,(eds.), Reresentations of Discrete Functions, Kluwer Academic Publishers, 1996. [52] Shanks, J.L., ”Computation of the fast Walsh-Fourier transform”, IEEE Trans. on Computers, Vol. C-18, May 1969, 457-459. [53] Silverman, H.F., ”An introduction to programming the Winograd Fourier transform algorithm (WFTA)”, IEEE Trans. Acoust., Speech, Signal Processing, Vol. ASSP-25, No. 2, April 1977, 152-165. [54] Stanković, M., Janković, D., Stanković, R.S., “Efficient algorithms for Haar spectrum calculation”, Scientific Review, No. 21-22, 1996, 171-182. [55] Stanković, R.S., “Matrix interpretation of the fast Fourier transform on finite non-Abelian groups”, Proc. Int. Conf. on Signal Processing/ Beijing’90, October 24.-26., 1990, Beijing, P.R. China, 1187-1190. [56] Stanković, R.S., Astola, J.T., Spectral Interpretation of Decision Diagrams, Springer, 2003. [57] Stanković, R.S., Falkowski, B.J., ”Spectral transform calculation trough decision diagrams”, VLSI Design, Special Issue on Spectral Techniques and Decision Diagrams, Vol. 14, No. 1, 2002, 5-12. [58] Stanković, R.S., Stanković, M., Moraga, C., “Design of Haar wavelet transforms and Haar spectral transform decision diagrams for multiple-valued functions”, Proc. 31st Int. Symp. on Multiple-Valued Logic, Warzsawa, Poland, May 22-24, 2001, 311-316. [59] Stumpff, K., Tafeln und Aufgaben zur Harmonischer Analyse und Periodogrammrechnung, Berlin: Julius Springer, 1939.

[60] Theilheimer, F., ”A matrix version of the fast Fourier transform”, IEEE Trans., Vol. AU-17, No. 2, June 1969, 158-161. [61] Thomas, L.H., ”Using a computer to solve problems in physics”, in Application of Digital Computers, Boston, Mass., Ginn, 1963. [62] Thornton, M.A., Drechsler, R., Gnther, “Probabilistic equivalence checking using partial Haar spectral diagrams”, Proc. 4th Int. Workshop on Applications of the Reed-Muller Expansion in Circuit Design, August 2021, 1999, Victoria, B.C., Victoria, Canada, 123-132. [63] Ulman, L.J., ”Computation of the Hadamard transform and the R transform in ordered form”, IEEE Trans., C19 , 1970, 359-360. [64] Vitkus, R.Yu., Yaroslavsky, L.P., ”Matrix Representation of the Algorithms for Transposing of Large Matrices”, Radioelectronics and Communication Systems, No.12, 1987, 14-20. [65] Whelchel, J.E., Guinn, D.F., ”The fast FourierHadamard Transform and its use in signal representation and classification”, Tech. Report. PRC 68-11, Melpar. Inc. Falls Church, Va 22046, 1968. [66] Winograd, S., ”Some bilinear forms whose multiplicative complexity depends on the field of constants”, Mathematical Systems Theory, Vol. 10, 1977, 69-180. [67] Winograd, S., ”On computing the discrete Fourier transform”, Mathematics of Computation, Vol. 32, Jan. 1978, 75-199. [68] Yaroslavsky, L.P., An Introduction to Digital Picture Processing, Sov. Radio Publ., Moscow, 1979. [69] Yaroslavsky, L.P., FDigital Picture Processing, Introduction, Springer, Heidelberg, 1985. [70] Yaroslavsky, L.P., Merzlyakov, N.S., Methods of Digital Holography, Moscow, Nauka, 1977. [71] Yaroslavsky, L.P., Merzlyakov, N.S., Methods of Digital Holography, Plenum, 1980. [72] Yates, F., The Design and Analysis of Factorial Experiments, Harpenden: Imperial Bureau of Soil Science, 1937.

8

Addendum

There are many monographs considering various applications of Fourier analysis and spectral methods. The following list reviews monographs where discrete transforms are the main subject. 1. Agaev, G., Vilenkin, Ya.N., Dzhafarli, G.M., Rubinshtein, A.I., Mul’tiplikativnye sistemy funkcii i garmonicheskii analiz na nul’-mernykh gruppakh, Baku, Elm, 1981. 2. Agaian, S., Astola, J., Egiazarian, K., Binary Polynomial Transforms and Nonlinear Digital Filters, Marcel Dekker, 1995. 3. Ahmed, N., Rao, K. R., Orthogonal Transforms for Digital Signal Processing, Springer-Verlag, Berlin, 1975. 4. Akhmed, N., Rao, K.R., Ortogonal’ nye preobrazovaniya dlya cifrovoi obrabotki signalov, Svyaz’, Moskva, 1980, prevod sa en-gleskog. 5. Beauchamp, K.G., Walsh Functions and Their Applications, Academic Press, New York, 1975.

6. Beauchamp. K.G., Transforms for Engineers, Oxford University Press, Oxford, 1987. 7. K.G. Beauchamp, Applications of Walsh and Related Functions with an Introduction to Sequency Theory, Academic Press, New York, 1984. 8. Beauchamp, K.G., Yuen, C.K., Digital Methods for Signal Analysis, Georg Allen and Unwin, London, 1973. 9. Besslich, Ph. W., Lu, T., Diskrete Orthogonaltransformationen, Springer-Verlag, Berlin, 1990. 10. Beth, Th., Verfahren der schnellen Fourier Transformation, Teubner Verlag, Berlin, 1984. 11. Blahut, R.E., Fast Algorithms for Digital Signal Processing, Reading, M A Addison-Wesley, 1984. 12. Bleikhut, R., Bystrie algortimy cifrovoi obrabotki signalov, Mir, Moskva, 1989. 13. Bracewell, R.N., The Fourier Transform and its Applications, McGraw-Hill, New York, 1978. 14. Bracewell, R.N., The Hartley Transform, Oxford University Press, Oxford, 1985. 15. Brigham, E.O., The Fast Fourier Transform, PrenticeHall, Englewood Cliffs, New Jersey, 1974. 16. Burrus, C.S., Parks, T.W., DFT/FFT and Convolution Algorithms, John Wiley, New York, 1985. 17. Butzer, P.L., Stanković, R.S., Theory and Applications of Gibbs Derivatives, Matematiˇcki institut, Beograd, 1990. 18. Creutzburg, R., Ed., Selected Topics in Number Theoretic Transforms, ZKI Informationen 2, Akademie der Wissenschaften der DDR, 1988. 19. Cadykhov, P.Kh., Chegolin, P.M., Shmerko, V.P., Metody i sredstva obrabotki signalov v diskretnykh bazisakh, Navuka i tekhnika, Minsk, 1987. ˇ zek, V., Discrete Fourier Transforms and their Ap20. Ciˇ plications, Adam Hilger, Bristol, 1986. 21. Dagman, E.E., Kukharev, G.A., Bystrye diskretnye orthogonal’ nye preobrazovaniya, Nauka, Sibirskoe otdelenie, Novosibirsk, 1983. 22. Dyaglov, A.P., Evdokimov, Yu, F., Bystroe korrelacionoe preobrazovanie, Izd-vo Rostovskogo un-ta, 1983. 23. Eliot, D.F., Rao, K.R., Fast Transforms, Algorithms, Analysis, Applications, Academic Press, New York, 1982. 24. Evteev, Yu.I., Kushchev, B.I., Pikulin, V.S., i dr., Apparaturnaya realizacya diskretnogo preobrazovanya Fur’e, Energiya, Moskva, 1978. 25. Fukui, I., Study of The Properties of The Walsh Transform and its Applications, Technical Report of Mechanical Engineering Laboratory, No.124, Namiki, Sakuramura, Niihari-gun, Ibaraki, Japan, 1982, in Japanese, English summary. 26. Golubov, B.I., Efimov, A.V., Skvorcov, V.A., Ryady i preobrazovaniya Uolsha, teoriya i primeneniya, Nauka, Moskva, 1987.

27. Grigoryan, A.M., Agaian, S.S., Multidimensional Discrete Unitary Transformations: Representation, Partitioning, and Algorithms, Marcel Dekker, New York, 2003. 28. Harmuth, H.F., Transmission of Information by Orthogonal Functions, Springer-Verlag, Berlin, 1972. 29. Kharmuth, Kh., F., Peredacha informacii ortogonal’nym signalami, Svyaz’ , Moskva, 1975. 30. Harmuth, H.F., Sequency Theory, Foundations and Applications, Academic Press, New York, 1977. 31. Kharmuth, Kh., F., Teoriya sekventivnogo analiza, Osnovy i primeneniya, Mir, Moskva, 1980, prevod sa engleskog. 32. Huang, T.S., Ed., Two-dimensional Digital Signal Processing II, Transforms and Median Filters, SpringerVerlag, Berlin, 1981. 33. Khuang, T.S., Bystrye algoritmy v tsifrovoi obrabotke izobrazhenii, preobrazovanya i mediannye fil’try, Radio i Svyaz’, Moskva, 1984. 34. Hurst, S.L., The Logical Processing of Digital Signals, Crane Russak and Edward Arnold, England, 1978. 35. Hurst, S.L., Miller, D.M., Muzio, J.C., Spectral Techniques in Logic Synthesis, Academic Press, Toronto, 1985. 36. Karpovsky, M.G., Finite Orthogonal Series in the Design of Digital Devices, Wiley, New York and JUP, Jerusalem, 1976. 37. Karpovsky, M.G., (Ed.), Spectral Techniques and Fault Detection, Academic Press, Orlando, FL, 1985. 38. Karpovsky, M.G., Moskalev, E.S., Spektral’nye melody analiza i sinteza diskretnykh ustroistv, Energiya, Leningrad, 1973. 39. Krot, A.M., Diskretnye modeli dinamicheskikh sistem no osnove polinomial’noi alhgebry, Navuka i tekhnika, Minsk, 1990. 40. Labunets, V.G., Algebricheskaya teoriya signalov i sistem, tsifrovoya obrabotka signalov, Izd-vo Krasnoyarskogo un-ta, 1984. 41. McClellan, J.H., Rader, C.M., Number Theory in Digital Signal Processing, Prentice-Hall, Englewood Cliffs, New Jersey, 1979. 42. Makklellan, Dzh., Kh., Reider, Ch.M., Primenenie teorii chisel v cifrovoi obraqbotke signalov, Radio i svyaz’, Moskva, 1983, Translation from English. 43. Moraga, C., Spectral Methods in Logic Design, Interne berichte und skripten an der abteilung Informatik der Universitat Dortmund, Dortmund, 1987. 44. Maqusi, M., Applied Walsh Analysis, Heyden, London, 1981. 45. Nussbaumer, H.J., Fast Fourier Transforms and Convolution Algorithms, Springer-Verlag, New York, 1981. 46. Nussbaumer, G.Dzh., Bystroe preobrazovanie Fur’e i algoritmy vychishleniya svertok, Radio i svyaz’ , Moskva, 1984. 47. Poida, V.N., Spektral’nyi analiz v diskretnykh ortogonal’nykh bazisakh, Navuka i tekhnika, Minsk, 1978.

48. Rao, K.R., Ed., Discrete Transforms and Their Applications, Van Nostrand Reinhold Company, New York, 1985. 49. Rao, K.R., Yip, P., Discrete Cosine Transform, Algorithms, Advantages, Applications, Academic Press, New York, 1990. 50. Siddiqi, A.H., Walsh Functions, Dept. Math., Aligarh Muslim Univ., Aligarh 202001, India, 1977. 51. Sobol, I.M., Mnogomernye kvadraturnye formuly i funkcii Khaara, Nauka, Moskva, 1969. 52. Solodovnikov, A.I., Spivakovski, A.M., Fundamentals of Theory and Methods of Spectral Processing of Information, Leningrad University Publishing, Leningrad, 1986. 53. Stanković, R.S., Spectral Transform Decision Diagrams in Simple Questions and Simple Answers, Nauka, Belgrade, 199 54. Stanković, R.S., Astola, J.T., Spectral Interpretation of Decision Diagrams, Springer, 2003. 55. Stanković, R.S., Stanković, M., Janković, D., Spectral Transforms in Switching Theory, Nauka, Belgrade, 1998. 56. Stanković, R.S., Stojić, M.R., Bogdanović, S.M., Fourierovo predstavljanje signala, (Fourier Representation of Signals), Nauˇcna knjiga, Beograd, 1989. 57. Stanković, R.S., Stojić, M.R., Stanković, M.S., Recent Developments in Abstract Harmonic Analysis with Applications in Signal Processing, Nauka and Elektronski fakultet, Belgrade and Niˇs, 1995.

64. Vlasenko, V.A., Lappa, Ya. M., Yaroslavski, L.P., Methods of Synthesis of Fast Algorithms for Convolution and Spectral Analysis of Signals, Nauka, Moscow, 1990. 65. Weixing Deng, Weiyi Su, Fuxian Ren, Theory and Applications of Walsh Functions, Science and Techniques Press, Shanghai, China, 1983, (in Chinese). 66. Yaroslavski, P.L., Vvedenie v cifrovuyu obrabotku izobrazhenii, Sovetskoe radio, Moskva, 1979. 67. Yaroslavski, L.P., Digital Signal Processing in Optics and Holography, Introduction to Digital Optics, Radio i Svyaz, Moscow, 1987. 68. Yaroslavky, L.P., Eden, M., Fundamentals of Digital Optics, Birkhauser, Boston, 1996. 69. Zadiraka, V.K., Teoriya vychisleniya preobrazovaniya Fur’ e, Naukova dumka, Kiev, 1983. 70. Zalmazon, L.A., Preobrazovaniya Fur’e, Uolsha, Khaara i ikh primenenie v upravlenii, svyazi i drugikh oblastyakh, Nauka, Moskva, 1989. 71. Zhang Gongli, Pan Ailing, Theory and Applications of Digital Spectral Techniques, National Defence Industry Press, Beijing, 1992, in Chinese. 72. Bass, C.A., ed., Proc. Symp. Applic. Walsh Functions, 1970 73. Zeek, R.W. Showalter, A.E., Eds., Proc. Symp. Applic. Walsh Functions, 1971. 74. Zeek, R.W. Showalter, A.E., Eds., Proc. Symp. Applic. Walsh Functions, 1972.

58. Stojić, M.R., Stanković, M.S., Stanković, R.S., Diskretne transformacije u primeni, (Discrete Transforms in Application), Nauˇcna knjiga, Beograd, 1985, first edition, 1993 second extended and up-dated edition, (in Serbian).

75. Zeek, R.W. Showalter, A.E., Eds., Proc. Symp. Applic. Walsh Functions, 1973

59. Tolimieri, R., An, M., Lu, Ch., Algorithms for Discrete Fourier Transform and Convolution, Springer, New York, 1997.

77. Proc. Int. Symp. Walsh and Other Non-sinusoidal Functions Applications, Hatfield Polytechnic, England, 1971, 1973, 1975.

60. Tolimieri, R., An, M., LU, Ch., Burrus, C.S., Mathematics for Multidimensional Fourier Transform Algorithms, Springer, New York, 1997.

78. Brahmall, J.N., An annotated bibliography on Walsh and Walsh related functions, John Hopkins Univ., Applied Physics Lab., Tech. Memoir TG 1198B, Baltimore, Maryland, 1974. National Technical Information Service Data Base, Walsh Functions 91964-1982), PB83-800938 U.S. National Technical Information Service, Springfield, Virginia.

61. Thornton, M.A., Drechsler, R., Miller, D.M., Spectral Techniques in VLSI CAD, Kluwer Academic Publishers, 2001. 62. Trakhtman, A.M., Vvedenie v obobshchenuyu spektral’nuyu teoriyu signalov, Sovetskoe radio, Moskva, 1972. 63. Trakhtman, A.M., Trakhtman, V.A., Osnovy teorii diskretnykh sig-nalov na konechnykh intervalakh, Sovetskoe radio, Moskva, 1975.

76. Schreiber, H., Sandy, G.P., Eds., Proc. Symp. Applic. Walsh Functions, 1974.

79. Heidtman, M.T., Burrus, C.S., A bibliography of fast Fourier transform and convolution algorithms, II, Dept. of Electrical Engineering Technical Report, No. 8402, Rice University, Houston, Tx 77251, 1984.