SOFTWARE IMPLEMENTATION OF MODIFICATIONS OF ... - ijpam

International Journal of Pure and Applied Mathematics Volume 76 No. 4 2012, 489-500 ISSN: 1311-8080 (printed version) url: http://www.ijpam.eu

AP ijpam.eu

SOFTWARE IMPLEMENTATION OF MODIFICATIONS OF ITERATIVE ALGORITHMS FOR SOLVING LINEAR SYSTEMS OF EQUATIONS Angel Golev1 § , Anna Malinova2 , Desislava Zaharieva3 1,2,3 Faculty

of Mathematics and Informatics University of Plovdiv 236, Bulgaria Blvd., Plovdiv, 4003, BULGARIA

Abstract: This paper presents a software package developed in Mathematica that contains implementations of various, mostly proposed during the last three years, iterative algorithms, including combined algorithms, for solving linear systems of equations. The accent falls on algorithms of type successive overrelaxation which are modifications of the Gauss-Seidel algorithm (forward and reverse iteration), as well as newer modifications applicable to systems with iterative M -matrices. The package provides the algorithms’ implementations, a set of functions that perform convergence analysis using criteria described in literature, and additional utility functions. Described are the iterative algorithms and the software implementation. A numerical example demonstrating the use of the developed package is also given. AMS Subject Classification: 65F10 Key Words: solving linear systems of equations, iterative algorithms, Nekrassov-Mehmke algorithm (forward and reverse iteration), generalized Nekrassov-Mehmke algorithms, modifications of the Nekrassov-Mehmke algorithm, generalized accelerated over relaxation algorithms based on Nekrassov-Mehmke scheme, M-matrices, Mathematica package, parameterized template, code generation, rule-based programming, patterns, help documentation Received:

December 14, 2011

§ Correspondence

author

c 2012 Academic Publications, Ltd.

url: www.acadpubl.eu

490

A. Golev, A. Malinova, D. Zaharieva 1. Introduction

The recent paper presents a software package developed in Mathematica that contains implementations of various, mostly proposed during the last three years, iterative algorithms, including combined algorithms, for solving linear systems of equations. The accent falls on algorithms of type successive overrelaxation which are modifications of the Gauss-Seidel algorithm (forward and reverse iteration), as well as newer modifications applicable to systems with iterative M -matrices. Implemented are a number of algorithms (classical and their new modifications), a set of functions that perform convergence analysis using criteria described in literature, and additional utility functions providing functionality not included in the Mathematica standard packages. Our approach to begin the development of this software package from scratch is a result of the fact that most of the implemented algorithms, being various modifications of classical ones, are nonstandard and thus not provided as built-in Mathematica functions [13, 16]. At the same time no legacy software developed in other languages and providing the same functionality was found in our research work. The existence of legacy software would give us the option to seek solutions for modernizing it in order to reuse it, for instance provide wrappers of the legacy software, as is presented in [7]. Another approach is described in [9, 10], where the implementation of a parametric residual iteration for linear interval systems by advanced MathLink communication between the system Mathematica and the library C-XCS supporting rigorous complex interval arithmetic is discussed. This implementation demonstrates the embedding of higher-level C-SCS functions of complex interval arguments into Mahtematica. The newly developed in the last two years iterative algorithms, based on classical algorithms and their modifications (see [4, 6, 15]), can, using the developed package, be easily implemented and added to the package, which we hope would increase the interest in practical and theoretical research in the field of computer algebra. Further, the paper structure is as follows: section 2 describes in short the algorithms that have been implemented; section 3 presents the software implementation; in section 4 is given a numerical example and concluding remarks are given in section 5.

SOFTWARE IMPLEMENTATION OF MODIFICATIONS OF...

491

2. Iterative Algorithms in Focus The first task in front of us was to collect a set of new iterative algorithms for solving linear systems of equations in a single software package, which would become a basis for future research, development and obtaining of new algorithms and techniques. At this stage of our work, the object of implementation were both classical algorithms and their modifications, proposed and considered by our team or other research groups in the last few years. The detailed description of the iterative algorithms is beyond the scope of this paper and is discussed in a series of other publications referenced below. The focus of our work up until now have been iterative overrelaxation algorithms for solving linear systems of equations, which are modifications of the Gauss-Seidel algorithm (forward and reverse iteration), as well as some newer modifications, applicable to systems with iteration M -matrices. Special attention is paid to the decomposition A = Tm − Em − Fm , where Tm is a banded matrix of bandwidth 2m+1, defined as [1, 3]: ( aij , |i − j| ≤ m , tij = 0, otherwise where the matrices E and F are the following  −am+2,1  .. .. Em =  . . −an,1

Fm



 =

···

−a1,m+2 · · · .. .



−an,n−m−1 −a1,n .. .

−an−m−1,n

 ,

(1)



 .

The iterative algorithms implemented in the developed software package are proposed or considered in [1, 3, 5, 8, 11, 12, 17, 18, 19] where descriptions of the algorithms and a number of corresponding convergence theorems are provided. In brief, the main iterative algorithms covered until now are listed as follows: 1. Nekrassov-Mehmke iteration scheme (or Gauss-Seidel scheme), (see Mehmke and Nekrassov [8]), referenced further as NM1: =− xk+1 i

i−1 X aij j=1

aii

− xk+1 j

n X aij k bi x + , i = 1, 2, . . . , n; k = 0, 1, 2, . . . . (2) aii j aii

j=i+1

492

A. Golev, A. Malinova, D. Zaharieva

2. Modification of the Nekrassov-Mehmke algorithm (see Faddeev, D. and Faddeeva, V. [1]), referenced further as NM2: =− xk+1 i

i−1 X aij j=1

aii

xkj −

n X aij k+1 bi + x , aii j aii

j=i+1

(3)

i = n, n−1, . . . , 1; k = 0, 1, 2, . . . . 3. Modification of the Nekrassov-Mehmke algorithm NM1 (see Iliev, Kyurkchiev and Petkov [3]):   n i−1 X X 1 aij xkj − bi  , aij xk+1 + aii xki + xk+1 = xki − k  j i Ni (4) j=i+1 j=1 i = 1, 2, . . . , n; k = 0, 1, . . . ,

where Nik = max

 

aii ,



i−1 Y

(xki −xk+1 ) j

n Y

  (xki −xkj ) , i = 1, 2, . . . , n; k = 0, 1, 2, . . . 

j=i+1

j=1

and the sign of the product is defined as equal to the sign of aii . 4. Modification of the Nekrassov-Mehmke algorithm NM2 (see Zaharieva, Kyurkchiev and Iliev [18]):   n i−1 X X 1 aij xk+1 − bi  , aij xkj + aii xki + xk+1 = xki − k  j i Di j=1 (5) j=i+1 i = 1, 2, . . . , n; k = 0, 1, . . . ,

where Dik = max

  

aii ,

i−1 Y

(xki −xkj )

j=1

n Y

  (xki −xk+1 ) , i = 1, 2, . . . , n; k = 0, 1, 2, . . . j 

j=i+1

and the sign of the product is defined as equal to the sign of aii . 5. Generalized Nekrassov-Mehmke algorithm, based on NM1, with the application of the decomposition A = Tm − Em − Fm (see Salkuyeh [11]): xk+1 = (Tm − Em )−1 Fm xk + (Tm − Em )−1 b, k = 0, 1, 2, . . . .

(6)

SOFTWARE IMPLEMENTATION OF MODIFICATIONS OF...

493

6. Generalized Nekrassov-Mehmke algorithm, based on NM2, with application of the decomposition A = Tm − Em − Fm (see Zaharieva, Kyurkchiev and Iliev [19]): xk+1 = (Tm − Fm )−1 Em xk + (Tm − Fm )−1 b, k = 0, 1, 2, . . . .

(7)

7. Modification of the Nekrassov-Mehmke algorithm, based on NM1, with application of the regular splitting A = D − L + U , where D is a diagonal matrix and L and U are strictly lower and strictly upper triangular matrices, respectively, and L ≥ 0, U ≥ 0 (see Kinashi, Sawami and Niki [5]): xk+1 = −(D − L)−1 U xk + (D − L)−1 b, k = 0, 1, 2, . . . .

(8)

8. Modification of the Nekrassov-Mehmke algorithm, based on NM2, with application of the regular splitting A = D − L + U , where D is a diagonal matrix and L and U are strictly lower and strictly upper triangular matrices, respectively, and L ≥ 0, U ≥ 0 (see Zaharieva, Kyurkchiev and Iliev [19]): xk+1 = −(D − U )−1 Lxk + (D − U )−1 b, k = 0, 1, 2, . . . .

(9)

9. Generalized Accelerated Over Relaxation algorithm, based on NM1, with application of the decomposition A = Tm − Em − Fm (see Salkuyeh [12]):
xk+1 = (Tm − γEm )−1 (1 − ω)Tm + (ω − γ)Em + ωFm xk +

+ ω(Tm − γEm )−1 b, k = 0, 1, 2, . . . , (10)

where 0 ≤ γ < ω ≤ 1. 10. Generalized Accelerated Over Relaxation algorithm, based on NM2, with application of the decomposition A = Tm − Em − Fm (see Zaharieva, Malinova [17]):
xk+1 = (Tm − γFm )−1 (1 − ω)Tm + (ω − γ)Fm + ωEm xk +

+ ω(Tm − γFm )−1 b, k = 0, 1, 2, . . . , (11)

where 0 ≤ γ < ω ≤ 1. Some of the above listed iterative algorithms allow 2-stage modifications. For instance, (12) and (13) are constructed on the basis of algorithms (8) and (9): 1 1 xk+1 = (D−L)−1 (D−L−U )xk + (D−L)−1 b, k = 0, 1, 2, . . . , 2 2

(12)

494

A. Golev, A. Malinova, D. Zaharieva 1 1 xk+1 = (D−U )−1 (D−U −L)xk + (D−U )−1 b, k = 0, 1, 2, . . . , 2 2

(13)

as it is discussed in [5] and [19] respectively. It can also be observed that the products taking part in factors D and N in (4) and (5) can successfully be used for approximating the coefficients aii in these algorithms, which we call combined algorithms.

3. Software Implementation The developed software – the LSIterativeMethods package, is a Mathematica package that is installed as a Mathematica add-on (automatically or ondemand). The package also provides help information, which is integrated in and accessible through the Mathematica Help browser, as it is shown in Figure 1. The content of the package can be grouped as follows: 1. Iterative algorithms: the implementation involves standard procedures for setting: the maximum number of iterations; precision (number of digits); tolerance for the approximate error; initial guess of the solution vector, relaxation parameters for the relaxation algorithms; the value of m for algorithms with an iteration M -matrix; termination criteria; the printing of an approximate solution from each iteration and of the last iterative solution vector. The algorithms also contain tests for specific requirements for the matrices, if they are included in the algorithm definition; this is provided by functions realized in the package. Some of the functions are for instance: • GaussSeidelClassic – impelemnts (2); GaussSeidel2 – implements (3); • GaussSeidelMod – implements (4); GaussSeidelMod2 – implements (5); • GenGaussSeidel – implements (6); GenGaussSeidel2 – implements (7); • GAORSalkuyeh – implements (10); GAOR – implements (11); 2. Functions providing convergence analysis, for example: • StandardConvTest – analysis of the spectral radius of the iteration matrices of the implemented algorithms; • GAORconv – analysis of the spectral radius of the iteration matrix of algorithm (11) and of the fulfillment of the requirements of Theorem 2 from [17], which guarantee convergence of the iterative algorithm;

SOFTWARE IMPLEMENTATION OF MODIFICATIONS OF...

495

Figure 1: Help documentation for the developed software package LSIterativeMethods • GaussSeidelClassicConvSPD – analysis of positive definiteness as a condition for convergence of the algorithm; • GaussSeidelClassicConvSDD – analysis of diagonal dominance as a condition for convergence of the algorithm. 3. Utility functions – the implementation of these functions involves using Mathematica’s rule-based programming, patterns, transformation rules and extensive capabilities for manipulating strings [14, 2]. The package provides functions such as: • Generating banded matrices with bandwidth 2m+1 for the decomposition A = Tm − Em − Fm , where T is a banded matrix with bandwidth 2m+1: functions T m, Em, F m; • Predicate functions for testing for specific requirements for the matrices, for instance: if the matrix is an M -matrix, if it is diagonally dominant, if

496

A. Golev, A. Malinova, D. Zaharieva

Figure 2: The package function GenerateCode with options to control the code generation it has a positive leading diagonal, if it is a square matrix, etc. (IsMMatrix, IsDiagonallyDominant, IsStrictlyDiagonallyDominant, HasPositiveLeadingDiagonal). Note that the test for positive definiteness is implemented using the built-in Mathematica function PositiveDefiniteMatrixQ. • Functions for partial automation of the process of generating code for new methods, as described below. A considerable part of the base code for the function bodies is common to all iteration algorithms. Due to that, a convenient facilitation for partial automation of the process of code implementation for the newly created algorithms is provided in the package by the function GenerateCode. Created is a parameterized template on the basis of which the user could generate a personalized version of the skeleton of the function code for an iterative algorithm. The code generation is controlled by the GenerateCode function options, such as: the name of the method, whether to add/remove the code for printing the results of each iteration, whether to add/remove code for printing the results obtained by a direct method, adding/removing the parameter m for the methods with M -matrix, etc., as it is shown in Figure 2. More options are to be added, since this is a work in progress. The generated code is a function that provides input parameters for: the matrices A and B, the initial guess for the solution vector and additional optional input parameters with default values, such as precision (number of digits) and tolerance for the approximate error. The code also provides input of the maximum number of iterations and a pre-defined termination criteria. The generated code is automatically saved in a separate Mathematica notebook file, titled with the name of the new function; then the user can make changes and additions. For instance, the code shown in Figure 2 creates a Mathematica notebook titled Method5.nb containing a function Method5, which

SOFTWARE IMPLEMENTATION OF MODIFICATIONS OF...

497

includes the code for inputting a value for the parameter m and for printing the results of each iteration. Although the code generation is partial and concerns the most basic aspects of the iteration algorithm, it is convenient and time saving in the process of research work. It is also intended to add functionality to explicitly point to an existing package algorithm and reuse its entire code without generating code on the basis of the parameterized template.

4. Numerical Example The following example demonstrates using the developed software package for illustrating the results of Theorem 2, proved in [17]: Theorem 1. (Theorem 2 in [17]) If A is an M -matrix and 0 ≤ γ < ω ≤ 1 with ω 6= 0 then method (11) is convergent, i.e.   N,(m) ρ GAOR (γ, ω) < 1. Consider the M-matrix (example by Salkueh [12]) 

 4 −2 −1 −2 −1 5 −5 −1  A= −2 −1 9 −1 . −1 −1 −1 5 Let γ = 0.5, ω = 0.9. For algorithm (11) and m = 1, using the package functions IsMMatrix and GAORconv, we have (see Figure 3):   N,(1) ρ GAOR (0.5, 0.9) = 0.701942 < 1. Similarly, for m = 2 and γ = 0.5, ω = 0.9 we have   N,(2) ρ GAOR (0.5, 0.9) = 0.495377 < 1.

These results show that the considered Theorem holds true. The example   N,(t1 ) also shows that, if t1 > t2 , then in general ρ GAOR (γ, ω) is not less than   N,(t ) ρ GAOR2 (γ, ω) .

498

A. Golev, A. Malinova, D. Zaharieva

Figure 3: Numerical example

5. Conclusions The proposed Mathematica implementation of various, mostly developed during the last three years, iterative algorithms for solving linear systems of equations could contribute to research work aimed at developing successful modifications of existing iterative algorithms. Specifically, the presented software can be used in the process of studying the spectral radiuses of the iteration matrices

SOFTWARE IMPLEMENTATION OF MODIFICATIONS OF...

499

of newly-appearing and insufficiently analyzed algorithms in the field of applied analysis, in order to prove the right of existence of these algorithms. The developed software package, in our opinion, successfully complements the existing Mathematica packages. An additional benefit is the possibility to enhance university education in subjects involving applied algorithms in the fields of algebra and analysis such as Applied mathematics, Numerical methods, Insurance mathematics and Econometrics. Acknowledgments This work was partially supported by the NI11–FMI–004 project of the Scientific Fund of the University of Plovdiv “Paisii Hilendarski”, Bulgaria. References [1] Faddeev, D., V. Faddeeva, Numerical Methods of Linear Algebra, 2nd ed., Fizmatgiz, M. (1963). [2] Gocheva-Ilieva, S., Introduction into system Mathematica, Exs-Press, Gabrovo, Bulgaria (2009). (in Bulgarian) [3] Iliev, A., N. Kyurkchiev, M. Petkov, On Some Modifcations of the Nekrassov Method for Numerical Solution of Linear Systems of Equations, Serdica Journal of Computing, 3 (2009), 371–380. [4] Iliev, A., N. Kyurkchiev, Nontrivial Methods in Numerical Analysis: Selected Topics in Numerical Analysis, LAP LAMBERT Academic Publishing, Saarbrucken (2010). [5] Kinashi, Y., H. Sawami, H. Niki, An Iterative Method Applied to Nonsymmetric Linear Systems, Japan J. Indust. Appl. Math., 13 (1996), 235-241. [6] Kyurkchiev, N., M. Petkov, A. Iliev, A modification of Richardson method for numerical solution of linear system of equations, Compt. rend. Acad. bulg. Sci., 61 (10) (2008), 1257–1264. [7] Malinova, A., Modernizing Legacy Physics Applications for Reuse in Web and SOA, Serdica Journal of Computing, 5 (1) (2011), 79–100. [8] Mehmke, R., P. Nekrassov, Solution of linear system of equations by means of successive approximations, Math. Sb., 16 (1892), 437–459. (in Russian)

500

A. Golev, A. Malinova, D. Zaharieva

[9] Popova, E., L. Kolev, W. Kramer, A Solver for Complex-Valuated Parametric Linear Systems, Serdica Journal of Computing, 4 (1) (2010), 123– 132. [10] Popova, E., Connectivity to Interval Libraries filib++ and C-XSC. In: Numerical Validation in Current Hardware Architectures (Eds. A. Cuyt, W. Kramer, W. Luther, P. Markstein), Lecture Notes in Computer Science 5492, Springer Berlin/Heidelberg (2009), 117–132. [11] Salkuyeh, D., Generalized Jacobi and Gauss-Seidel methods for solving linear system of equations, Numer.Math. A J. of Chinese Iniv. (English Ser.), 16 (2) (2007), 164–170. [12] Salkuyeh, D., Generalized AOR method for solving system of linear equations, Australian J. of Basic and Appl. Sci., 5 (3) (2011), 351–358. [13] Trott, M., The MATHEMATICA GuideBook for Numerics, Springer Science+Business Media, Inc. (2006). [14] Trott, M., The MATHEMATICA GuideBooks to Programming Graphics Numerics - Symbolics, Springer Verlag, New York, Inc. (2004). [15] Vatti, V., G. Gonfa, Refinement of generalized Jacobi (RGJ) method for solving system of linear equations, Int. J. Contemp. Math. Sciences, 6 (3) (2011), 109– 116. [16] Wolfram Mathematica 8, Documentation center, http://reference.wolfram.com/mathematica/guide/Mathematica.html [17] Zharieva, D., A. Malinova, On some (AOR) iterative algorithms for solving system of linear equations, Plovdiv Univ. ”P. Hilendarski” Sci. Works Math., 38 (3) (2011), 133–144. [18] Zaharieva, D., N. Kyurkchiev, A. Iliev, A SOR-Nekrassov-Mehmke procedure for numerical solution of linear systems of equations, Plovdiv Univ. ”P. Hilendarski” Sci. Works - Math., 37 (3) (2010), 121–134. [19] Zaharieva, D., N. Kyurkchiev, A. Iliev, Generalized Nekrassov-Mehmke procedures for solving linear system of equations, Compt. rend. Acad. bulg. Sci., 64 (4) (2011), 487–496.