SCIENCE CHINA Physics, Mechanics & Astronomy • Article •
February 2014 Vol.57 No.2: 1–13 doi: 10.1007/s11433-013-5203-5
Parallel computing study for the large-scale generalized eigenvalue problems in modal analysis FAN XuanHua1,2*, CHEN Pu1*, WU RuiAn2 & XIAO ShiFu2 1
Department of Mechanics and Aerospace Engineering, College of Engineering, Peking University, Beijing 100871, China; 2 Institute of Systems Engineering, China Academy of Engineering Physics, Mianyang 621900, China Received June 11, 2012; accepted May 30, 2013
In this paper we study the algorithms and their parallel implementation for solving large-scale generalized eigenvalue problems in modal analysis. Three predominant subspace algorithms, i.e., Krylov-Schur method, implicitly restarted Arnoldi method and Jacobi-Davidson method, are modified with some complementary techniques to make them suitable for modal analysis. Detailed descriptions of the three algorithms are given. Based on these algorithms, a parallel solution procedure is established via the PANDA framework and its associated eigensolvers. Using the solution procedure on a machine equipped with up to 4800 processors, the parallel performance of the three predominant methods is evaluated via numerical experiments with typical engineering structures, where the maximum testing scale attains twenty million degrees of freedom. The speedup curves for different cases are obtained and compared. The results show that the three methods are good for modal analysis in the scale of ten million degrees of freedom with a favorable parallel scalability. modal analysis, parallel computing, eigenvalue problems, Krylov-Schur method, implicitly restarted Arnoldi method, Jacobi-Davidson method PACS number(s): 46.15.-x, 46.40.-f, 43.40.At, 02.60.-x Citation:
Fan X H, Chen P, Wu R A, et al. Parallel computing study for the large-scale generalized eigenvalue problems in modal analysis. Sci China-Phys Mech Astron, 2014, 57: 113, doi: 10.1007/s11433-013-5203-5
1 Introduction Modal analysis is an important numerical tool to obtain dynamic properties of engineering structures. Mathematically, modal analysis is equivalent to computing a number of lower eigenpairs of the generalized eigenvalue problem Kx Mx, K , M R n n ,
(1)
using various algorithms, where K and M denote stiffness and mass matrix, respectively. Both K and M can be obtained by finite element discretization and they are usually large, sparse, and symmetric positive definite. *Corresponding author (FAN XuanHua, email:
[email protected]; CHEN Pu, email:
[email protected]) © Science China Press and Springer-Verlag Berlin Heidelberg 2013
For some large complex structures in civil engineering, aeronautics and space realms, in order to describe their dynamic characteristics more exactly, the degrees of freedom of the finite element model, which corresponds to the order of K and M, can attain ten million or above . In the case of millions of degrees of freedom, the computation of equation (1) becomes difficult. Both algorithms and parallel computing techniques should be therefore considered. Researchers have proposed and implemented a large variety of algorithms [1–9] for large-scale eigenvalue problems. All these algorithms start with the techniques of subspace iterations and projections. The general framework of these methods is to generate a sequence of subspaces V1, V2, ... of small dimensions commensurate in size with the number of desired eigenvalues and project the large matriphys.scichina.com
link.springer.com
2
Fan X H, et al.
Sci China-Phys Mech Astron
ces onto these small subspaces, and then use these small subspaces computing part eigenpairs of original eigenvalue problems [1,2]. Of these algorithms, three predominant methods, i.e., Implicitly Restarted Arnoldi (IRA) method [3,10], Krylov-Schur (K-S) method [6] and Jacobi-Davidson (J-D) method [5], have been regarded as the most successful and flexible ones for finding a few eigenpairs of a large, sparse matrix. IRA and K-S belong to Krylov subspace methods and J-D belongs to Davidson-based methods. For the standard Hermitian eigenvalue problems, IRA and K-S are also named Implicitly Restarted Lanczos method [11] and Krylov-Spectral method [6,12], respectively. However, the applications or comparisons of these methods on large-scale modal analysis are seldom reported in the literature. We know that Peter Arbenz and his collaborators [13] did a parallel modal analysis of an aircraft carrier using several algorithms in 2005, the order n of K and M being 1892644 and the numbers of parallel CPU processors being 256. They believed that was the most challenging problem at that time. The goal of our work is to propose a strategy of parallel modal analysis by modifying the existing, mature algorithms with some complementary techniques. These techniques include spectral transformation, solution strategy of linear systems for the three methods (IRA, K-S and J-D), as well as restarting and deflation techniques for the J-D method. The rest of the paper is organized as follows. Sect. 2 gives a brief review of the subspace algorithms, based on which three predominant algorithms (i.e., IRA, K-S and J-D) are chosen for modal analysis. Detailed algorithms with modifications for modal analysis are given. Sect. 3 discusses the parallel implementation, including pre-processing modeling, generation of K and M, parallel process management and solution with eigensolvers. Two representative examples with different scales in modal analysis are given to compare the performance of the three algorithms in sect. 4. Finally, the paper ends with a brief conclusion in sect. 5.
2 Algorithm design for modal analysis For large-scale eigenvalue problems, subspace-based algorithms are initially presented for standard eigenvalue problems Ax = x, most of which come from the power method and Rayleigh quotient iteration [14–16], providing good approximations quickly to the largest eigenvalues. However, what we deal with in modal analysis is a generalized eigenvalue equation, and the eigenpairs most relevant to modal analysis are the lowest ones, which affect dynamic behavior of engineering structures. Therefore, the key problem in algorithm design for modal analysis is to modify these algorithms so that they can solve the corresponding generalized eigenvalue equation and approximate the lowest eigenvalues efficiently.
2.1
February (2014) Vol. 57 No. 2
A brief review of subspace-based methods
Subspace-based methods differ from each other in the ways the subspaces are generated. The dimension of subspaces is either fixed or variable. The classical algorithms working with fixed dimensions mainly include power method [1,2,14], Rayleigh quotient iteration [1,2,15] and subspace iteration [1,16]. Starting from a subspace Vk, these methods generate the next subspace Vk+1 of the same dimension by applying a linear operator A on Vk. As k increases, Vk contains better approximate eigenvectors corresponding to the eigenvalues of A with a larger magnitude. A further class of subspace methods involves those whose dimensions increase as the iteration proceeds. Usually one starts with a subspace of dimension one and increases the dimension at each iteration step. These methods are in general more efficient than fixed-dimension ones and have become the mainstream for large-scale eigenvalue problems. The most popular subclass in this class is the Krylov subspace method [1,2,4], which can be traced back to the Lanczos method [17] for symmetric matrices and the Arnoldi method [18] for nonsymmetric matrices. There are many other updated developments for the two basic Krylov subspace methods. A significant improvement of such subclass appertains to Sorensen’s IRA/Lanczos method [10,11]. Later, Stewart [6] proposed the K-S method by expanding Arnoldi decomposition to a general Krylov decomposition. The IRA/Lanczos method and the K-S method are mathematically equivalent, recognized as the most successful algorithms in Krylov subspace methods. There exists another subclass with increasing subspace dimension, but without using Krylov subspaces. A Newton iteration step or an approximate Newton iteration step can be applied to expand the subspace. The typical representations for this subclass are Davidson-based methods [1,2,5,19]. Based on the standard Davidson method [19], some generalized Davidson methods [1,2,20–22] were presented by using different preconditioning. In 1996, Sleijpen and Vorst [5] presented a J-D algorithm, which speeded up the development of Davidson-based methods. So far, the J-D method has been extended to various eigenvalue problems [23–26]. For details of the J-D algorithm, refer to refs. [2,27]. With the above algorithms, a number of codes were developed for the numerical solution to large-scale eigenvalue problems. Hernandez et al. [28] made a survey of freely available software tools, including a list of libraries, programs or subroutines. Among these codes, ANASAZI [29] and SLEPc [30] are recognized as two predominant tools that come closest to a robust, efficient and general purpose code. So far, both ANASAZI and SLEPc are being actively developed. 2.2
Krylov subspace methods for modal analysis
Krylov subspace methods [1,14] start from a single vector space K1(A,v) = span{v}, and then expand the Krylov sub-
Fan X H, et al.
Sci China-Phys Mech Astron
space Kk(A,v) = span{v, Av, A2v, …, Ak1v} in the k-th iteration to Kk+1(A,v). We focus our study on IRA and K-S methods in this section. Since the implicitly restarted Lanczos method and the Krylov-Spectral method can be
February (2014) Vol. 57 No. 2
3
viewed as a specialized variant of IRA and K-S, we describe the IRA and K-S algorithms in a general sense for applications in other realms. Our description starts with the basic Arnoldi method [2,10] shown in Algorithm 1.
Algorithm 1 Basic Arnoldi method Input Matrix A(n×n), number of steps m(m n), and initial vector v1 of norm 1. Output Vm (n×m), Hm (upper Hessenberg matrix with m×m), fm and so that AVmVmHm= f m emT , =|| fm ||2. 1: for j = 1, 2, …, m 2: Increase Krylov vector: = Avj. 3: Orthogonalize with respect to Vj and obtain hj (i≤j): 4: f j h1 j v1 ... hij vi ... h jj v j with hij T vi . 5: hj+1, j = || fj||2 . 6: if hj+1,j = 0 or j = m, = hj+1, j, stop. 7: vj+1 = fj / hj+1, j. 8: end for When A is a symmetric matrix, the orthogonal process in step 3 of Algorithm 1 becomes a three-term recurrence formula [1,18], i.e., f j w h j 1 j v1 h jj v j , due to the symmetry. Thus the upper Hessenberg matrix Hm becomes a tridiagonal one and Algorithm 1 turns into a basic Lanczos algorithm. It is desirable for to become small enough in Algorithm 1, because this indicates that the eigenvalues of Hm are accurate approximations to the eigenvalues of A. However, a small may not appear until m becomes very large, which leads to difficulties of storage and numerical orthogonality. In this case, a restarting technique should be considered. The IRA method offers an implicit restarting technique which combines the implicitly shifted QR scheme with an m-step Arnoldi factorization to obtain a truncated form of the Arnoldi factorization [2,10]. Implicit restarting provides a means to extract interesting information from large Krylov subspaces while avoiding the storage and numerical difficulties associated with the basic approach. The IRA method has been remarkably successful and has been implemented in the widely used ARPACK package [31]. However, for the use of IRA in modal analysis, two aspects should be emphasized [7]. One is the reduction of generalized eigenvalue problems to standard ones; the other is the fast convergence to small eigenvalues, which are our interest for modal analysis. These two aspects can be treated with a spectral transformation technique, Shift-and-Invert (S-I) or Cayley transformations [32], for example. For eq. (1), the S-I transformation leads to
Kx Mx K M
1
1 Mx x,
(2) 1
where denotes a user-selected shift. Let A = (KM) M and = 1/(), and we can theoretically transform (1) to a
standard problem Ax = x,
(3)
which can be directly solved via IRA. Meanwhile, we can select an effective shift, for example, a little lower than the minimum eigenvalue of original problems. Then the fast convergence to larger eigenvalues in eq. (3) is transformed to that of smaller ones in eq. (1), due to the invert relation between and . The S-I spectral transformation provides a powerful tool in the treatment of modal analysis, but brings a new problem. In each Krylov subspace iteration, the new subspace vector = A (Step 2 of Algorithm 1) becomes = (KM)1Mvj, and an equivalent form is (KM)= Mvj.
(4)
To obtain requires solving a linear equation of eq. (4). Due to , the coefficient matrix KM is usually ill-conditioned or even nearly singular. Thus an iterative solution to eq. (2.3) becomes difficult [1,2]; instead a direct method is often recommended. Considering the symmetry of the coefficient matrix (KM), a sparse LU or a Cholesky factorization [33,34] will do the job. The implementations of these factorizations are included in many software packages, such as MUMPS [35]. Let KM = LU
(5)
represent some convenient factorization of KM, and = Av can be calculated as follows: v =Mv, Lw= v , Uw.
(6)
Based on the above consideration, we describe the IRA method for modal analysis in Algorithm 2.
4
Fan X H, et al.
Sci China-Phys Mech Astron
February (2014) Vol. 57 No. 2
Algorithm 2 IRA method for modal analysis Input the matrices K and M, initial vector v1 of norm 1, number of maximum dimensions of searching subspace m, num-
ber of desired eigenvalues k (k
