convergence begins to slow down; and 4) ... tions constrained to full rank converged more slowly than ... routines has a sparse storage option for A as well as ...
Iterative Algorithms
for Solving Mixed
Model
Equations ~
P. J. BERGER, t G. R. LUECKE, ~ and J. A. HOEKSTRA 3 Iowa State University Ames 50011
ABSTRACT
rank converged more unconstrained equations.
Iterative algorithms for obtaining solutions for sire effects and breeding value estimates from progeny with records in mixed model procedures were compared. Successive overrelaxation with adaptive acceleration and a Jacobi conjugate gradient method seem to be more generally useful for two general areas of interest: 1) sire evaluation models, where some effects are absorbed; and 2) reduced animal models, where no effects are absorbed. An inverse of the relationship matrix may be included as part of the coefficient matrix and its inclusion will not prevent convergence. Key features include: 1) scaling the equations so that all diagonal elements are 1 and the scaled coefficient matrix remains real, symmetric, and positive definite; 2) calculation of a new relaxation parameter during run time to approximate the value that will yield fastest convergence; 3) calculation of a new relaxation parameter after convergence begins to slow down; and 4) use of numerically accurate and efficient convergence criteria. The Jacobi conjugate gradient method was 55% more efficient than successive overrelaxation in solving reduced animal model equations of order 3356. Number of iterations and total execution times for all iterations were: 83, 2.89 s and 169, 6.44 s, respectively. Another reduced animal model application with equations of order 38,139 converged in 38 s (50 iterations) using successive overrelaxation. Solutions for sire equations constrained to full
than
INTRODUCTION
Received November 18, 1987. Accepted September 30, 1988. ~Journal Paper Number J-12774 of the Iowa Agriculture and Home Economics Experiment Station, Ames. Project Number 2721. 2 Department of Animal Science. Department of Mathematics. a Computation Center.
1989 J Dairy Sci 72:514--522
quickly
514
Mixed linear models are now being applied in animal evaluation programs in many parts of the world. The desirable properties of BLUP identified through the pioneering work of C. R. Henderson (2) and many of his students now make it possible to utilize the large banks of data available through DHI, experimental herds, and other sources to exploit the random elements or linear functions of fixed and random elements of the model for breeding decisions. Because it is easy to encounter a problem that exceeds the memory capacity of modern computers, many applications must be solved iteratively. So,me national evaluations are of such magnitude that direct inversion of the coefficient matrix is not practical, even with the memory available on the current generation of supercomputers, thereby supporting the need for more refined techniques for solving mixed model equations. Solving equations by iteration is not an exact science. The method of choice depends on the properties of the equations and how the method makes use of these properties. In some cases, the model and amount of data will influence the method of choice. Gauss-Seidel (GS) and successive overrelaxation (SOR) methods of obtaining solutions for sire effects from equations rising from progeny with records in mixed model procedures were compared by Van Vleck and Dwyer (8); SOR was found to be more efficient than GS. Solutions for equations constrained to full rank converged more slowly than unconstrained equations. More recently, indirect procedures that iterate on the data have been described by Misztal and Gianola (4) and Schaeffer and Kennedy (6). The latter procedures are appealing because they eliminate the need to form a coefficient matrix. Recent advances have yielded methods that are powerful tools for solving mixed model equations. To choose among competing methods for obtaining solutions iteratively re-
SOLVING MIXED MODEL EQUATIONS quires some knowledge of the mathematicaI processes used by each method. Our objectives are to: 1) review recent advances in algorithms for iterating solutions to mixed model equations, 2) present the main idea behind SOR and Jacobi conjugate gradient (JCG) type algorithms with adaptive acceleration, and 3) demonstrate an improved efficiency of these procedures over the standard SOR method. Our requirements were few; we asked that the algorithms be capable of handling sire evaluation models in which some effects might be absorbed and also reduced animal models in which no effects were absorbed. Each algorithm was to be capable of handling the appropriate inverse of relationships among animals included in the evaluation. MATERIALS AND METHODS Adaptive Successive Overrelaxation
Adaptive SOR is an iterative method used for solving the linear system Av = b when the matrix A is real, symmetric, and positive definite. This method is typically applied to large sparse linear systems that arise in the numerical solution of elliptic partial differential equations. However, we have found adaptive SOR to be very efficient for solving numerically the large, sparse linear systems that arise in genetic modelling problems even though A is not necessarily positive definite for these problems. In many mixed model applications, A may not be positive definite. With a constraint on the equations to account for the dependency between fixed effects, the equations have that property. Algorithm 586, "ITPACK 2C: A F O R T R A N Package for Solving Large Sparse Systems by Adaptive Acceleration Iterative Methods" (3), is a particular implementation of various adaptive iterative methods that includes an adaptive SOR routine for solving linear systems. The subroutines are adaptive in the sense that optimum parameters are chosen for the particular problem to be solved. Each o f these routines has a sparse storage option for A as well as using only those elements of A on or above the diagonal (inasmuch as A is symmetric). Four features of the adaptive SOR routine in this package are particularly worthwhile, without complicating the process of setting-up the
515
system of mixed model equations. 1) To achieve greater numerical accuracy and to decrease computing time, the original system of equations is scaled so that all diagonal elements are 1 and so that the scaled coefficient matrix remains real, symmetric, and positive definite. (Details are given in the Appendix 1 for scaling the matrix.) 2) The relaxation parameter is not fixed throughout all iterations but is calculated during run time as part of the algorithm in a way that approximates the value that will yield fastest convergence. (Details for this are given in the Appendix 2.) 3) A new relaxation parameter is not calculated for each iteration but only after convergence begins to slow down; this gives numerical stability (1) increased efficiency to the method. 4) The key to numerical accuracy and efficiency is knowing when to terminate the iteration. Most iterative methods are terminated when an error estimate is smalI enough. If the error estimate is not accurate, then two possible problems can result: 1) the iteration is terminated too soon, and one thinks that the answer is accurate enough, when it is not; and 2) the computed solution may be sufficiently accurate, but the error estimate indicates that the error is still very large. In this second instance, one would continue to perform many unnecessary iterations, which typically are very expensive. Kincaid et al. (3) used an asymptotic formula to approximate the relative error that yields, in our experience, a good stopping condition for the iteration. Explanation is in Appendix 4. Jacobi Conjugate Gradient
The JCG method applies the three-term conjugate gradient method to the scaled linear system, calculates iteration acceleration parameters, and then uses a sophisticated method of estimating the relative error to determine if the iteration should be terminated. Details for this are in Appendix 3. Testing and Validation
Algorithms found in ITPACK (3) were investigated for solving mixed model equations iteratively; ITPACK is a collection of seven F O R T R A N subroutines designed to solve iteratively large sparse systems of equations. Five of the subroutines were tested on four important applications of mixed models (Table Journal of Dairy Science Vol. 72, No. 2, 1989
516
B E R G E R ET AL.
1) to determine which subroutines were most suitable for handling large problems and the general utility of adaptive iterative methods over standard nonadaptive techniques. The stopping criteria were the ratio of the Euclidean norm of the pseudoresidual vector and the norm of the current iteration vector times a constant involving an eigenvalue estimate (3). Iteration was terminated when this ratio was less than 1 ' 1 0 -1° . The same stopping criteria were used for all test cases. Two of the ITPACK subroutines were not suitable for solving mixed model equations. Reduced Animal Model (RAM-I) equations (5) were developed using data from a repeatsire mating selection experiment in which Tribolium castaneum were selected for pupa weight over 10 cycles of selection. Heritability of pupa weight was .3 and there were 12,493 individual pupa. The model included a total of 3356 equations, 3230 for parents and 126 for fixed contemporary groups of set-sex-cycle subclasses. Second was a very large RAM application (RAM-II) being developed in our laboratory to predict the genetic merit of racing speed in American Quarter horses. Statistics comparing the ITPACK subroutines were obtained from Wilson (1986, personal communication) while more specific detail about the data and model may be obtained from (9). The third application was a BLUP sire evaluation model (Model l i d for milk yield. Herd-year-seasons were absorbed. The remaining 890 equations were for 885 sires and 5 birth year groups. Fourth was a BLUP sire evaluation model (Model IV) for calving ease. Herd-year-seasons were absorbed and the remaining equations included fixed effects of sex (2), parity (3), and birth-year group of sire (8), and random sires (9698). An inverse of the complete relationship matrix among parents was used with each RAM application, while the sire evaluation models included an inverse of relationships among sires and their male ancestors. All computations were performed on a National Advanced System AS/9160 computer with 16 megabytes of main memory. (The NAS 9160 is an IBM "plug compatible computer".) A vector version of ITPACK is available; however, all of the results in this paper were run in scalar mode Journal of Dairy Science Vol. 72, No. 2, 1989
,.C
o
8
.~ .£ .£ e~ ,4 ,-d
