A Fast LDL-factorization Approach for Large Sparse Positive Definite ...

International Journal of Automation and Computing

04(1), January 2007, 88-94 DOI: 10.1007/s11633-007-0088-2

A Fast LDL-factorization Approach for Large Sparse Positive Definite System and Its Application to One-to-one Marketing Optimization Computation Min Wu1∗ 1

Bei He1

Jin-Hua She2

School of Information Science and Engineering, Central South University, Changsha 410083, PRC 2

School of Bionics, Tokyo University of Technology, Tokyo 192-0982, Japan

Abstract: LDL-factorization is an efficient way of solving Ax = b for a large symmetric positive definite sparse matrix A. This paper presents a new method that further improves the efficiency of LDL-factorization. It is based on the theory of elimination trees for the factorization factor. It breaks the computations involved in LDL-factorization down into two stages: 1) the pattern of nonzero entries of the factor is predicted, and 2) the numerical values of the nonzero entries of the factor are computed. The factor is stored using the form of an elimination tree so as to reduce memory usage and avoid unnecessary numerical operations. The calculation results for some typical numerical examples demonstrate that this method provides a significantly higher calculation efficiency for the one-to-one marketing optimization algorithm. Keywords:

1

Sparse matrix factorization, elimination tree, structure prediction, one-to-one marketing optimization.

Introduction

One-to-one marketing optimization is a crucial technology for analytical customer relations management (CRM). Its main function is to assist an enterprise in one-to-one marketing activities by providing suitable decisions. It is also used as an approach to understand and influence customer behavior through meaningful communication in order to improve customer retention, customer loyalty, and profitability. Since optimization technology brings great benefits to an enterprise, increasing numbers of enterprises have been researching and employing it[1−3] . A one-to-one marketing optimization problem involves finding a binary vector (the best marketing plan) that maximizes a linear objective function (profit), which is subject to constraints, i.e. max s.t.

κT ξ Φξ ≤ γ, ξ = [ξ1 · · · ξn ]T , ξi ∈ {0, 1}, i = 1, · · · , n

(1)

where ξ ∈ Rn is the design vector, κ ∈ Rn is the vector of coefficients of the objective function, Φ ∈ Rm×n is a constraint matrix, and γ ∈ Rm is the right-hand-side vector of the constraint. Clearly, the key to the computations is how to solve the optimization problem (1) for ξ quickly and accurately because Φ is usually a huge sparse matrix. Since a large enterprise has a great variety of products, many communication channels, and millions of customers, the number of variables and the number of constraints may reach the order of 108 and 106 , respectively, in a one-to-one marManuscript received September 19, 2005; revised July 31, 2006. This work was supported in part by the National Natural Science Foundation of PRC (No. 60425310) and the Teaching and Research Award Program for Outstanding Young Teachers in Higher Education Institutions of MOE, PRC. *Corresponding author. E-mail address: [email protected]

keting optimization model. An optimization problem for a customer base of order of 105 or higher is usually called a large-scale optimization problem. Wu, He and Chen examined the relationships between the scale (i.e., the size of the customer base) on the one hand, and the computation time and memory usage needed to solve the problem on the other hand[4] . Their results show that conventional linear programming (LP) methods (e.g., Simplex and Depth-First Search) cannot solve a large-scale one-to-one marketing optimization problem because both the computation time and the memory usage increase exponentially with the number of customers. Over the past few years, some interior-point methods of LP for large-scale problems have been developed based on an iterative algorithm(e.g., the predictor-corrector and Krylov-subspace methods[5] ) . All of the methods developed so far involve solving the following linear matrix equation: Ax = b

(2)

where A is a constant matrix and b is a constant vector. Since A is huge for a large-scale one-to-one marketing optimization problem, the problem is how to solve (2) quickly. In fact, how to efficiently handle a large sparse matrix is a common problem in many engineering fields, such as fluid dynamics, structural engineering, electrical circuits, heat transport, semiconductor modeling, molecular dynamics, geophysical reservoirs, etc. Along with the rapid progress in computer technology, techniques of information processing have also been advancing quickly. Not only have existing algorithms been improved, but new methods of handling large sparse matrices have also been devised. This paper considers the problem of using LDLfactorization to solve the linear matrix equation (2) for a large sparse symmetric positive definite matrix, A. An elimination tree and a two-stage strategy are employed in conjunction with LDL-factorization to derive a new computa-

M. Wu et al./A Fast LDL-factorization Approach for Large Sparse Positive Definite System and Its Application to · · ·

tion method. Based on the structure of the elimination tree for a given matrix, this method performs LDL-factorization in two stages: structural prediction (symbolic stage) and numerical computation (numerical stage). It provides an efficient storage scheme for storing the elimination tree and a sparse matrix. It thus avoids redundant memory usage and requires less time for numerical calculations. As a result, this method substantially improves the speed and reduces memory usage in the computation of the LDL factor. Throughout this paper, for a matrix A = (aij ) ∈ Rn×n , GA = (V, E) is an undirected graph of A, where the nodes of V ≡ {1, 2, · · · , n} correspond to the rows and columns of A, and the edges of E ≡ {(j, i)|j > i and aij 6= 0} correspond to the nonzero entries of A. Ai:j,k:l means a submatrix of A obtained by extracting rows i through j and columns k through l. Ai:i,k:l is abbreviated as Ai,k:l , and likewise for a column submatrix of A. In particular, A(k) is a leading principal submatrix of A of order k, i.e. A(k) = A1:k,1:k . In is an n × n identity matrix, and 1n is an n-dimensional column vector in which all the entries are 1. The rest of this paper is organized as follows. Section 2 introduces some related works about factorization methods for solving the linear matrix equation (2). Section 3 analyzes the LDL-factorization algorithm and presents new algorithms for LDL-factorization based on the theory of elimination trees. Section 4 describes the storage scheme for the LDL factor and the elimination tree. Section 5 presents the calculation results for typical numerical examples. Section 6 explains the application of this method to computations for one-to-one marketing optimization. Finally, Section 7 contains some concluding remarks.

2

Related works

At present, several robust algorithms and software packages have been developed to solve the linear matrix equation (2). Specifically, the algorithms presented in [6] are mainly used in Cholesky factorization; they factor A into CC T , where C is a lower triangular matrix called the Cholesky factor of A. To avoid the storage of, and operations on, the zero entries of C, Rose devised a new computational method for Cholesky factorization[7] , in which the computation is broken down into two stages: First, in the symbolic stage, the pattern of nonzero entries of C is predicted; and then, in the numerical stage, this structure is used to carry out numerical calculations. The advantage of this method is that the time needed for the first stage is proportional to the size of C. This method was improved by George and Reid, who developed SPARSPAK[8] , which uses asymptotically fastest algorithms to compute the Cholesky factorization in the symbolic stage. Later, Liu presented a compact row-oriented storage scheme for Cholesky factorization[9] . This structural representation can be viewed as the minimal structure of a given matrix that preserves the symbolic Cholesky factor. On the other hand, Gilbert and Peierls devised a new solution for general Gaussian elimination, A = GU , where G is a lower triangular matrix and U is an upper triangular matrix. It also breaks the computation of the factorization factor into a symbolic and a numerical stages[10] . In the calculation procedure, the pattern of nonzero entries in each

89

column of the factor is first predicted. Then, numerical calculations are carried out based on that structure. This method restricts the numerical factorization to necessary arithmetic, and requires fewer operations. LDL-factorization involves factoring symmetric positive definite sparse matrix A into LDLT , where D is a diagonal matrix and L is a unit lower triangular matrix called the LDL factor. It is also viewed as a special case for Cholesky factorization. Timothy A. Davis presented an LDL software package to illustrate much of the basic theory of sparse matrix algorithms in as concise a code as possible, including an elegant method of sparse symmetric factorization that computes the factorization row-by-row but stores it columnby-column[11] .

3

Algorithms

This section considers how to use LDL-factorization to solve (2), where A ∈ Rn×n is a large sparse symmetric positive definite matrix. The elimination tree is defined specifically for use in LDL-factorization, and some new algorithms for LDL-factorization are presented. Special care is taken in the construction of algorithms with the goal of reducing computation time and memory usage. Consider the following LDL-factorization: A = LDLT .

(3)

Assume that matrix A has already been ordered by some fill-reducing symmetric permutation (e.g., minimum-degree ordering [12] or nested-dissection ordering[13] ). A wellknown algorithm that performs LDL-factorization[14] is Algorithm 1. LDL-factorization of A ∈ Rn×n L(1) = 1, D(1) = a11 . for k = 2 to n Step 1. solve L(k − 1)x = A1:k−1,k for x. Step 2. Lk,1:k−1 = (D−1 (k − 1)x)T . Step 3. lkk = 1. Step 4. dkk = akk − Lk,1:k−1 x. end for In this algorithm, lii , dii , and aii (i = 1, · · · , n) are the diagonal entries of L, D, and A, respectively, l11 = L(1) = 1, and d11 = D(1) = a11 . The k-th row of L and the (k, k) entry, dkk , of D (k = 2, · · · , n) are found in the k-th step for a lower triangular matrix belonging to R(k−1)×(k−1) . To simplify the explanation in the rest of the paper, the ˜ = b, where b := equation in Step 1 is abbreviated as Lx ˜ when there is A1:k−1,k and L(k − 1) is abbreviated as L no possibility of confusion. χ and β represent the sets of indices of the nonzero entries of x and b, respectively. In fact, χ is the set of nodes reachable via paths in the graph GL˜ from all the nodes in the set β. The closure of β is the smallest set containing β [15] , i.e. \ closure(β) = {γ : β ⊆ γ and γ is closed} . (4) With this preparation, we are ready to deal with the problem of simplifying the calculations. In Algorithm 1, the computational complexity is mainly centered in Step 1, ˜ = b is solved for x. Note that Algorithm 1 in which Lx ˜ and performs requires the storage of the zero entries of L

90

International Journal of Automation and Computing 04(1), January 2007

calculations on them, which is unnecessary. The following algorithm was created to avoid that. ˜ = b for the lower triangular Algorithm 2. Solve Lx ˜ which has unit diagonal entries matrix L, Step 1:. χ=closure(β). Step 2:. x = b. Step 3:. for i ∈ χ ˜ i+1:k−1,i xi ; xi+1:k−1 = xi+1:k−1 − L end for In this algorithm, the first step is the symbolic stage, in which the nonzero structure of x is predicted, and the other steps constitute the numerical stage. In other words, it first finds the nonzero pattern, χ, and then computes the value of x using that structure. It is known that, given the struc˜ and b, there exist nonzero values of χ=closure(β) tures L [15] . Note that the time needed to compute χ is proportional to the number of floating-point operations performed, and the same is true for x. As can be seen in Algorithm 2, GL must be predicted before χ can be computed. Note that the LDL-factorization is given by √ “ √ ”T A=L D L D (5) and the Cholesky factorization is A = CC T .

(6)

So, we can define √ C := L D = Ldiag (δ1 , δ2 , · · · , δn )

(7) √ where δi > 0 (i = 1, · · · , n) are the diagonal entries of D. From (7), L and C have the same nonzero pattern, i.e., GL = GC . The relationship between LDL-factorization and Cholesky factorization is described in the following lemma. Lemma 1. For a given sparse symmetric positive definite matrix A, the LDL factor L and the Cholesky factor C have the same nonzero pattern. As described in [16], for each column j (< n) of C, removing all the nonzero entries in the column, except for the first one below the diagonal, yields the matrix Lt . Define Ft = Lt + LT t . Then, the graph GFt has a tree structure that depends entirely on the structure of C and its initial ordering. This tree structure is denoted by the symbol TC and is called the elimination tree of C. TC has the same node set as GC . The structure of TC can be represented by the Parent vector of Ft , which is defined as follows: Let cij (i, j = 1, · · · , n) be the entries of C. Then, for each column j (< n) of C, P arent[j] = min{i > j|cij 6= 0}

(8)

is the parent node of node j in the elimination tree TC . Node j does not have a parent if the entries below the diagonal in column j are all zero. In this case, node j is a root of the elimination tree. And we define P arent[j] = 0. For completeness, we also define P arent[n] = 0. According to Lemma 1, the elimination trees TC (strictly speaking, they constitute an elimination forest) that represent the nonzero pattern of L for matrix A are the same as

those for the Cholesky factor of A; and the elimination tree vectors are (j < n) P arent[j] = min{i > j|lij 6= 0} P arent[n] = 0

(9)

where lij is the (i, j) entry of L (i, j = 1, · · · , n). Instead of using the traversal of the general graph GL through the nodes reachable from the nodes of β, this paper employs the traversal of a simplified elimination tree representing the nonzero pattern, χ, which is the set of all paths from each node j (j < n) for which bj 6= 0 to the root of the tree. The process of constructing the elimination tree is recursive, as can be seen in Algorithm 1. The elimination trees of L(k) are determined based on the elimination trees of L(k − 1) (2 6 k < n). If the parent node of node j in the elimination tree, TL(k) , is i, then i = min{i > j|lij 6= 0} and ¯ 6 n), and j 6 k. Since GL(k) is a subgraph of GL(k) ¯ (k < k the nodes that do not belong to GL(k) are all larger than k, the parent node of node j in TL(k) ¯ remains i. Summarizing the above result yields the following lemma. Lemma 2. If node j has a parent node i in TL(k) , then ¯ 6 n. node i remains the parent node of j in TL(k) ¯ for k < k Based on Lemma 2, we obtain the following algorithm for computing the structure of the elimination trees of the LDL-factorization, where P arentk means the parent vector for the elimination tree of L(k). Algorithm 3. Calculation of P arentk if k = 0 P arentk = 0; else P arentk = P arentk−1 ; //Call construction algorithm P arentk−1 . for j ∈ {lkj 6= 0, j 6 k} if P arentk [j] = 0 P arentk [j] = k; end if end for end if This algorithm is employed in Step 1 (symbolic stage) of Algorithm 2. Consider Step 3 of Algorithm 2: ˜ i+1:k−1,i xi . xi+1:k−1 = xi+1:k−1 − L

(10)

˜ = b. If This step computes the numerical solution of Lx there is a path from j to i in the elimination tree TL˜ , then the solution of xj+1 requires the computation of xi . So, sorting χ is necessary. Sorting it in a natural order is not the best way because it has the drawback of a very large computational load. In order to avoid unnecessary operations, a new topological sorting algorithm for χ was developed in which the result is assumed to be stored in the stack xs . Algorithm 4. Sorting x Step 1. for i ∈ β j = 0; while i is not marked Step 1.1. xs [j] = i; j = j + 1;

M. Wu et al./A Fast LDL-factorization Approach for Large Sparse Positive Definite System and Its Application to · · ·

Step 1.2. mark i; Step 1.3. i = P arent[i]; end while end for Step 2. Adjust the storage order of xs . //Let i be at the top of the stack and //the root be at the bottom. Algorithm 4 first selects a nonzero entry, bi , and stores the path from i to the root of the elimination tree. Then, it marks the nodes along the path and stores the path in xs with i at the top and the root at the bottom. It repeats this procedure for all the nonzero entries of bi and stops when it reaches a marked node. Note that the result may be empty if i is already in xs . So, xs contains the topological ordering, χ, of the nonzero pattern of x, and thus ˜ = b. In fact, this can be used in Algorithm 2 to solve Lx algorithm is employed before Step 3 in Algorithm 2 to obtain χ, which is used in the conditional part of Step 3. It is worth pointing out that the time needed to compute χ using Algorithm 4 (traversal of an elimination tree) is much shorter than that for the transversal of a general graph, and is proportional to the size of χ rather than to the number of floating-point operations in the computation of x. So, Algorithm 4 markedly reduces the computation time.

4

91

The dimension of A is n = 8 and the number of nonzero entries in the upper triangular matrix is m = 17. 8 T 9 > > Ap = [1, 2, 3, 4, 6, 8, 10, 14, 18] ∈ R , > > > < Ai = [1, 2, 2, 3, 4, 3, 5, 3, (12) 6, 1, 3, 6, 7, 2, 3, 5, 8]T ∈ R17 , > > > Ax = [7, 1, 2, 6, 2, 6, 3, 9, 6, 3, 2, > > : 1, 4, 1, 3, 6, 1]T ∈ R17 . As explained in Section 2, the elimination trees are stored in the array P arent. Fig. 1 shows the elimination tree of L, the LDL factor of A, and its P arent.

Format of storage

Since Algorithm 2 traverses L in column order, L is stored in that order. The Yale sparse matrix[17] is a storage format for sparse matrices that stores the nonzero entries in column order. More specifically, for a sparse matrix, A, with n columns and m nonzero entries, the Yale sparse matrix contains three arrays: Ax : Holds the values of the nonzero entries in column order. Ai : Holds the row indices in column order. Ap : Holds the index of the first entry of each column in Ai and Ax , and the last entry is set to the number m + 1. The length of Ap is n + 1, and the lengths of Ai and Ap are both m. The numerical values of the entries of column j are given by Ax [Ap [j], · · · , Ap [j + 1] − 1]; and the corresponding row indices are in Ai [Ap [j], · · · , Ap [j + 1] − 1]. Clearly, Ap [1] = 1; and the number of entries in A is m = Ap [n + 1] − 1. In order to reduce the amount of memory needed to handle A in (2), we store only the entries of the upper triangular matrix of A in the Yale sparse-matrix format. We call this format a compressed Yale sparse-matrix format. It is illustrated in the following example. Consider 3 2 7 0 0 0 0 0 3 0 6 0 1 2 0 0 0 0 1 7 7 6 7 6 6 0 2 6 0 6 9 2 3 7 7 6 6 0 0 0 2 0 0 0 0 7 6 7. (11) A=6 7 6 0 0 6 0 3 0 0 6 7 6 0 0 9 0 0 6 1 0 7 7 6 7 6 4 3 0 2 0 0 1 4 0 5 0 1 3 0 6 0 0 1

Fig. 1

5

The elimination tree TL and its Parent.

Implementation and tests

Note that L is a lower triangular matrix with diagonal entries equal to 1. In Algorithm 1, the format for storing L can be further compressed by storing only the entries of the lower triangular part of L, except for the diagonal entries. Let the LDL-factorization be ˆ + I)D(L ˆ + I)T A = (L

(13)

ˆ is a lower triangular matrix with diagonal entries where L equal to 0. Then, Step 3 in Algorithm 1 can be omitted. The elimination tree, TL , in Algorithm 3 can be constructed directly from GA . Specifically, TL(k) can be computed by traversing only the nonzero pattern of A in the order k = 1, 2, · · · , n. So, the algorithm can be further accelerated by using a form of path compression to avoid unnecessary traversals. This is accomplished by using a temporary vector, Ancestor[n], based on P arent[n]. Ancestor provides shortcuts to determine the roots of the subtrees containing the node i. That is, if node i is visited for the first time, then we set Ancestor[i] = k; and if node i has previously been visited, in which case Ancestor[i] = k, then the traversal is stopped. To evaluate the performance of the LDL-factorization method described above, we implemented it using C++ in a VC++ 6.0 development environment. The tests employed examples in the Harwell-Boeing collection of sparse

92

International Journal of Automation and Computing 04(1), January 2007

matrices, which can be downloaded from the following Web site: http://math.nist.gov/MatrixMarket/data/HarwellBoeing/ The examples selected are listed in Table 1. The keys provided by Harwell-Boeing are used to identify the matrices. The problems were carefully selected and are representative of the entire collection. A desktop computer (2.4-GHz Pentium 4, Windows XP, 480-MB RAM) was used to carry out the tests. For each example, the fill-reducing minimumdegree algorithm was used to order the equations (version: COLACM2.3[18] ). Table 2 shows the test results. Clearly, the elimination-tree-based LDL-factorization method presented in this paper efficiently solved problems with large sparse matrices containing thousands of nonzero entries.

vector γj for this kind of customer to be (i = 1, · · · , m, j = 1, · · · , q) γj = [γ1j · · · γmj ]T ∈ Rm . (19)

6

and the second constraint can be 8 Φξ 6 1pn , > > 2 T > > > 1l 0 ··· > > < 6 .. 6 0 . 1T 6 l > Φ = 6 > . > . . 6 > .. .. > 4 .. > > : 0 ··· 0

Application

This section discusses the application of the method described above to the solution of a one-to-one marketing optimization problem. Assume a company has p products and wants to sell them to n customers through l channels (such as e-mail, telemarketing, direct mail, etc.). It makes m = p × l offers. We introduce the variable ξij (i = 1, · · · , m, j = 1, · · · , n), which can take the following values: ( 1, if offer i is made to customer j, ξij = (14) 0, otherwise. And we define ( ξi = [ξ1i · · · ξmi ]T ∈ Rm , i = 1, · · · , n ξ = [ξ1T · · · ξnT ]T ∈ Rmn .

(15)

Note that ξi (i = 1, · · · , n) is arranged as [ξ1i · · · ξli ξ(l+1)i · · · ξ(2l)i ξ(2l+1)i · · · ξmi ]T ; the first l entries are for the first product, the next l are for the second product, and so on. Let κ ˜ i (i = 1, · · · , m) be the profit made from offer i when the product is sold to a customer. Define ( κi = [˜ κi · · · κ ˜ i ]T ∈ Rm , i = 1, · · · , n (16) T T mn κ = [κT . 1 · · · κn ] ∈ R Now, we choose the objective function for the one-to-one marketing optimization problem to be J(ξ) = κT ξ.

ξij 6 1, j = 1, · · · , pn.

(18)

i=1

Let the number of customers who ordered the product for offer i before receiving the offer be q. We define the

written as 3 0 .. 7 . 7 7 7 ∈ R(pn)×(mn) . 7 0 5 1T l

(21) Summarizing the above discussion, we formulate the oneto-one marketing optimization problem as follows[19] : max s.t.

J(ξ) = κT ξ Γ ξ = 1q , Φξ 6 1pn , ξij ∈ {0, 1}, i = 1, · · · , m, j = 1, · · · , n.

(22)

As pointed out in [20], we can relax the constraint ξij ∈ {0, 1} to 0 6 ξij 6 1 (i = 1, · · · , m, j = 1, · · · , n) and convert this 0-1 programming problem to the following LP problem: max s.t.

J(ξ) = κT ξ Γ ξ = 1q , Φξ 6 1pn , 0 6 ξij 6 1, i = 1, · · · , m, j = 1, · · · , n.

(23)

Introducing the slack variables υ ∈ Rpn and s ∈ R , and defining ξ˜ = [ξ T υ T ]T ∈ R(m+p)n and T T T c = [κ 0 ] ∈ R(m+p)n allows us to write the LP problem (23) in the following standard form: (m+p)n

cT ξ˜ Ω ξ˜ = 1q+pn ξ˜ + s = 1(m+p)n ξ˜ > 0, s > 0

max s.t.

(17)

In the marketing process, the following constraints must be considered: 1) Order Form Constraint: If customer j (j = 1, · · · , n) has already ordered the product for offer i (i = 1, · · · , m) before receiving the offer, then, ξij must be set to be 1. 2) Single Offer Constraint: Since only one channel is needed to sell a product to a customer, l X

If customer j has already ordered the product for offer i, set γij = 1; otherwise, set γij = 0. Now, the first constraint can be written as 8 Γ ξ = 1q , > > 2 T 3 > > > γ1 0 ··· 0 > > < 6 .. 7 .. 6 0 . (20) γ2T . 7 6 7 > Γ = 6 7 ∈ Rq×(mn) > > .. .. 6 .. 7 > > 4 . . . 0 5 > > : 0 ··· 0 γqT

where

#

" Ω=

(24)

Γ Φ

0 I

∈ R(q+pn)×[(m+p)n] .

(25)

Instead of solving the primal LP problem (24), we solve the following dual problem: min s.t.

y−w Ω Ty + z − w = c z > 0, w > 0

(26)

M. Wu et al./A Fast LDL-factorization Approach for Large Sparse Positive Definite System and Its Application to · · · Table 1 Key BCSPWR03 BCSPWR09 BCSPWR10 BCSSTK13 BCSSTK28 BCSSTK30

Number of eqs 118 1 723 5 300 2 003 4 410 28 924

Test examples selected from Harwell-Boeing

Number of nonzero entries in L 263 4 496 22 711 287 867 348 304 3 724 867

Table 2 Key BCSPWR03 BCSPWR09 BCSPWR10 BCSSTK13 BCSSTK28 BCSSTK30

Table 3

93

Description IEEE standard 118 bus test case power network Western US power network – 1723 bus Western US power network – 5300 bus Fluid flow generalized eigenvalues Solid element model, linear statics Statics module of an off-shore generator platform

Test Result of the LDL-factorization

Number of nonzero entries in L 263 4 496 22 711 287 867 348 304 3 724 867

Time for multiplication 1 173 27 382 253 573 69 220 200 382 242 000 865 532 000

Time for symbolic calculation (s) 0.001 800 0.000 241 0.001 088 0.003 119 0.004 632 0.053 670

Total calculation time (s) 0.004 340 0.001 837 0.008 636 0.580 561 0.247 724 7.340 698

Comparison of calculation results obtained by Cholesky-Infinity and LDL-factorization.

Scale of one-to-one marketing optimization problem Rows Columns Nonzero entries 113 1 320 5 280 203 2 400 9 600 1 003 12 000 48 000 2 003 24 000 96 000 10 003 120 000 480 000 60 003 720 000 2 880 000 100 003 1200 000 4 800 000

Cholesky-Infinity Memory (MB) 0.9 1.3 7.0 15.1 93.3 455.4 672.3

Time (s) 0.06 0.46 2.81 6.95 96.32 3 522.73 59 348.53

where y is a dual variable, and z and w are slack variables. The solutions of both the primal and dual LP problems satisfy the following Karush-Kuhn-Tucker conditions: 0 1 Ω ξ˜ − 1q+pn B ξ˜ + s − 1 C (m+p)n C B B T C F (u) = B Ω y + z − w − c C = 0 (27) B C ˜ @ A ξz sw h iT T ξ˜ z T sT wT > 0 (28) iT h where u = ξ˜T z T sT wT y T . He and Wu used the predictor-corrector method to solve this linear quadratic equation (27) with a non-negativity[19] . The algorithm is iterative; and u can be made to approach the best solution in the k-th iteration by setting ( uk+1 = uk + αk ∆uk (29) ∆uk = ∆p uk + ∆c uk where ∆p uk and ∆c uk are the predictor and corrector directions of uk , respectively. In the k-th iteration, the main computation involved in the calculation of (29) is the solving of ∆yk , which is a sub-vector of ∆uk , for (Ω Dk Ω T )∆yk = −(rbk + Ω Dk rck )

(30)

where Dk is a diagonal constant matrix, and rbk and rck are constant vectors. Since Ω is a huge sparse constraint matrix, Ω DΩ T is a huge square symmetric sparse matrix.

LDL

Objective value 32 366.90 58 703.92 297 475.66 598 514.60 2 997 575.03 18 023 178.50 30 104 984.02

Memory (MB) 0.9 1.2 6.5 14.2 90.5 432.3 520.0

Time (s) 0.05 0.35 2.14 4.32 32.82 158.75 400.65

Objective value 32 366.90 58 703.92 297 475.66 598 514.60 2 997 575.03 18 023 178.50 30 104 984.02

The Cholesky-Infinity factorization proposed by Zhang[21] (which is contained in the one-to-one marketing optimization software package LPMDP[19] ) and the method described in this paper were used to solve ∆yk in (30). The results in Table 3 show that elimination-tree-base LDL-factorization yields the same values for the objective function as Cholesky-Infinity factorization does. In other words, our method produces the same optimal results. However, compared with the Cholesky-Infinity factorization method, our method requires significantly less computation time and uses much less memory, especially when the scale is large. For example, for the last case in Table 3, the computation time for our method is only 0.68%, and the memory usage is only about 77% of the Cholesky-Infinity factorization.

7

Conclusions

This paper has presented a method of performing LDLfactorization on a sparse symmetric positive definite matrix and described its application to one-to-one marketing optimization. The computations for LDL-factorization are broken down into two stages: structural prediction and numerical calculation. This method first uses an elimination tree to construct the nonzero pattern of the factor, and then uses this structure to compute numerical values. It uses markedly less memory and involves much fewer numerical calculations when solving a problem with a large symmetric sparse matrix. Tests using numerical examples of typical sparse matri-

94

International Journal of Automation and Computing 04(1), January 2007

ces and one-to-one marketing optimization problems have revealed that this method is an efficient way of solving the LDL-factorization problem for a large sparse matrix, and thus provides significantly better performance in the solution of one-to-one marketing optimization problems.

References [1] A. Berson, S. Smith, K. Thearling. Building Data Mining Applications for CRM, McGraw-Hill Press, New York, 1999. [2] Lange, Kirsten. The Benefits of Crm. Pulp and Paper International, vol. 45, no. 11, pp. 40−42, 2003. [3] M. Tim. Crm Goes Small to Get Big Again. Telephony, vol. 246, no. 19, pp. 32−35, 2005. [4] M. Wu, B. He, S. Chen. A Benchmark Method for One-to-one Marketing Optimization Algorithms. Computer Engineering and Applications, vol. 40, no. 19, pp. 45−47, 2004. (in Chinese) [5] E. John, Mitchell, K. Farwell, D. Ramsden. Interior Point Methods for Large-scale Linear Programming, Technical Report, Mathematical Sciences Rensselaer Polytechnic Institute, Troy, NY, 2004. [6] G. W. Stewart. Building an Old-fashioned Sparse Solver, Technical Report, University of Maryland, USA, 2003. [7] D. J. Rose. A Graph-theoretic Study of the Numerical Solution of Sparse Positive Definite System of Linear Equation. Graph Theory and Computing, Academic Press, New York, pp. 183−217, 1972. [8] A. George, J. K. Reid. Computer Solution of Large Sparse Positive Definite System, Prentice-Hall, Englewood Cliffs, NJ, 1981. [9] J. W. H. Liu. A Generalized Envelope Method for Sparse Factorization by Rows. ACM Transations on Mathematical Software, vol. 17, no. 1, 1991. [10] J. R. Gilbert, T. Peierls. Sparse Partial Pivoting in Time Proportional to Arithmetic Operations. SIAM Journal on Scientific and Statistical Computing, vol. 9, no. 5, pp. 862−874, 1988. [11] T. A. Davis. User Guide for LDL, a Concise Sparse Cholesky Package, Technical Report, Department of Computer and Information Science and Engineering, University of Florida, Gainesville, FL, USA, 2005. [12] P. R. Amestoy, T. Davis, I. Du. An Approximate Minimum Degree Ordering Algorithm, SIAM Journal on Matrix Analysis and Applications, vol. 17, no. 3, pp. 886−905, 1996. [13] I. Brainman, S. Toledo. Nested-dissection Orderings for Sparse Lu with Partial Pivoting. SIAM Journal on Matrix Analysis and Applications, vol. 23, no. 4, pp. 998−1012, 2001. [14] Q. Li, N. Wang, D. Yi, Eds., Numerical Analysis, 4th ed., Qinghua University Press, Beijing, China, 2004. (in Chinese) [15] J. R. Gilbert. Predicting Structure in Sparse Matrix Computations. SIAM Journal on Matrix Analysis and Applications, vol. 15, no. 1, pp. 62−79, 1994. [16] J. W. H. Liu. The Role of Elimination Trees in Sparse Factorization. SIAM Journal on Matrix Analysis and Applications, vol. 11, no. 1, pp. 134−172, 1990. [17] S. C. Eisenstat, M. C. Gursky, M. H. Schultz, A. H. Sherman. Yale Sparse Matrix Package I: the Symmetric Codes. International Journal for Numerical Methods in Engineering, vol. 18, no. 4, pp. 1145−1151, 1982. [18] P. R. Amestoy, T. A. Davis, I. S. Du. Algorithm 837: Amd, an Approximate Minimum Degree Ordering Algorithm. ACM Transactions on Mathematical Software, vol. 30, no. 3, pp. 381−388, 2004.

[19] B. He, M. Wu, H. Mitsuma, Y. C. Zhou. An One to One Marketing Optimization Algorithm Based on Minimum Degree Permutation. In Proceedings of 23rd Chinese Control Conference, X. W. Daizhang Cheng, Ed., East China University of Science and Techology Press, Shanghai, pp. 451−453, 2004. (in Chinese) [20] K. D. Lu. Single Objective, Multiple Objective and Integer Programming, Tsinghua University Press, Beijing, 1999. (in Chinese) [21] Y. Zhang. Solving Large-scale Linear Programs by Interiorpoint Methods Under the Matlab Environment. Optimization Methods and Software, vol. 10, no. 1, pp. 1−31, 1998.

Min Wu was born in 1963. He received his B.Sc. and M.Sc. degrees in engineering from the Central South University, Changsha, China in 1983 and 1986, respectively. In July, 1986, he joined the staff of the university, where he is currently a professor of automatic control engineering. He was a visiting scholar at the Department of Electrical Engineering, Tohoku University, Sendai, Japan, from 1989 to 1990, and a visiting research scholar at the Department of Control and Systems Engineering, Tokyo Institute of Technology, Tokyo, Japan, from 1996 to 1999. He received his Ph.D. degree in engineering from Tokyo Institute of Technology, Tokyo, Japan in 1999. His research interests include robust control and its application, process control, and intelligent control. Dr. Wu received the best paper award at IFAC 1999 (jointly with M. Nakano and J.-H. She). He is a member of the Nonferrous Metals Society of China and the Chinese Automation Association. Bei He was born in 1979. She received her B.Sc. degree in engineering from Xiangtan Polytechnic University, Xiangtan, China in 2001, and her M.Sc. degree in engineering from Central South University, Changsha, China in 2003. She is currently pursuing her Ph.D. degree in control engineering at the Central South University. Her research interests include system engineering, optimization methods, and marketing optimization. Jin-Hua She received his B.Sc. degree in engineering from Central South University, Changsha, China, in 1983, and his M.Sc. and Ph.D. degrees in engineering from the Tokyo Institute of Technology, Tokyo, Japan, in 1990 and 1993, respectively. In 1993, he joined the Department of Mechatronics, School of Engineering, Tokyo University of Technology; and in April, 2004 he transferred to the University’s School of Bionics, where he is currently an associate professor. His current research interests include the application of control theory, repetitive control, expert control, Internet-based engineering education, and robotics. Dr. She received the control engineering practice paper prize at IFAC 1999 (jointly with M. Wu and M. Nakano). He is a member of IEEE, the Society of Instrument and Control Engineers (SICE), and the Institute of Electrical Engineers of Japan (IEEJ).