solving strike force asset allocation problems: an ...

SOLVING STRIKE FORCE ASSET ALLOCATION PROBLEMS: AN APPLICATION OF 0/1 MULTIDIMENSIONAL KNAPSACK PROBLEMS WITH MULTIPLE-CHOICE CONSTRAINTS

A Dissertation by VINCENT CHI-WEI LI

Submitted to the Oce of Graduate Studies of Texas A&M University in partial fulllment of the requirements for the degree of DOCTOR OF PHILOSOPHY

August 2001

Major Subject: Industrial Engineering

SOLVING STRIKE FORCE ASSET ALLOCATION PROBLEMS: AN APPLICATION OF 0/1 MULTIDIMENSIONAL KNAPSACK PROBLEMS WITH MULTIPLE-CHOICE CONSTRAINTS A Dissertation by VINCENT CHI-WEI LI Submitted to Texas A&M University in partial fulllment of the requirements for the degree of DOCTOR OF PHILOSOPHY Approved as to style and content by:

Guy L. Curry (Co-Chair of Committee)

E. Andrew Boyd (Co-Chair of Committee)

Brett A. Peters (Member)

Daniel R. Lewis (Member)

Bryan L. Deuermeyer (Head of Department) August 2001 Major Subject: Industrial Engineering

iii

ABSTRACT Solving Strike Force Asset Allocation Problems: An Application of 0/1 Multi-dimensional Knapsack Problems with Multiple-Choice Constraints. (August 2001) Vincent Chi-Wei Li, B.B.A., National Chiao-Tung University M.S., Texas A&M University Co{Chairs of Advisory Committee: Dr. Guy L. Curry Dr. E. Andrew Boyd The strike-force asset-allocation problem consists of grouping strike force assets into packages and assigning these packages to targets and defensive assets in a way that maximizes the strike force potential. Integer programming formulations are developed, semi-real and random problems are built up, various approaches of a tabu search heuristic are developed, and computational results via both the CPLEX MIP Solver and the tabu search heuristic are presented. The use of tight oscillation near the feasibility boundary in the tabu search heuristic helps to obtain good solutions very quickly. The merits of using surrogate-constraint information versus using a Lagrange multiplier approach are demonstrated. Intensifying the neighborhood of good solutions via various trial solution approaches helps to obtain solutions of high quality.

iv

To Vivian

v ACKNOWLEDGMENTS First, I would like to thank Dr. Andy Boyd for suggesting a very challenging topic and providing funding support to start my research. Dr. Boyd gave me many valuable suggestions and helped me to build my professional strength. I wish to express my gratitude to Dr. Guy Curry for his guidance, encouragement, patience and strong support in this research. I also would like to thank the other committee members, including Dr. Brett Peters, Dr. Dan Lewis, and Dr. Rekha Thomas for their valuable comments and suggestions on my research proposal and dissertation. Many thanks to Dr. Fred Glover for his invaluable suggestions and comments on my research. Dr. Glover's thorough support and guidance helped me tremendously to go through the dicult time of my Ph.D. study. I also would like to extend my thanks to Dr. Way Kuo and Dr. Richard Feldman for their support in my study. Special thanks to Dr. John Yen, Monica Her, ChiLeung Chu, Debra Elkins, Judy Meeks, and Jasmine Chang, for their friendship and various help during my study. I am indebted to Vivian, Wilson and Eric. Their strong support and patience makes my Ph.D. dream come true.

vi

TABLE OF CONTENTS CHAPTER

Page

I

INTRODUCTION : : : : : : : : : : : : : : : : : : : : : : : : : : A. Literature Survey . . . . . . . . . . . . . . . . . . . . . . . . B. Research Scope . . . . . . . . . . . . . . . . . . . . . . . . .

1 1 6

II

FORMULATIONS OF PROBLEMS : : : : : : : : : : : : : : : : A. Strike Force Asset Allocation Problem without Suppression . B. Strike Force Asset Allocation Problem with Suppression . . .

8 8 10

III

METHODOLOGY : : : : : : : : : : : : : : : : : : : : : : : : : A. Development of Asset Allocation Problems without Suppression 1. Semi-Real Problems for Model 1 . . . . . . . . . . . . . 2. Random Problems for Model 1 . . . . . . . . . . . . . . B. Development of Asset Allocation Problems with Suppression 1. Semi-Real Problems for Model 2 . . . . . . . . . . . . . 2. Random Problems for Model 2 . . . . . . . . . . . . . . C. Sizes of Asset Allocation Problems for Both Models . . . . .

17 19 19 22 23 24 25 26

IV

COMPUTATIONAL RESULTS MIP SOLVER : : : : : : : : : : A. Results for Model 1 . . . . . B. Results for Model 2 . . . . .

29 29 38

V

HEURISTIC APPROACHES : : : : : : : : : : : : : : : : : : : A. Problem Formulations and Complexity . . . . . . . . . . . . B. Heuristics for the 0-1 Multi-Dimensional Knapsack Problem . C. Heuristics for the Multi-Dimensional Multiple-Choice Knapsack Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . D. Lagrange Multiplier Approaches . . . . . . . . . . . . . . . .

41 41 43

TABU SEARCH HEURISTICS : : : : : : : : : : : : : : : : : : A. An Overview of Meta-Heuristics . . . . . . . . . . . . . . . . B. An Overview of Tabu Search . . . . . . . . . . . . . . . . . .

55 55 56

VI

VIA ::: ... ...

CPLEX ::::: ..... .....

PARALLEL :::::::::: .......... ..........

43 45

vii CHAPTER

Page C. Review of Tabu Search Approaches for the 0-1 MultiDimensional Knapsack Problem . . . . . . . . . . . . . . D. Method in Detail . . . . . . . . . . . . . . . . . . . . . . E. Choice Rules . . . . . . . . . . . . . . . . . . . . . . . . . 1. Lagrangian Relaxation Based Choice Rules . . . . . 2. Surrogate Constraint Based Choice Rules . . . . . . F. Normalization for Surrogate Constraint . . . . . . . . . . G. Memory Management and Oscillation Scheme . . . . . . . 1. Penalty Adjusted Ratio . . . . . . . . . . . . . . . . 2. GUB-size Adjusted Tenures . . . . . . . . . . . . . . 3. Controlling Span . . . . . . . . . . . . . . . . . . . . H. Generating Trial Solutions . . . . . . . . . . . . . . . . . I. Tight-Oscillation Trial Solutions . . . . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

57 59 62 64 67 68 70 70 72 73 73 75

VII

COMPUTATIONAL RESULTS FROM TABU SEARCH HEURISTIC APPROACHES : : : : : : : : : : : : : : : : : : : : : : : : A. Preliminary Results Based on Choice Rules . . . . . . . . . . B. Preliminary Results of Strong Surrogate Constraint Approach C. Preliminary Results of Basic Trial Solution Approach . . . . D. Rening Experiments . . . . . . . . . . . . . . . . . . . . . . E. Results of Tight Oscillation . . . . . . . . . . . . . . . . . . .

VIII

CONCLUSIONS AND FUTURE RESEARCH : : : : : : : : : : 102 A. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 B. Future Research Directions . . . . . . . . . . . . . . . . . . . 104

80 83 87 91 92 93

REFERENCES : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 106 APPENDIX A : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 114 VITA : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 116

viii

LIST OF TABLES TABLE

Page

I

Notation Used in Figures 1, 2 and 3 : : : : : : : : : : : : : : : : : :

19

II

Comparison of Semi-Real Problem Sizes : : : : : : : : : : : : : : : :

28

III

Performance of Three Semi-Real Problems in Model 1 via CPLEX 6.6 Parallel Solver : : : : : : : : : : : : : : : : : : : : : : : : : : : :

32

IV

Performance of Nine Instances of Random Problem 1 in Model 1 via CPLEX 6.6 Parallel Solver : : : : : : : : : : : : : : : : : : : : :

33

V


34

VI


35

VII

Performance of Random Problem 3 in Model 1 with 100% Variables at Dierent RHS Values via CPLEX 6.6 Parallel Solver : : : :

37

VIII

Performance of Nine Instances of the Random Problem in Model 2 via CPLEX 6.6 Parallel Solver : : : : : : : : : : : : : : : : : : : : :

39

IX

Performance of the Random Problem in Model 2 with 100% Variables at Dierent RHS Values via CPLEX 6.6 Parallel Solver : : : :

40

X

Results of Random Problem 1 in Model 1 with 100% Variables at Dierent RHS Values by Heuristic MJS : : : : : : : : : : : : : : : :

52

XI


53

XII


54

XIII

Size of Thirty Problem Instances : : : : : : : : : : : : : : : : : : : :

81

ix TABLE

Page

XIV

Performance Comparison between Simple Surrogate and Lagragian Relaxation Approaches of Problem 1 : : : : : : : : : : : : : : :

84

XV


85

XVI


86

XVII

Performance Comparison among Dienent Approaches Using Surrogate Constraints and Trial Solutions of Problem 1 : : : : : : : : : :

88

XVIII Performance Comparison among Dienent Approaches Using Surrogate Constraints and Trial Solutions of Problem 2 : : : : : : : : : :

89

XIX

Performance Comparison among Dienent Approaches Using Surrogate Constraints and Trial Solutions of Problem 3 : : : : : : : : : :

90

XX

Comparison of Results by Dierent Power Values in the Strong Surrogate Approach : : : : : : : : : : : : : : : : : : : : : : : : : : :

93

XXI

Performance Comparison between Strong Surrogate and Basic Trial Solution Approaches of Problem 1 : : : : : : : : : : : : : : : :

94

XXII

Performance Comparison between Strong Surrogate and Basic Trial Solution Approaches of Problem 2 : : : : : : : : : : : : : : : :

95

XXIII Performance Comparison between Strong Surrogate and Basic Trial Solution Approaches of Problem 3 : : : : : : : : : : : : : : : :

96

XXIV Performance of Tight Oscillation Using the \Complete" and the \Fast" Strategies of Problem 1 : : : : : : : : : : : : : : : : : : : : :

99

XXV

Performance of Tight Oscillation Using the \Complete" and the \Fast" Strategies of Problem 2 : : : : : : : : : : : : : : : : : : : : : 100

XXVI Performance of Tight Oscillation Using the \Complete" and the \Fast" Strategies of Problem 3 : : : : : : : : : : : : : : : : : : : : : 101 XXVII Comparison between Complete Tight Oscillation and CPLEX : : : : 101

x

LIST OF FIGURES FIGURE

Page

1

WA Map : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

20

2

NY Map : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

20

3

CA Map : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

21

1 CHAPTER I INTRODUCTION Strike force planning is one of the fundamental problems faced by the armed services. In a strike force planning problem, there is a set of potential targets and a collection of assets that may be allocated to targets. There may also be a set of threats that serve as defenders of the targets. The goal is to determine the allocation and the routing of strike force assets so as to maximize strike eectiveness while limiting potential damage to the strike force from the defenders. The strike force planning problem is solved before an engagement, although the problem of strike force management during the engagement is also a question of practical importance. With recent advances in information processing technology, especially in parallel computation, the potential for \real time" strike force planning| on the order of a few minutes| is now realizable. A. Literature Survey The strike force planning problem includes routing and allocating the available assets. The routing problem has received attention in several papers (Boroujerdi et al. 1993 Boroujerdi 1994 Zuniga et al. 1994 Lee 1995 Zuniga and Gorman 1995). Zuniga and Gorman (1995) discuss strike route optimization and threat site overight modeling. They present some general graph theoretic techniques for route optimization. Zuniga et al. (1994) focus on the inter-dependent joint routing problem. They analyze routing a collection of assets to a collection of targets. Lee (1995) constructs a network and uses a shortest path algorithm to obtain a very good route The journal model is Technometrics.

2 for the strike force aircraft. As to the asset allocation problem, Matlin (1970), Eckler and Burr (1972) and Cheong (1985) give reviews on the missile allocation problem. Cho (2000) discusses how to allocate a given number of weapons to targets with target-dependent kill probabilities. While most of the literature models the weapon-target allocation problem in a stationary way, Hosein (1989) treats it as a dynamic-nonlinear resource-allocation problem. Hosein and Athans (1990) give analytic results for the dynamic nonlinear weapon-target allocation problem in simple cases. They give asymptotic results when the number of targets becomes very large. Work by Jakobovits (1995) highlights the core of the asset allocation problem. Jakobovits separates a strike plan into an attack plan , which focuses on primary targets, and a suppression plan , which seeks to suppress defensive threats. He addresses the asset allocation problem in this general setting and solves some small problem instances, but he points out that larger instances quickly become very dicult to solve. In this dissertation, routing information is omitted, and two models of the striking asset allocation problem are considered. In the rst model, the problem is formulated as a large-scale integer program consisting of a collection of 0-1 knapsack constraints and a collection of mutually disjoint multiple-choice constraints. A knapsack constraint has the form ax b, where x is a column vector of binary variables, a is a nonnegative row vector and b is non-negative. The multiple-choice knapsack problem, also known as the knapsack problem with generalized upper bound (GUB) constraints, is a 0-1 knapsack problem in which a partition of the variable set is given, and it is required that exactly one variable per subset is selected based upon a denition stated by Martello and Toth (1990). The multiple-choice constraints are all bounded above by 1. The problem we study requires that at most one variable

3 instead of exactly one variable per subset be selected. However, it is very straight forward to transform the problem to the form just mentioned, simply by adding a slack variable to each multiple-choice constraint. In this dissertation, we will use multiple-choice and GUB interchangeably for convenience. The second model further incorporates precedence constraints in addition to a collection of knapsack constraints and a collection of multiple-choice constraints. A wealth of literature exists on the polyhedral structure of knapsack constraints and how this structure can be applied to help solve integer programs (Balas 1975 Hammer, Johnson and Peled 1975 Wolsey 1976 Balas and Zemel 1978 Crowder, Johnson and Padberg 1983 Boyd 1993). Salkin and de Kluyver (1975) present a survey on the solution approaches to one dimensional knapsack constraint. Numerous exact algorithms and heuristics have been developed since then. There are many papers on GUB constraints, and more generally on the set packing problem (Padberg 1973 Nemhauser and Trotter 1974 Trotter 1975 Wolsey 1976 Padberg 1979 Padberg 1980 Nemhauser and Wolsey 1988). As the computational results demonstrate, the problem structures aorded by these constraints yield integer programs that can be solved close to 1% from the best bound of the optimal solutions very quickly via the Parallel MIP Solver (MIP is an abbreviation for mixed integer programming) of CPLEX 6.6 (1999), even for relatively large problem (15,000 binary variables and 90 constraints) instances. In addition to the models presented and studied computationally, a set of semi-real problems has been developed based on data procured from the United States Geological Survey. In addition, a set of randomly generated problems were created. It is hoped that other researchers will nd this set of test problems interesting and useful for future research in this problem area. The problems and contour maps are available upon request. Although CPLEX 6.6 solved most of test set problems very eciently, it was

4 much more harder to solve these problems using CPLEX version 4 when this research was started. These original computational problems motivated the investigation of heuristic methodologies targeted to the class of problems illustrated in Chapter II. The rst formulation can be categorized as a multi-dimensional multiple-choice knapsack problem or 0-1 multi-dimensional knapsack problem with GUB constraints . There are many applications for multi-dimensional multiple-choice knapsack problems, such as capital budgeting and resource allocation. The 0-1 knapsack problem can be seen as a special case of the multi-dimensional multiple-choice knapsack problem. Since the former problem is NP-complete (see Garey and Johnson 1979), the latter problem is also NP-hard. The multi-dimensional multiple-choice knapsack problem can be treated as a special case of the 0-1 multi-dimensional knapsack problem. The 0-1 multi-dimensional knapsack problem is NP-hard (Magazine and Oguz 1984) too since it is a generalization of the 0-1 knapsack problem. Although fully polynomial approximation algorithms (see Magazine and Chern 1984) for the 0-1 knapsack problem have been developed (Magazine and Oguz 1981), nding fully polynomial approximation algorithms when there is more than one constraint is NP-complete (Magazine and Chern 1984). This feature makes it necessary to design good heuristics to solve 0-1 multi-dimensional knapsack problems in polynomial time. A comprehensive survey of approximation algorithms and heuristics for the 0-1 multi-dimensional knapsack problem can be found in Lin (1998) as well as in Chu and Beasley (1998). To solve the multi-dimensional multiple-choice knapsack problem, Moser et al. (1997) propose a heuristic based on the Lagrange multiplier method developed by Everett (1963). The authors point out the heuristic runs well for small problem instances. However, we show the heuristic fails to be eective in handling most of our large-scale instances. This gap in the eectiveness of this heuristic is probably due

5 to the fact that Moser et al. were only interested in solving some rather small applications (Moser, 1998, pers. comm.). Some literature also discusses multiple-choice programming, but most of these papers focus on the multiple-choice problem with a single knapsack constraint. A state-of-the-art review on multiple-choice programming can be found in Lin (1994). The simple (integer) dominance concept is widely used in the multiple-choice programming literature. However, the value of this concept diminishes when the number of constraints increases given that the number of variables in each GUB set is xed (see Dyer and Walker 1998). Consequently, the simple dominance concept only has a signicant eect on reducing the size of multi-dimensional multiple-choice knapsack problems when the number of knapsack constraints is quite small. Traditional ascent or descent heuristics often become trapped in local optima when dealing with hard or large-scale combinatorial-optimization problems. To overcome the problem of being conned by local optimality, meta-heuristics, including but not limited to simulated annealing, genetic algorithms, tabu search, neural networks, greedy randomized adaptive search procedures (also known as GRASP), scatter search, and ant colony systems, guide and modify subordinate heuristics to efciently nd near-optimal solutions. An overview of meta-heuristics can be found in Osman et al. (1996) and Aarts et al. (1997). A number of heuristics and metaheuristics have been proposed to solve the 0-1 multi-dimensional knapsack problem, and a comprehensive survey can be found in Chu and Beasley (1998). Among those meta-heuristics, tabu search has been successfully applied to solving 0-1 multi-dimensional knapsack problems (Battiti and Tecchiolli 1995 Glover and Kochenberger 1996 Hana and Freville 1998 Lokketangen and Glover 1998). Therefore, it is reasonable to assume that tabu search could be an eective method for the class of multi-dimensional multiple-choice knapsack problems. An ecient tabu

6 search heuristic is developed to solve the multi-dimensional multiple-choice knapsack problem in this dissertation. The tabu search term was introduced by Glover (1986). Tabu search which incorporates adaptive memory and responsive exploration as a basis for solving problems, has proved highly eective in diverse applications. A comprehensive guide for tabu search can be found in Glover and Laguna (1998). Glover and Kochenberger (1996) create a tabu search approach (referred to as the GK heuristic) whose underlying memory structures are arranged according to critical events. A balance between intensication, which searches through attractive regions, and diversication, which searches through new regions, is achieved by an oscillation strategy that navigates around the feasible boundary. Surrogate constraints (developed by Glover 1968) are used to obtain choice rules for this approach. The performance of this approach is superior to previous heuristics for the 0-1 multi-dimensional knapsack problem. Hana and Freville (1998) also propose a tabu search heuristic similar to the GK heuristic and get improvements in some of the problems tested by the GK heuristic. B. Research Scope The goal of this study has been to identify ways to eciently solve the strike force asset allocation problem. The problem is formulated as a multi-dimensional multiplechoice knapsack problem. Suppression of defenders (threats) is also considered. Data sets of semi-real and random characteristics were generated to form a set of test problems. The test problems were rst solved by CPLEX MIP Solver (version 4.0). The results were not good for several test problems. The diculty of solving some of the problems motivated the investigation of heuristics for this problem class. One of the existing heuristics was critiqued. To design good heuristics for this problem

7 class, the tabu search methodology was studied. Chapter II presents two models for the asset allocation problem with/without suppression of threats. Chapter III covers the methodology for generating the semireal and the random problems for both of the models. The CPLEX computational results for test problems of both of the models are presented in Chapter IV. In Chapter V, an overview is given for heuristics designed for the 0-1 multi-dimensional knapsack problem and for the multiple-choice knapsack problem. Particularly, one of the published methods using a Lagrange multiplier approach for the multi-dimensional multiple-choice knapsack problem is critiqued. In Chapter VI, various approaches of a tabu search heuristic are developed for this problem class. The merits of using surrogate constraint information are demonstrated. The methods and advantages of exploring the neighborhood of good solutions are presented. The computational results from several versions of tabu search approaches are reported in Chapter VII. Finally, Chapter VIII summarizes the research contributions and highlights directions for future research.

8 CHAPTER II FORMULATIONS OF PROBLEMS A practical aspect of strike force asset allocation is that often more than one asset is required to attack a target or suppress a threat. We will use threats and defenders interchangeably in this dissertation. There are several types of strike force assets represented in the form of dierent aircraft and missiles. To illustrate, a radar site might require two missiles of type I, and a naval base might require one aircraft of type I and one missile of type II. Moreover, there might be other combinations which can be used to attack a target or suppress a threat. In this dissertation, the collection of assets capable of destroying a target and a threat will be referred to as an attack package and a suppression package , respectively. Model 1 considers only targets. This model is generalized to incorporate threat suppression as Model 2. A. Strike Force Asset Allocation Problem without Suppression Let C be the set of all asset classes available and T be the set of all targets available. Identical assets are referred to as the same asset type (independent of location). An asset class is an asset type with a specic location. For example, two identical aircraft in dierent locations are of the same asset type but dierent asset classes. The cost of assigning the same asset at dierent locations to a given target may be dierent due to the distance between the target and the asset. For simplicity, assume that an asset close to its corresponding target will have less opportunity for asset loss, therefore, the distance between each asset in an attack package and the corresponding target contributes location dependent values associated with the attack package value. Before stating the formulation, dene some notation as follows:

9

A : the set of all possible target attack packages, At : a subset of A which consists of all attack packages associated with target t where the cardinality of At is jAt j, gta : the prot associated with assigning attack package a to target t, where a 2 At , ecta : number of class c assets contributing to attack package a for target t, where a 2 At , b : a column vector of size jC j, where bc is the number of assets available in asset class c, xta : equals 1 if attack package a is assigned to target t, and equals 0 otherwise, where a 2 At . The goal is to maximize the damage yield which is both target and package dependent as expressed in Equation 3.1 by the assignment of attack packages subject to asset class congurations and availability constraints. Each attack package has an associated binary variable in the model, and each attack package aects the objective value in an additive manner according to its coecient if the attack package is chosen each attack package contributes nothing to the objective value if it is not chosen. It is assumed each target is assigned at most one attack package. This model can be described in matrix notation (see Appendix A). However, to be consistent with the notation of the suppression model elaborated later, the explicit summation representation is used to describe the model as follows:

10

Model 1 max s.t.

XX

t2T a2At

XX

t2T a2At X

gta xta

(2.1)

ecta xta bc 8c 2 C

(2.2)

xta 1 8t 2 T

a2At xta 2 f0 1g

8t 2 T a 2 At:

(2.3) (2.4)

The objective function (2.1) assigns a prot gta for package a corresponding to target t (a 2 At ) if the package is chosen (xta = 1). Constraints (2.2) guarantee that whatever attack packages are chosen, the overall demand for each asset class will not exceed the supply. Constraints (2.3) guarantee that no target will be assigned more than one attack package which is an assumption for Model 1. As formulated, practical instances will have a reasonably small number of constraints and a large number of variables. For instance, the number of classes in C and the number of targets in T may typically sum to one hundred. However, the number of attack packages can increase quickly as the number of asset classes increases. B. Strike Force Asset Allocation Problem with Suppression Next, threat suppression is incorporated into Model 1. Each threat can defend one or more targets in its neighborhood. Each target can have one or more threats protecting it as well. To eectively destroy the target, suppression needs to be considered for the threats that protect the target. One method of increasing the gain of an attack on a target is to conduct suppression of the corresponding threats another method is to assign multiple packages to the same target. Dene the maximum number of attack packages assigned to a target as the attack level and the maximum

11 number of suppression packages assigned to a threat as the suppression level. Thus, dierent levels of attack can be made on each target. Similarly, dierent levels of suppression can be assigned to each threat. In Model 1, only one level of attack for each target is allowed and no suppression of threat is considered. Now divide the targets into targets and threats. More precisely, we add threats to the model and dene threats as both a target and a target defender. A threat is a target in its own right as well as a defender of more important targets. Each target has its own value, as does each threat. (This issue will be discussed in Chapter III.) Based on the value of a target, one or more attack packages are chosen to attain sucient destruction. (To match the requirement of attacking a target, a package is required to create a prot, i.e., destroying power, in some threshold.) By the same token, suppression packages are chosen for each threat. The degree of success in attacking the target is related to the level of suppression associated with the threats defending the target. The eectiveness of the attack packages assigned to a given target is reduced if the corresponding defenders are not suppressed. The target damage eectiveness would be improved with higher levels of defender suppression. The eectiveness of the suppression packages assigned to a given threat is solely determined by the suppression itself and does not depend on attacking the defended targets. To develop the suppression model, the following notation is needed:

D : the set of all threats (defenders), S : the set of all possible threat suppression packages, p] : the set of f 1, 2, , p g, where p is a positive integer,

Sd : a subset of S which consists of all suppression packages associated with threat d

12 where the cardinality of Sd is jSdj,

M : the highest desirable level of attack: Attack levels range from 1 to M where M is a positive integer and M is xed for each target. N : the highest desirable level of suppression: Suppression levels range from 1 to N where N is a positive integer and N is xed for each threat. wta : the valid fraction of attacking power for attack package a on target t without considering suppression, where a 2 At, n : the additional fraction of attacking power for attack package a on target t when wtad P P n = 1:0, threat d is suppressed at level n, where a 2 At and wta + n2 N d2D wtad ]

gta : the prot associated with assigning attack package a to target t before evaluating suppression, where a 2 At, hds : the prot associated with assigning suppression package s to threat d, where s 2 Sd (Each threat and each target have their own values.) fcds : number of class c assets contributing to suppression package s, where s 2 Sd , yds : equals 1 if suppression package s is assigned to threat d, and equals 0 otherwise, where s 2 Sd , tm : equals 1 if target t is attacked at level m or above, that is, there are at least m attack packages assigned to target t, and equals 0 otherwise, dn : equals 1 if threat d is suppressed at level n or above, that is, there are at least n suppression packages assigned to threat d, and equals 0 otherwise.

13 The goal is to maximize the target and threat damage values subject to asset class assignment congurations and availability constraints. The model formulation is as follows:

Model 2 max s.t.

XX

t2T a2At

XX

t2T a2At X

a2At X

s2Sd tm

(wta +

n2 N d2D

n )g x + dn wtad ta ta

]

ecta xta +

xta ;

yds ;

+1

X X

X

n2 N m ; t 0

dn ; dn +1

xta 2 f0 1g yds 2 f0 1g tm 2 f0 1g dn 2 f0 1g

d2D s2Sd

hdsyds

fcdsyds bc 8c 2 C

(2.5) (2.6)

d2D s2Sd m t = 0 8t 2 T

X

m2 M

XX

XX

(2.7)

]

dn = 0 8d 2 D

(2.8)

]

8t 2 T m 2 M ; 1] 0 8d 2 D n 2 N ; 1] 8t 2 T a 2 At 8d 2 D s 2 Sd 8t 2 T m 2 M ] 8d 2 D n 2 N ]:

(2.9) (2.10) (2.11) (2.12) (2.13) (2.14)

For each attack package a corresponding to target t (a 2 At ), the objective function (2.5) assigns a prot wta gta , which represents the gain without considering the P P n gta , suppression of the corresponding threats, and assigns a prot n2 N d2D dnwtad which represents the marginal gains associated with the suppression of the corresponding threats. For each suppression package s 2 Sd, the objective function assigns a prot hds , which represents the gain due to the suppression of the corresponding threat d itself. Notice that the coecient for each suppression package is much simpler than the coecient for each attack package. The gain associated with the assignment of an ]

14 attack package to a target depends on the suppression of the target's defenders. But threats do not have defenders and, hence, the gain associated with the assignment of a suppression package on a threat is less complex. The objective function is nonlinear in this model formulation. This issue will be addressed later in this chapter. Constraints (2.6) guarantee that whatever packages are chosen, the overall demand for each asset class will not exceed the supply. Constraints (2.7) guarantee that no target will be assigned more than M attack packages, while constraints (2.8) guarantee that no threat will be assigned more than N suppression packages. Constraints (2.9) enforce the precedence relation that higher levels of attack on a target will be considered only after all lower levels of attack on that target have been assigned. Similarly, constraints (2.10) enforce the precedence relation that higher levels of suppression of a threat will be considered only after all lower levels of suppression of that threat have been assigned. Model 2 contains nonlinear terms (products of dn and xta for all d 2 D, n 2 N ], n , the fraction of t 2 T , and a 2 At ). By introducing a continuous variable set ztad additional gain of the attack package a 2 At associated with suppression of threat d at level n, the nonlinear objective (2.5) of Model 2 can be transformed into a linear objective as max

XX

t2T a2At

wta gta xta +

XX

d2D s2Sd

hds yds +

XX X X

t2T a2At n2 N d2D

n g zn wtad ta tad

(2.15)

]

n is between 0 and min(n xta ). That is, if there is no suppression of threat where ztad d n will be zero and no additional value will be added into d at level n (dn = 0), then ztad n the objective function if there is suppression of threat d at level n (dn = 1), then ztad n 's have will be bounded above by xta . As this is a maximization problem and the ztad n 's will be at their upper bounds in the optimal positive coecients, the values of ztad

15 n 's to be 0 or 1 in any optimal solutions. solution. This characteristic enforces all ztad n can be treated as a set of binary variables, and the whole model is a Therefore, ztad linear integer programming problem. The linear transformation of Model 2 is listed below. XX

max

wta gta xta +

t2T a2At XX X X t2T a2At n2 N d2D

XX

d2D s2Sd n g zn wtad ta tad

hdsyds + (2.16)

]

s.t.

XX

t2T a2At X a2At X

ecta xta +

xta ;

X

(2.17) (2.18)

]

dn = 0 8d 2 D

(2.19)

8t 2 T m 2 M ; 1] dn ; dn 0 8d 2 D n 2 N ; 1] n ; n 0 8t 2 T a 2 A d 2 D n 2 N ] ztad t d n ;x ztad ta 0 8t 2 T a 2 At d 2 D n 2 N ] n 2 f0 1g 8t 2 T a 2 A d 2 D n 2 N ] ztad t xta 2 f0 1g 8t 2 T a 2 At yds 2 f0 1g 8d 2 D s 2 Sd tm 2 f0 1g 8t 2 T m 2 M ] dn 2 f0 1g 8d 2 D n 2 N ]:

(2.20)

s2Sd tm

yds ;

fcdsyds bc 8c 2 C

d2D s2Sd m t = 0 8t 2 T

X

m2 M

XX

+1

+1

n2 N m ; t 0 ]

(2.21) (2.22) (2.23) (2.24) (2.25) (2.26) (2.27) (2.28)

16 The objective function (2.16) assigns a prot wta gta xta representing the gain without considering suppression of the corresponding threats, and assigns a prot P P n n n2 N d2D wtad gta ztad , representing the marginal gains with suppression of the corresponding threats. For each suppression package s 2 Sd , the objective function assigns a prot hds , which represents the direct gain of suppression of the corresponding threat d. ]

17 CHAPTER III METHODOLOGY Important issues in designing realistic asset allocation problems was discussed in the Real Time Strike Force Planning Workshop in July 1995 at the Naval Research Laboratory in Washington, D.C. Because real problems were not obtainable for our computational research, we constructed an algorithm to generate problems that were as similar as possible to realistic scenarios. Two types of problems were created: semireal problems, which represent possible military scenarios in the dierent regions of the United States, and randomly generated problems, which do not have associated topographical areas. We will discuss characteristics that both problem types share in the remainder of this section. In the following sections, we will discuss how the problem types dier. For Model 1, all problems consist of two types of data: asset classes and targets for Model 2, all problems consist of three types of data: asset classes, targets and threats. An asset class is dened as an asset type assigned to a particular location. Assets include carrier-based ghters and cruise missiles, targets include airports, metropolitan areas, bridges, railroads, military tactical headquarters, satellite stations, radar locations, and chemical plants, and threats include military bases, air force bases, and naval bases. The unit cost per asset is assumed known. Six types of planes and six types of missiles are considered. The sets fC, D, E, F, G, Hg and fS, T, U, V, W, Xg (The set notation used here is not related to any notation dened before in Chapter II.) represent the dierent ghter types and missile types, respectively. Each problem uses some subset of the plane and missile types as possible asset types. For every problem, an asset class ratio of two locations per type of ghter to three locations per type of missile was used to specify how asset types are assigned

18 to class locations. For example, in a problem with twenty asset classes, eight of the classes specify locations for types of planes and twelve of the classes specify locations for types of missiles. Given the information about the asset types available and the unit cost per asset, a list of available options is then specied. Each option is a combination of dierent asset types, with the number of each asset type being specied (note that an option diers from a package in that packages are a collection of asset classes). For example, an option j , denoted 2F2U4V, is composed of two planes of type F, two missiles of type U, and four missiles of type V. The cost per option is the sum of the costs per asset multiplied by the number of assets used in the option. Note that the cost per asset type is assumed independent of location. Continuing with the example for option j , the cost associated with this option is 2CF + 2CU + 4CV , where CF is the cost of using a unit of asset type F, etc. Each asset class has associated with it the (X, Y) map location, the cost per unit of asset type, and the capacity (number) of assets at this location. For example, class 7, located at (457.95, 26.65), is comprised of ten missiles of type T, each of which has a cost of 11.0 associated with it. Similarly, targets are identied by an ID number, (X, Y) location, prot value associated with destroying the target, the number of dierent possible options available to destroy the target, and which particular options are available. For example, target ID 10 has a prot value of 60.37, is located at (217.29, 160.74), and has three candidate options which can be used to destroy this target. The three options are: 2F 3V , 2F 3U , and 2E 3V . The semi-real and random problems dier in how this information is obtained, and the dierences are discussed further in the following two subsections (for Model 1 and Model 2, respectively).

19

TABLE I. Notation Used in Figures 1, 2 and 3 Symbol Target Symbol Target A Airport M Metropolitan Area AB Air Force Base MB Military Base B Bridge NB Naval Base C Chemical Plant R Radar H Tactical Headquarters S Satellite Station J Aircraft Carrier X Railroad K Cruiser A. Development of Asset Allocation Problems without Suppression We rst discuss the semi-real and random problems for Model 1 in this section and for Model 2 in Section B. 1. Semi-Real Problems for Model 1 For semi-real problems, contour maps of dierent regions of the United States were generated in MATLAB using data from the United States Geological Survey. Three dierent map regions were developed: western Washington State, including the Seattle metropolitan area, the Northeast, including New York, New Jersey, Connecticut, and Massachusetts, and Northern California, including the San Francisco Bay area. The Washington State map, Figure 1, is a problem of 10 asset classes and 20 targets. The New York map, Figure 2, is a problem of 20 asset classes and 40 targets, and the California map, Figure 3, is a problem of 30 asset classes and 60 targets. Table I describes the notation used in these three gures.

20

WA Elevation Contours (feet) (750,1500,3000,6000,9000) 200

latitude (46−49) Distance (Miles)

R

R

H

150 NB R M

NB 100

NB M H

K K

50

H

A X AB MB MB

A X A M

K

J J 0 0

50

100

150 200 250 Longitude (128−120) Distance (Miles)

300

350

FIG. 1. WA Map NY Elevation Contours (feet) (250,500,750,1000,2000,3000) 200

R

MB

Latitude (40−43) Distance (Miles)

X AB 150

NB

A M

A M

MB

R MB

J

X S

H M C

B

R 100

J

K K

H

R S

K

C S AB

AB

MB

NB A B X X

50

A

M NB H K

A

M X

J J

NB

K K

0 0

50

100

150 200 250 Longitude (76−68) Distance (Miles)

FIG. 2. NY Map

300

350

400

21

CA Elevation Contours (feet) (1000,2000,3000,4000,6000,8000,10000) 200

J J

K K K

NB O

D

latitude (37−40) Distance (Miles)

O

S MB

C C

150

B N

K C C NB

K

A M

N

X

B

M

A

KJ

R

AB AB

X S

X

A M M B E H X E O O

J

R X

NB B A

K

H

NB

50

MB MB M

K 100

AB

R N

K

MB MB A

N

J J

R H

S

D

R

R

B

0 0

50

100

150 200 250 300 Longitude (126−118) Distance (Miles)

350

400

FIG. 3. CA Map The maps were partitioned into dierent sectors, and asset classes and targets were assigned specic (X, Y) coordinates within the sectors in order to construct plausible asset allocation scenarios. The number of assets at each location was xed and the unit cost per asset was assumed known. Each target was assigned an ID number and given a prot value if destroyed. The candidate list of options that are available to destroy each target is xed as well. After asset classes and targets are identied, all possible packages that may be used to destroy each target are enumerated according to the target's candidate list of options. For example, if one of the target's options is 2F2U4V, then two planes of type F, two missiles of type U, and four missiles of type V must be selected from all possible classes that contain F, U, and V, respectively. As dened, a package species how to ll an option from dierent asset class locations. The prot for each package a associated with target t is determined as

22

X

t : 0 4

c2

8c

package

;

nc costc d(c t); : 0 5

(3.1)

a

where t denotes the prot associated with the target, c denotes the class of asset, nc denotes the number of assets in the class to be used in package a, costc denotes the cost per unit of the asset class, and d(c t) denotes the distance between the asset class and the target. This prot value is used as the coecient gta of the corresponding xta variable in the asset allocation model. In the semi-real problems, the number of packages is xed and all packages are included in the integer programming formulation. Note that each package represents a variable in the formulation of the asset allocation model without suppression. 2. Random Problems for Model 1 For random problems, an (X, Y) coordinate space is partitioned into dierent sectors, but each asset class and target is now randomly assigned to a sector. Asset classes are restricted to be assigned to only a subset of all sectors. A similar restriction is imposed on targets. Each asset class and target is then assigned exact location coordinates, which are generated randomly within its particular sector. Again, the number of assets at each location is xed and the unit cost per asset is assumed known. Each target is assigned an ID number as before. However, the prot value in destroying a target using any option is now generated from a uniform distribution. The low end of the range of prot values is a xed percentage below the minimum cost over all options, while the high end of the range of prot values is some xed percentage above the maximum cost over all options. Next, the candidate list of options that are available to destroy each target is

23 randomly selected. An option j may be used to destroy target t if the ratio of the cost of option j to prot in destroying target t falls into some acceptable range. For simplicity, an upper limit of three candidate options per target is imposed. If there are more than three acceptable options for target t, three options are selected randomly from among the set of acceptable options. If there are no acceptable options for a target, the acceptance range is enlarged so that each target will have at least one candidate option. After asset classes and targets have been identied, all possible packages that may be used to destroy each target are enumerated according to the target's candidate list of options. As in the semi-real case, the prot for each package a associated with target t is calculated and is used as the coecient of the corresponding xta variable in the asset allocation model. For the random problems, the size of the integer programming formulation is restricted by limiting the number of variables to be included. A percentage is specied to decide how many packages among the set of all eligible packages to include for each target. For example, if the percentage is 100%, then every eligible package will be included for the target if the percentage is 33%, only one third of the packages will be selected. Again, note that each package that is chosen represents a variable in the formulation of the asset allocation model. B. Development of Asset Allocation Problems with Suppression In this section, we generalize the ideas presented in Section A to illustrate the semi-real and random problems for the asset allocation problems with suppression of defenders. For completeness, some description in Section A are repeated here.

24 1. Semi-Real Problems for Model 2 For the semi-real problems, the same topographical maps (Figure 1, 2, and 3) were applied with the original locations of targets partitioned into locations of targets and threats. All locations of three types of data in the semi-real problems are assigned in accordance with the topographical nature. In addition, the capacity of each asset class is given and all eligible packages for targets and threats are included in every instance. Figure 1 represents a problem of 10 asset classes, 10 targets and 10 threats, Figure 2 represents a problem of 20 asset classes, 20 targets and 20 threats, and Figure 3 represents a problem of 30 asset classes, 30 targets and 30 threats. After asset classes, targets and threats were identied, all possible packages that might be used to destroy each target and threat were enumerated according to the target's and threat's candidate list of options. If the package is used for some target, it is referred as an attack package if the package is used for some threat, it is referred as a suppression package. The prot for each attack package a associated with target t can be evaluated by the formula shown in Section 1. The values of wta and wta are input parameters. This prot value is used as the coecient of the corresponding xta variable in the asset allocation with suppression model. Similarly, the associated prot for each suppression package s 2 Sd can be expressed by previous expression except the subscript a is changed to subscript s. This prot value is used as the coecient of the corresponding yds variable in the asset allocation with suppression model. In the semi-real problems, the number of packages is xed and all packages are included in the mixed integer programming formulation.

25 2. Random Problems for Model 2 For the random problem cases, an (X, Y) coordinate space is partitioned into dierent sectors, but each asset class, target and threat is now randomly assigned to a sector. Asset classes are restricted to be assigned to only a subset of all sectors. A similar restriction is imposed on targets and threats. Each asset class, target and threat is then assigned exact location coordinates, which are generated randomly within its particular sector. Again, the number of assets at each location is xed and the unit cost per asset is assumed known. Each target and threat is assigned an ID number. The prot value in destroying a target or a threat using any one of the suitable options is generated from a uniform distribution. The low end of the range of prot values is a xed percentage below the minimum cost over all options, while the high end of the range of prot values is some xed percentage above the maximum cost over all options. Next, the candidate list of options that are available to destroy each target and threat is randomly selected. An option j may be used to destroy target (or threat) i if the ratio of the cost of option j to prot in destroying target i falls into some acceptable range. For simplicity, an upper limit of three candidate options per target is imposed. If there are more than three acceptable options for target i, three options are selected randomly from among the set of acceptable options. If there are no acceptable options for a target, the acceptance range is enlarged so that each target will have at least one candidate option. After asset classes and targets have been identied, all possible attack packages that may be used to destroy each target are enumerated according to the target's candidate list of options. As in the semi-real case, the prot for each attack package a 2 At is calculated and is used as the coecient of the corresponding xta variable in

26 the model. Similarly, all possible suppression packages that may be used to destroy each threat are enumerated according to the threat's candidate list of options. The prot for each suppression package s 2 Sd is calculated and is used as the coecient of the corresponding yds variable in the model. For the random problems, the size of the mixed integer programming formulation is restricted by limiting the number of variables to be included. A percentage is specied to decide how many attack packages among the set of all eligible attack packages to include for each target. If the percentage is 66%, only two-thirds of the attack packages will be selected. Similarly, we can choose suppression packages for the threats by specifying the percentage. C. Sizes of Asset Allocation Problems for Both Models Having in mind the structure of the asset allocation problems for Models 1 and 2, we now show the corresponding size of each model. For Model 1, there are jAj binary variables and jC j + jT j constraints. Model 2 involves more variables and constraints than Model 1, and the size of Model 2 can be computed as follows. In general, if there are M levels of attack on targets and N levels of suppression of threats, there exist jAj binary variables for attack packages, jS j binary variables for suppression packages, jT j binary variables for each level of attack, jDj binary variables for each level of suppression, and at most jAj jDj continuous variables for the marginal gain of each level of suppression of corresponding threats. The reason why jAj jDj is an upper bound for the number of continuous variable is that not every threat can defend a given target. If a threat does not defend a given target, the suppression of that threat will not contribute to the target value. The n represents the additional gain of the attack package a 2 At continuous variable ztad

27 associated with the suppression of threat d at level n. If threat d does not defend n is zero. target t, then for every attack package a at level n for t, the value of ztad Hence, we do not include those continuous variables in the problem instance. Thus there will be at most

jAj + jS j + M jT j + N jDj (jAj + 1) variables in the model. There are jC j asset class constraints for class capacity in Constraints (2.6), jT j target assignment constraints for targets in Constraints (2.7), and jDj threat assignment constraints for threats in Constraints (2.8). There are also (M ; 1) jT j precedence constraints for all levels of attack on targets in Constraints (2.9), (N ; 1) jDj precedence constraints for all levels of suppression of threats in Constraints (2.10), and at most 2jAj jDj upper bound constraints for the marginal gain of attack packages at all attack levels in Constraints (2.22) and (2.23). Again, the reason that 2jAj jDj is n 's are known to be zean upper bound instead of an exact number is because some ztad ros before attempts are made to solve problems. Together, the number of constraints can possibly reach

jC j + jT j + jDj + (M ; 1) jT j + (N ; 1) jDj + 2N jAj jDj: Without loss of generality, we limit our discussion to the case when N is 2 and M is 1. In this case, there are at most jAj + jS j+2jDj + jT j+2jAj jDj variables and at most jC j + jT j+2jDj +4jAj jDj constraints. The number of threats (defenders, jDj), is usually small. Thus the number of attack packages, jAj, is the most important factor aecting the size of the problem. Table II compares the sizes of the corresponding semi-real problems without and with defenders. The numbers of constraints and variables for the asset allocation

28 problems without suppression are exact numbers, while the numbers of constraints and variables for the asset allocation problems with suppression are upper bound values. The sizes of the asset allocation problems with suppression are much larger than those of the asset allocation problems without suppression. The number of variables increases by an order of magnitude. The most signicant increase in problem size is due to the constraints. For example, in comparison of the 10 class 20 target and the 10 class 10 target 10 threat asset allocation problems without and with suppression, the number of variables increased from 678 to 6,888 whereas the number of constraints increased from 30 to 12,400. TABLE II. Comparison of Semi-Real Problem Sizes # of classes # of targets # of threats # of constraints # of variables Asset Allocation Problems without Suppression of Defenders 10 20 0 30 678 20 40 0 60 1468 30 60 0 90 4347 Asset Allocation Problems with Suppression of Defenders 10 10 10 12400 6888

29 CHAPTER IV COMPUTATIONAL RESULTS VIA CPLEX PARALLEL MIP SOLVER In this chapter, the computational results for the test problems via CPLEX Parallel MIP Solver are demonstrated. The computational platform used to solve the test problems consisted of the CPLEX Parallel Solver Version 6.6 (1999) on a Silicon Graphics Power Challenge with four 194 MHz R10000 processors and 1 giga byte of shared memory. The operating system was IRIX 64 Release 6.5. A. Results for Model 1 In Model 1, for each of the semi-real and random runs, three asset allocation problems were generated respectively, with each problem based on one of three different engagement scenarios. Each problem can be distinguished by the number of asset classes and targets. Problem 1 consisted of 10 asset classes and 20 targets, Problem 2 consisted of 20 asset classes and 40 targets, and Problem 3 consisted of 30 asset classes and 60 targets. Problem 1 has 30 constraints. One third of the constraints are knapsack constraints corresponding to the resources (asset classes) while two-thirds of them are mutually disjoint multiple-choice constraints corresponding to the targets. Similarly, Problem 2 has 60 constraints (20 knapsack constraints and 40 mutually disjoint multiple-choice constraints), and Problem 3 has 90 constraints (30 knapsack constraints and 60 mutually disjoint multiple-choice constraints). In the semi-real runs, one instance was generated for each of the three problems. Note that a problem is distinguished by the number of asset classes and by the number of targets. The capacity of each asset class (the right-hand-side value of each knapsack constraint) and the number of packages (the number of variables included) are both xed at the \instance" level. The number of packages and the capacity of each class

30 of asset were xed. Semi-Real Problems 1, 2, and 3, correspond to the Washington State map (Figure 1), the New York Metropolitan Area map (Figure 2), and the Northern California map (Figure 3), respectively. Variations in the random runs were made as follows. To test the impact of increasing the number of packages (variables in the integer programming formulation) problem instances with 33%, 66%, and 100% of the packages available were generated for each problem. In addition, the vector b, representing the number of assets available in each asset class, was varied to assess the impact of asset availability. In particular, all bi were set equal to a single parameter , where was chosen to be 5, 10 or 15. While the bi would not usually be equal in a realistic problem instance, the decision to x b was intended to consider the cases of limited, moderately limited, and nearly unlimited asset availability through the choice of the single parameter. By allowing the package inclusion percentage and to vary over three dierent values within each of three problems, 27 problem instances for the random runs were generated. In each instance for semi-real and random runs, the relative dierence between the optimal LP relaxation value of the original problem and the best IP value found was calculated to indicate how closely the LP relaxation approximates the IP. The dierence is termed \lpipgap". Specically, lpipgap was calculated as best IP solution found 100%: lpipgap = LP relaxationLP; relaxation

(4.1)

We also calculated the relative dierence between the best node remaining in the branch-and-bound tree and the best IP value found to indicate the solution quality. Note in a maximization problem, the best node is an upper bound for the MIP objective. In particular, the gap was calculated by best node ; best IP solution found 100%: best node

(4.2)

31 This gap is 0 if the optimal solution is found. Since the value of the LP relaxation is an upper bound of the value of the best node in the remaining branch-and-bound tree, the gap calculated by Equation 4.2 will be bounded above by the corresponding lpipgap calculated by Equation 4.1. While CPLEX has a stand-alone MIP solver with default settings, it also provides a signicant level of user control over parameter settings that direct the MIP solution process. With the default settings, decisions on whether or not to add cutting planes are made by CPLEX, and mipgap (the relative tolerance on the gap between the best integer objective and the best node remaining in the branch-and-bound tree) is 0.01%. For practical consideration, we relaxed mipgap to 1%. That is, the branchand-bound process will terminate whenever the gap between the best node and the best IP solution found is less than or equal to 1%. In each integer programming run, the size of the branch-and-bound tree was limited to 1 giga byte of memory. To be practical, we also limited the run time for each of the four processors to one hour, so all runs either reached a 1% gap solution or the one hour run time limit. Summary statistics for the three semi-real problem instances of Model 1 are shown in Table III. The column labeled No. of Vars.g shows the number of variables, and the column labeled No. of Cons.g shows the number of constraints (including both knapsack constraints and multiple-choice constraints) in the problem. The column labeled Simplex Iter.g represents the total number of simplex iterations needed to solve the problem, while the column labeled No. of Nodes represents the total number of nodes explored in the branch-and-bound tree. The column labeled IP Obj.g gives the best integer solution found within the one hour time limit. The columns labeled lpipgap shows the value calculated by Equation 4.1, indicating how close the original LP relaxation optimal solution is to the best IP solution found. The columns labeled mipgap displays the result by Equation 4.2, indicating the quality of the corresponding

32

TABLE III. Performance of Three Semi-Real Problems in Model 1 via CPLEX 6.6 Parallel Solver Problem No. No. Simplex No. IP lpipgap mipgap Run of of of Time ID Vars. Cons. Iter. Nodes Obj. (%) (%) (sec.) WA 678 30 248 0 199780 0.00 0.00 0.4 NY 1468 60 364 1 484151 0.07 0.05 1.4 CA 4347 90 582 1 1015870 0.12 0.12 4.8 IP solution. The run times (elapsed times in seconds) using four processors are shown in the column labeled Run Time. As we can see all these three problems were solved within ve seconds by CPLEX Parallel MIP Solver. Summary statistics for the nine random problem instances of Model 1 are shown in Tables IV to VI. In each instance of Random Problem 1 in Table IV, the original LP relaxation solution was very close (less than 2%) to the best solution found. Similar results were obtained for the nine instances of Random Problem 2 and for the nine instances of Random Problem 3 in Tables V and VI, respectively. The best solution found is within 1% of the best node for each instance in Random Problem 1 (Table IV). For Random Problem 2 and Random Problem 3 (Tables V and VI), the instances were much harder than those in Random Problem 1, especially when = 5, and there were four instances that required more than one hour to converge to 1% of gap. However, the gaps of these four instances were all within 2%. (Notice the lpipgap values for these four instances were within 2%, so it guaranteed the quality of the best IP solution ever found.) Moreover, we found that solutions with good quality (5% of gap) were already available in the rst minute of the solution process

33

TABLE IV. Performance of Nine Instances of Random Problem 1 in Model 1 via CPLEX 6.6 Parallel Solver RHS No. of Simplex No. of IP lpipgap mipgap Run Time ( ) Vars. Iter. Nodes Obj. (%) (%) (sec.) 5 12467 1253 221470 1.96 0.90 22.9 10 3994 129 1 384476 0.99 0.92 6.4 15 210 1 447130 0.13 0.12 5.8 5 7640 750 221851 1.39 0.28 11.4 10 2645 978 89 383066 1.06 0.99 4.8 15 201 0 446650 0.00 0.00 2.6 5 1398 100 221421 0.86 0.17 2.7 10 1328 2653 195 380417 1.05 0.90 3.1 15 179 1 443334 0.02 0.01 1.3 number of constraints = 30 for each instance

34

TABLE V. Performance of Nine Instances of Random Problem 2 in Model 1 via CPLEX 6.6 Parallel Solver RHS No. Simplex No. IP lpipgap mipgap Run 1-min. of of Time mipgap ( ) Vars. Iter. Nodes Obj. (%) (%) (sec.) (%) 5 2755404 147497 328356 1.90 1.73 3600.2 2.95 10 8321 26302 883 623104 1.03 0.99 84.0 2.00 15 551 1 742640 0.06 0.05 16.2 0.05 5 4441622 242203 328406 1.81 1.61 3600.1 1.97 10 5511 247112 8182 625571 0.61 0.54 291.1 0.86 15 492 1 741746 0.07 0.06 8.5 0.06 5 8426082 464712 327253 1.99 1.65 3600.0 2.14 10 2765 284743 7805 623722 0.70 0.61 173.1 2.41 15 433 1 738595 0.21 0.21 3.8 0.21 number of constraints = 60 for each instance run time limit exceeded

35

TABLE VI. Performance of Nine Instances of Random Problem 3 in Model 1 via CPLEX 6.6 Parallel Solver RHS

No. Simplex No. IP lpipgap mipgap Run 1-min. of of Time mipgap ( ) Vars. Iter. Nodes Obj. (%) (%) (sec.) (%) 5 1350509 48117 571340 1.54 1.12 3600.3 1.92 10 15365 1016 15 1098985 0.27 0.23 52.2 0.23 15 1279 17 1374988 0.12 0.10 45.4 0.10 5 1431242 53484 571521 1.46 0.97 2508.7 4.52 10 10174 4276 159 1099138 0.20 0.15 34.7 0.15 15 929 1 1369247 0.36 0.35 20.8 0.35 5 1420299 50676 570207 1.52 0.93 1321.4 1.95 10 5102 27514 725 1087375 0.89 0.83 47.0 0.83 15 1270 36 1369088 0.09 0.08 10.0 0.08 number of constraints = 90 for each instance run time limit exceeded

36 for these more dicult instances. The columns labeled 1-min. mipgap in Tables V and VI indicate the solution quality evaluated in the rst minute during the run period. Good solutions, therefore, were not too hard to obtain in the rst minute of the run even if it might take more than one hour or much longer to get the optimal solution. Another observation is that the problems were relatively easy to solve when was relatively large (when = 15 as shown in the corresponding tables). This result can be explained in part by recognizing that as becomes large, the constraints Ax b cease to be restrictive. The only constraints remaining are the mutually disjoint SOS constraints. In this situation, in each SOS constraint the variable with the largest coecient in the objective function will be set equal to 1 and all others to 0. Table VII provides more detailed results for the collection of largest instances (100% of all eligible packages) of Random Problem 3. We extended the previous experiment by varying over the values f1 2 15g, while xing the package percentages at 100%. As can be seen from the test results, the gap between the optimal value of the original LP relaxation and the integer program is uniformly small except when = 1. However, this instance is trivial to solve when (asset class capacity) is so small since the majority of the variables (packages) can be eliminated from further consideration. The reduced problem becomes much smaller and easier to solve (It took four processors less than 1 second to reach a closing gap of 0.77%). Although the best solution found was not within 1% of the best node for the instances of = 3, 5, and 6, the corresponding gaps were all less than 4%. Moreover, the best solution in the rst minute for each value was less than 10%. The stateof-the-art CPLEX software enables us to get \real time" solutions very close to the optimal solution for this problem class.

37

TABLE VII. Performance of Random Problem 3 in Model 1 with 100% Variables at Dierent RHS Values via CPLEX 6.6 Parallel Solver RHS Simplex No. of IP lpipgap mipgap Run Time 1-min. ( ) Iter. Nodes Obj. (%) (%) (sec.) mipgap (%) 1 148 1 95535 20.76 0.77 0.6 0.77 2 3274 391 235154 2.25 0.98 32.9 0.98 3 1503852 75192 339933 4.65 3.31 3600.3 7.05 4 299509 15854 463886 1.25 0.90 1067.5 2.26 5 1350509 48117 571340 1.54 1.12 3600.3 1.92 6 1077204 34381 676311 1.71 1.56 3601.0 2.48 7 359954 12412 784595 1.19 0.97 1464.3 1.98 8 55790 1943 891112 0.88 0.78 295.8 2.20 9 330837 13155 991262 1.10 0.92 1239.3 2.58 10 1016 15 1098985 0.27 0.23 52.2 0.23 11 1911 31 1185117 0.90 0.85 61.8 0.85 12 71489 1358 1270873 0.70 0.64 318.7 2.21 13 20954 488 1329047 0.30 0.23 134.4 1.16 14 1401 12 1360195 0.01 0.00 49.5 0.00 15 1279 17 1374988 0.12 0.10 45.4 0.10 number of constraints = 90 for each instance number of variables = 15365 for each instance run time limit exceeded

38 B. Results for Model 2 In Model 2, we generated a random problem with 10 asset classes, 10 targets and 10 threats. We also considered the number of packages and suppression packages with 33%, 66%, and 100% of the packages available and set to 5, 10, and 15. Altogether nine random problem instances were generated. Summary statistics for the random problem instances of Model 2 are shown in Table VIII. The sizes of Model 2 problem instances are much larger than the problems from Model 1 due to the number of precedence constraints in Model 2. For example, there were 21,592 constraints for the 100% instance of the random problem in Model 2, while there were only 90 constraints for the largest problem instance in Model 1. However, we were still able to reach the 1% gap within one hour for all nine problem instances. In fact, most of the instances reach the 10% gap in the rst 10 minutes of run time as shown in the last column of Table VIII. The only exception was the rst instance with a gap of 12% in 10 minutes. Table IX provides more detailed results for the collection of largest instances (100% of all eligible packages) of the random problem. As can be seen from the test results, the gap between the optimal value of the original LP relaxation and the integer program was quite small for every instance except when 3. Even though the best solution found was not within 1% of the best node for the instances of = 3 and 4, both of the instances converged to a gap of less than 2%. Overall, the best solution in the rst 10 minutes for each value was less than 10% except when = 5, 9 and 12. Although the size of this problem class is very large, we were still able to obtain very good solutions in a very short period in general.

39

TABLE VIII. Performance of Nine Instances of the Random Problem in Model 2 via CPLEX 6.6 Parallel Solver RHS No. No. No. IP lpipgap mipgap Run 10-min. of of of Time mipgap ( ) Cons. Vars. Nodes Obj. (%) (%) (sec.) (%) 5 3826 172321 5.23 0.99 1336.8 12.06 10 21592 13851 1151 338974 1.98 0.54 847.1 3.53 15 840 484380 0.83 0.61 313.5 0.61 5 2803 171091 5.21 0.45 655.1 6.69 10 14304 9180 1164 333409 2.55 0.97 322.9 0.97 15 224 479243 1.00 0.81 68.0 0.81 5 2414 169479 5.59 0.96 259.3 0.96 10 7192 4620 1458 334668 2.30 0.97 143.7 0.97 15 2337 478625 0.91 0.68 98.1 0.68

40

TABLE IX. Performance of the Random Problem in Model 2 with 100% Variables at Dierent RHS Values via CPLEX 6.6 Parallel Solver RHS Simplex No. of IP lpipgap mipgap Run Time 10-min. ( ) Iter. Nodes Obj. (%) (%) (sec.) mipgap (%) 1 384 2 29466 25.49 0.00 0.4 0.00 2 19192 550 65226 14.64 0.42 255.8 0.93 3 438896 5283 99315 11.50 1.42 3601.8 5.47 4 385144 9713 134706 8.75 1.48 3602.1 8.83 5 144057 3826 172321 5.23 0.99 1336.8 12.06 6 364479 12783 207731 3.54 0.72 2678.3 4.60 7 124807 5785 241097 2.97 0.99 1941.9 6.86 8 101814 2622 275358 2.10 0.30 1262.5 0.76 9 121809 7859 306449 2.35 0.62 1889.3 12.91 10 39485 1151 338974 1.98 0.54 847.1 3.53 11 16592 37 372047 1.23 0.00 380.8 0.00 12 20284 561 399442 1.81 0.75 633.3 NA 13 59059 3799 426842 2.22 0.99 975.8 1.63 14 35392 4024 456758 1.63 0.86 1205.5 0.81 15 19519 840 484380 0.83 0.61 313.5 0.61 number of constraints = 21592 for each instance number of variables = 13851 for each instance run time limit exceeded

41 CHAPTER V HEURISTIC APPROACHES Chapter IV demonstrates the computational tractability of the problems using CPLEX 6.6. However, the problems were much harder to solve when we used CPLEX 4.0 in 1997. This motivated an investigation of heuristic methods to solve this problem class. The formulation in Model 1 is a multi-dimensional multiple-choice knapsack problem (referred to as MMKP). This formulation is also a special case of the 0-1 multi-dimensional knapsack problem (referred to as 0-1 MKP) by treating each multiple-choice constraint as a knapsack constraint. Before discussing the corresponding heuristics, a review of the general formulation of this problem class is presented rst. The notation used in this chapter is independent from the ones used in Chapter II. A. Problem Formulations and Complexity Throughout this chapter, symbols M , N , and G represent the sets of f1 2 mg, f1 2 ng, and f1 2 gg, respectively. A general 0-1 MKP can be formulated as follows: max s.t.

X

j 2N

X

cj xj aij xj bi i 2 M

j 2N xj 2 f0 1g

j 2 N:

It is assumed that all aij are non-negative, all cj are non-negative and all bi are positive. In addition, it is assumed that aij bi for all i 2 M and for all j 2 N , since any variable violating this rule will automatically be assigned a zero value in

42 any feasible solution. This type of formulation is very often encountered in the area of resource allocation where each of m resources (constraints) has its own capacity (right-hand-side value), each of n projects (variables) consumes some combination of resources (a column of the constraint coecient matrix), and the objective is to maximize the prot by choosing some of the n projects within the capacity. The 0-1 MKP is NP-complete even for the simplest case of m = 1 (Garey and Johnson 1979 Magazine and Chern 1984). The simplest case is commonly referred to as the 0-1 knapsack problem. A general formulation of the MMKP is as follows: max s.t.

X

j 2N

X

j 2N


X

where

j 2Nk

xj = 1 k 2 G

N = N N Ng 1

2

Np \ Nq = if p 6= q p q 2 G xj 2 f0 1g j 2 N: The multiple-choice constraints partition all variables into g disjoint GUB sets (with an upper bound of 1). Exactly one variable will be chosen in each GUB set. In the test problems of this dissertation, at most one variable can be chosen in each set. To transform the underlying application to the MMKP, simply add a slack variable to each GUB constraint. The multiple-choice knapsack problem is a special case of the MMKP when there is only one knapsack constraint (m = 1) in addition to the multiple-choice constraints. Since the 0-1 knapsack problem is a special case of the multiple-choice

43 knapsack problem and the former is NP-complete, the latter is NP-hard. It can be concluded that the MMKP is NP-hard as well. The relationship between the 0-1 MKP and the MMKP can be described as follows. The MMKP is a special case of the 0-1 MKP with every multiple-choice constraint being treated as a knapsack constraint. B. Heuristics for the 0-1 Multi-Dimensional Knapsack Problem There are several heuristics in the literature designed for the 0-1 MKP. In general, solution approaches to the 0-1 MKP either aggregate all constraints or decompose the multi-dimensional knapsack problem into several single-knapsack constraint problems for which there exist ecient heuristics. In particular, techniques like surrogate constraints (see Glover 1968 Greenberg and Pierskalla 1970), Lagrange relaxation (see Georion 1974 Fisher 1981), and the composite of surrogate constraints and Lagrange relaxation (Greenberg and Pierskalla 1970), have been widely used to solve the 0-1 MKP. A comprehensive survey of approximation algorithms and heuristics for the 0-1 MKP can be found in Lin (1998) and Chu and Beasley (1998). C. Heuristics for the Multi-Dimensional Multiple-Choice Knapsack Problem On the other hand, some literature does discuss multiple-choice programming, but most references focus on multiple-choice problems with only one knapsack constraint. The concept of simple (integer ) dominance is widely used by the multiplechoice programming literature. For example in a maximization problem, if two variables xi and xj are in the same GUB constraint the objective coecient of xi is less than xj while the knapsack coecient of xi is not less than xj , then xi is dominated by xj (i.e., xi = 0 no matter what is the value of xj in the optimal solution). To

44 utilize the concept of simple dominance, for each GUB constraint, rst sort its variables into an array in non-decreasing order of the objective function coecients, then use another array to sort the variables in the non-increasing order of the knapsack constraint coecients. If a variable x appears before y in both sorted arrays, the former is dominated by the latter. An exception occurs when two variables in the same GUB constraint have the same objective function coecients as well as knapsack constraint coecients. The tie can be broken in either way. If such an exception case does not occur, then the dominated variable will be 0 in an optimal solution. This technique is helpful to delete variables prior to a branch-and-bound enumeration or other heuristics. A comprehensive survey of approximation algorithms and heuristics for the multiple-choice knapsack problem can also be found in Lin (1998). The dominance issue in the MMKP is also discussed in Dyer and Walker (1998). The paper demonstrates that the expected proportion of undominated variables in the problem class is a function of m, the number of knapsack constraints, and jNk j, the cardinality of the kth GUB set. Explicit closed-form solutions are given for m equal to 1 and 2, and an approximation is developed for xed m and large jNk j. The paper shows that the expected proportion of undominated variables decreases with increasing jNk j for xed m and the expected proportion of undominated variables increases with increasing m for xed jNk j. The reduction technique is very useful when m is very small and jNk j is very big. However, the reduction eect is not signicant for the test problems in this dissertation due to the relatively large values of m in the test problems (10, 20, or 30). For example, the paper shows the expected proportion of undominated variables when m = 10 and jNk j = 200 is 0:942 (94.2%).

45 D. Lagrange Multiplier Approaches The limitation of the simple dominance issue on multiple knapsack constraints led us to examine the other heuristic applied to the MMKP by Moser et al. (1997). They utilize a Lagrange multiplier theorem, addressed in Everett (1963), on this problem class. The theorem was applied earlier to solve the 0-1 MKP by Magazine and Oguz (1984). A discussion of the Lagrange multiplier theorem by Everett and the heuristics used by the two papers (Magazine and Oguz 1984 Moser et al. 1997) follows.

Theorem 1 Given m non-negative Lagrange multipliers ( m), let X = (x xn), xj 2 f0 1g for j 2 N , be a solution of the Lagrange relaxation of 1

1

the MKP:

max

X

j 2N

cj xj ;

X

i2M

i

X

j 2N

aij xj

(5.1)

then X is also a solution of X

max s.t.

j 2N X

cj xj aij xj

X

aij xj i 2 M

j 2N j 2N xj 2 f0 1g j 2 N:

It is very clear how to solve the Lagrange relaxation of the MKP in Equation (5.1) once the values of all i are known by looking at the relative prot (relative cost if it has a minimization form) of each variable: simply let 8 > >

> :0

P

otherwise:

i2M i

P

j 2N aij ) > 0

(5.2)

46 However, it is not guaranteed that the solution X obtained from solving Equation (5.1) will be feasible to the original problem (MKP). The solution is feasible to P the MKP if the terms (bi ; j2N aij xj ) for all i 2 M are non-negative. The diculty here is how to eciently compute and update the Lagrange multipliers i to drive the solution of Equation (5.1) more closer to the optimal solution of the MKP. Given the theorem, the heuristic proposed by Magazine and Oguz (referred to as Heuristic MO) for the MKP is outlined as follows.

Step 1. Initialize. Set i = 0 for all i 2 M and xj = 1 for all j 2 N according to the rule specied in Equation (5.2).

Step 2. Normalize each coecient by dividing it by the corresponding right-handside value and let yi be the sum of all coecients in the ith constraint, that is,

aij = aij =bi 8i 2 M and

yi =

X

j 2N

aij 8i 2 M:

Step 3. Let i = arg maxi2M yi. This step locates the most violated constraint for 0

the current solution.

Step 4. Compute, for each j with xj currently set to 1, 8 > >
:

1

j = >

P

i2M i aij )=ai j 0

if ai j > 0 0

otherwise:

47 Let

j = min fj jxj = 1g: j 0

Notice j is the maximum increase in i without causing more than one variable, currently set to 1, to have a non-positive relative prot. 0

0

Step 5. Let i = i + j . Set xj = 0 and yi = yi ; aij , for all i 2 M . In 0

0

0

0

0

this step, the variable xj (which is 1) has a non-positive relative prot so it is changed to 0. The chosen multiplier (i ) and the feasibility measure (yi) for each constraint are modied accordingly. 0

0

Step 6. If yi > 1 for any i 2 M , go to Step 3, otherwise go to Step 7. Note if all feasibility measure yi

1, then the solution is already feasible.

Step 7. Check whether any zero-variables (variables currently set to 0) can be set back to 1 without violating any constraint. Choose the variable from the qualied zero-variables with the largest prot (objective coecient). Repeat this step until there are no more qualied zero-variables then stop. The solution X to the Lagrange relaxation in Equation (5.1) is optimal to the MKP when it satises X

i2M

i (bi ;

X

j 2N

aij xj ) = 0:

If the solution obtained from the heuristic is feasible but not necessarily optimal, an upper bound on the value of the optimal solution of the MKP is given by X

j 2Q

cj +

X

i2M

i(1 ; yi)

48 where Q represents the set of variables currently assigned a value of 1 by the heuristic and i are the corresponding Lagrange multipliers, according to Theorem 4.1 of Magazine and Oguz (1984). Volgenant and Zoon (1990) further show how to compute more than one Lagrange multiplier simultaneously to sharpen the upper bounds by evaluating the second scarce resource in addition to evaluating the most scarce resource. Everett's theorem is also applicable to the MMKP. However, the Lagrange multiplier heuristics designed for the MKP (e.g., Magazine and Oguz 1984 Volgenant and Zoon 1990) cannot be applied to the MMKP directly due to the presence of the multiple-choice constraints. Moser et al. (1997) propose a Lagrange multiplier heuristic (referred to MJS) to solve the MMKP. The MJS heuristic for the MMKP is summarized as follows.

Step 1. Initialize and normalize the coecients. (a) Let i = 0 for all i 2 M . (b) Choose the most valuable variable in each GUB set, and let jk be the index of the chosen variable for the kth GUB set. (c) Normalize each coecient by dividing it by the corresponding right-handside value, that is, aij = aij =bi for all i 2 M . P (d) Compute feasibility measure: yi = k2G aijk for all i 2 M .

Step 2. Relax the constraint violation. (a) Find i such that yi = maxi2M yi. This step locates the most violated constraint of the current solution. (b) For all zero-variables xj , compute the increase j of the Lagrange multiplier i relative to the one of the variables currently set to 1 in the corresponding 0

0

0

49 GUB set:

j =

cjk ; cj ;

P

i2M i (aijk ; aij ) ai jk ; ai j 0

0

0

j 2 N:

Find the GUB set k containing the smallest j and let j be the index of the variable with the smallest j . (c) Update the i th Lagrange multiplier and re-evaluate the feasibility measure as i = i + j , and yi = yi + aij ; aijk for all i 2 M . Change xj to 1 while changing xjk to 0. (d) If yi 1 for all i 2 M or no more variables can be swapped, go to Step 3, otherwise repeat Step 2. 0

0

0

0

0

0

0

0

0

0

Step 3. Improve solution and repeat until no more variables can be swapped. (a) For each GUB set, evaluate all swaps which are able to maintain the feasibility. Calculate #j , the dierence of objective function value by changing xj to 1 and changing corresponding xjk to 0. (b) Choose the largest #j and make the corresponding swap. The size of problem tested in Moser et al. (1997) has at most ve knapsack constraints, nine multiple-choice constraints and 100 variables. Since they obtained very good computational results in their experiments and our rst model has a formulation quite similar to theirs, their approach was explored on the large-scale asset allocation problems without suppression of defenders. Instead of having \ " in Equation 2.3, their heuristic deals with \=" for each GUB constraints. To t their heuristic, a slack variable was added in each GUB constraint of Equation 2.3. Data of Random Problems 1, 2, and 3 at 100% for all the right-hand-side values of f1 16g was used.

50 Computational results are presented for Random Problems 1, 2 and 3 in Tables X, XI and XII, respectively. In these three tables, the second column LP Relaxation shows the LP relaxation value of the corresponding right-hand-side value. The third column Best Solution represents the best solution found by the MJS heuristic. The fourth column Gap can be calculated by Equation 4.1 if a solution is available. The fth column Heuristic Iterations demonstrates how many heuristic iterations were used for the rst phase as well as the second phase if it is applicable. The last column Time displays time spent for the heuristic during the rst and second phases. For example, when the right-hand-side value is 11 for Random Problem 1, the MJS heuristic found 312 343 as the best solution. The gap between LP relaxation and the best solution found is 13.66%. It used 122 iterations in nding a feasible solution, and 10 iterations on improving the solution. Together it took 1.08 seconds in the rst phase and 5.60 seconds in the second phase. The MJS heuristic failed to generate a feasible solution in Step 2 for 10 out of 16, 13 out of 16, and 15 out of 16 instances for Random Problems 1, 2 and 3, respectively. Our results show that the MJS heuristic could handle small easy problems, but failed to be an eective heuristic when the problem is larger or tighter. As we have seen in the Chapter IV, each instance of Random Problem 1 can be solved very quickly no matter what the value is by CPLEX Parallel MIP Solver. The MJS heuristic starts from choosing the most valuable variable in each GUB set. The initial solution is infeasible (otherwise the initial solution is already optimal). The procedures in Step 2 do not guarantee that the feasibility will be recovered before exhausting all possible pairs of swaps within each GUB set. Moser (1998, pers. comm.) points out that once a variable has been dropped it is gone for good. This strategy certainly avoids cycling problems. However, this strategy also limits the ability for the heuristic to nd better solution or even a feasible solution. The paper by Moser

51 et al. (1997) doesn't clarify whether j in Step 2(b) is allowed to have a negative value. In the implementation of our test problems, the heuristic often runs out of options if j is not allowed to be negative. Allowing negative j in Step 2(b) will possibly make some Lagrange multipliers negative which is not meaningful. Therefore other eorts were made to improve the MJS heuristic. For example, the CPLEX MIP solver was used to get a feasible solution rst and then Step 3 of the MJS heuristic was used to improve the solution quality. Better feasible solutions for some of the problems were obtained but the improvements were not signicant. In fact, Moser revealed that he was interested in a solution for a rather small (little more than 10 constraints, 10 groups, and at most a couple of hundred variables) scheduling problem related to multimedia (Moser, 1998, pers. comm.). It is not surprising to see that the heuristic fails to be eective in large scale problem instances. Due to the disappointing results, the MJS heuristic was not applied to the Model 2 problems.

52

TABLE X. Results of Random Problem 1 in Model 1 with 100% Variables at Dierent RHS Values by Heuristic MJS RHS LP Best Gap Heuristic Time ( ) Relaxation Solution (%) Iterations (Seconds) 1 55812.38 NA NA 3671 15.09 2 109221.37 NA NA 3671 15.08 3 150191.29 NA NA 3085 14.19 4 189544.17 NA NA 3671 15.14 5 225903.65 NA NA 3527 14.93 6 261043.21 NA NA 3085 14.47 7 294863.49 NA NA 2876 13.64 8 328562.25 NA NA 3671 15.11 9 359791.58 NA NA 2769 13.29 10 388337.37 NA NA 3523 14.91 11 413173.47 312343 13.66 122+10 1.08+5.60 12 435965.88 327895 12.96 174+20 1.40+12.15 13 445087.25 329452 5.72 114+18 1.03+10.04 14 446911.50 329456 2.09 113+18 1.01+9.51 15 447711.00 301305 0.96 125+22 1.09+10.96 16 448276.50 329457 0.34 112+18 1.02+8.80 number of constraints = 30 number of variables = 3994 NA: the heuristic terminated without nding any feasible solution

53

TABLE XI. Results of Random Problem 2 in Model 1 with 100% Variables at Dierent RHS Values by Heuristic MJS RHS LP Best Gap Heuristic Time ( ) Relaxation Solution (%) Iterations (Seconds) 1 67582.92 NA NA 4806 62.41 2 135165.83 NA NA 4803 60.45 3 202324.59 NA NA 6817 72.17 4 268925.18 NA NA 4801 60.07 5 334718.53 NA NA 4417 61.59 6 398068.20 NA NA 6817 75.42 7 459646.67 NA NA 7172 78.77 8 519858.01 NA NA 4800 59.61 9 576548.39 NA NA 5787 68.05 10 629597.76 NA NA 4414 55.52 11 677569.97 NA NA 6410 70.21 12 718882.44 NA NA 6815 70.32 13 733420.00 NA NA 5668 64.88 14 739349.00 551040 4.25 3076+40 42.81+96.82 15 743075.25 621187 2.60 3599+30 48.31+82.77 16 746227.50 616012 1.99 1568+34 24.77+84.87 number of constraints = 60 number of variables = 8321 NA: the heuristic terminated without nding any feasible solution

54

TABLE XII. Results of Random Problem 3 in Model 1 with 100% Variables at Different RHS Values by Heuristic MJS RHS LP Best Gap Heuristic Time ( ) Relaxation Solution (%) Iterations (Seconds) 1 120570.60 NA NA 8169 283.95 2 240561.87 NA NA 8167 283.19 3 356502.02 NA NA 9824 309.16 4 469736.01 NA NA 8165 282.53 5 580259.31 NA NA 10535 330.64 6 688044.34 NA NA 9822 325.95 7 794017.03 NA NA 6939 242.49 8 899004.93 NA NA 8164 294.93 9 1002312.92 NA NA 8422 291.22 10 1101947.57 NA NA 10534 333.85 11 1195847.71 NA NA 8078 275.47 12 1279840.41 NA NA 9820 323.93 13 1333040.40 NA NA 8657 282.45 14 1360378.71 NA NA 6938 241.31 15 1376703.81 NA NA 7877 266.67 16 1386097.58 1173186 3.29 3274+7 144.42+72.80 number of constraints = 90 number of variables = 15365 NA: the heuristic terminated without nding any feasible solution

55 CHAPTER VI TABU SEARCH HEURISTICS A. An Overview of Meta-Heuristics Traditional ascent or descent heuristics often get trapped into local optimal solutions. A local optimal solution can be much worse than a global optimal solution. To conquer the problem of being conned by local optimality, especially when dealing with hard or large-scale combinatorial-optimization problems, meta-heuristics, a term coined by Glover (1986), have been widely used. A meta-heuristic can be described as a master strategy that guides and modies subordinate heuristics to explore the solution space beyond local optimality. Meta-heuristics, having had widespread successes in attacking a variety of dicult combinatorial optimization problems, are one of the most interesting developments in approximate optimization since their inception in the early 1980s. These families of approaches include, but are not limited to, simulated annealing, genetic algorithms, tabu search, scatter search, neural networks, greedy randomized adaptive search procedures (also known as GRASP), and ant colony systems. For an overview of meta-heuristics, readers are referred to Laporte and Osman (1995), Osman and Kelly (1996), and Aarts and Lenstra (1997). A number of meta-heuristics have been proposed to solve the 0-1 multi-dimensional knapsack problem. A comprehensive survey can be found in Chu and Beasley (1998). (In addition to a survey of meta-heuristics on 0-1 MKP, their paper also reviews literature using exact algorithms and early heuristics, i.e., local search approaches.)

56 B. An Overview of Tabu Search Tabu search has been successfully applied to a variety of dicult integer programming problems. In this chapter, a tabu search approach based on the strategic oscillation, surrogate information, and perturbation-guided search for the multi-dimensional multiple-choice knapsack problem is proposed. The method of this study is related to the work of Glover and Kochenberger (1996) on the 0-1 multi-dimensional knapsack problem. Both methods are based on strategic oscillation around the feasibility boundary and surrogate information is used to evaluate moves. The major dierences of the two methods include the consideration of GUB constraints and eorts made to explore trial solutions at critical events. An overview of tabu search follows. The term tabu search was rst introduced by Fred Glover (1986, 1989) with roots going back to the late 1960's and early 1970's (see Glover et al. 1993). Important ideas of the method are also developed by Pierre Hansen (1986) in a context of steepest ascent/mildest descent formulation. As Glover points out, tabu search is based on the premise that problem solving, in order to qualify as intelligent, must incorporate adaptive memory and responsive exploration (Glover 1995). A comprehensive guide to tabu search can be found from the Tabu Search book by Glover and Laguna (1998). The exible memory structure in tabu search can be divided into short term memory and long term memory. The most popularly used short term memory is recency-based memory, which keeps a record of the solution attributes that have been altered in the most recent history. Recency-based memory assigns tabu status to selected attributes as the search proceeds and prevents certain solutions from the recent past from being revisited. The tabu status of a solution can be disregarded if certain conditions are met, which are expressed in terms of aspiration rules. In some applications, the short term memory is sucient to produce high quality

57 solutions. However, tabu search becomes signicantly stronger in general by incorporating long term memory. Two types of frequency-based memory are essential to longer term consideration: residence frequencies keep track of relative duration attributes occurs in the solutions obtained while transition frequencies keep track of how often that attributes change. Intensication and diversication strategies are two important components of long term memory. The former strategy is designed to search deeper on the neighborhood of historical good solutions while the latter strategy is generated to nd new regions to explore. The use of adaptive memory in tabu search contrasts with the rigid design of branch and bound, and the memoryless styles of several other approaches such as the original form of simulated annealing as reported in Kirkpatrick et al. (1983). C. Review of Tabu Search Approaches for the 0-1 Multi-Dimensional Knapsack Problem There are various approaches to solve the 0-1 MKP using tabu search. An overview appears below. Dammeyer and Voss (1993) develop a tabu search heuristic for solving the 0-1 MKP, based on the reverse elimination method (REM), a dynamic strategy proposed by Glover (1990). Without relying on a tabu tenure at all, the REM updates a tabu list by considering logical relationships in the sequence of attribute changes. Aboudi and J$ornsten (1994) and Lokketangen et al. (1994) both superimpose a tabu search framework on the pivot and complement heuristic introduced by Balas and Martin (1980). By rst taking advantage of the well known result that optimal zero-one solutions can be found at an extreme point of the linear programming feasible region, the pivot and complement heuristic performs a series of pivots to recover the

58 binary restriction. The reactive tabu search (RTS) as developed by Battiti and Tecchiolli (1992, 1994) learns to adjust the tabu list size in an automatic way that reacts to the occurrence of cycles. Hashing and digital tree techniques are used to detect duplication of solutions eciently. This RTS has been applied to a considerable range of problems including the 0-1 MKP. Lokketangen and Glover (1996, 1998) bring in a tabu search method that utilizes the extreme point property of zero-one solutions. They also look at probabilistic measures in the framework and apply the learning tool target analysis to identify better decision rules and improved control structures. Finally, Glover and Kochenberger (1996) introduce a critical-event tabu search approach whose underlying memory structure is arranged around the feasibility boundary for the 0-1 MKP. The heuristic uses a strategic oscillation that navigates both sides of the boundary to achieve a balance between intensication and diversication. A parameter span is used to indicate the depth of the oscillation about the boundary, measured in the number of variables added after crossing the boundary in a constructive phase (here, variables are set to 1) and in the number of variables dropped after crossing back the boundary in a destructive phase (here, variables are set to 0). Starting from a minimum value, the span is gradually increased to a maximum value. A series of constructive phases and destructive phases is performed for each value of the span parameter. When span reaches the maximum value, then it is gradually decreased back to the minimum value. Once the span reaches the minimum value, then it is gradually increased to the maximum value once again, and this oscillation process continues. Similar to this critical event tabu search approach, Hana and Freville (1998) also present an ecient heuristic that balances the interaction between intensication and diversication strategies for the 0-1 MKP.

59 D. Method in Detail Based on the idea of the critical event tabu search for the 0-1 MKP by Glover and Kochenberger (1996), our method for the MMKP focuses on the exible memory structure updated at critical events. As dened by Glover and Kochenberger (1996), a critical event corresponds to \the construction of a complete feasible solution by the search process at the feasibility boundary". In other words, critical events are referred to solutions generated by the constructive phase at the nal moment prior to becoming infeasible and solutions generated by the destructive phase at the rst moment after becoming feasible. The major choice rules in this dissertation use surrogate constraint information to evaluate what variables to add, drop or swap. The feasibility of knapsack constraints can be relaxed while the feasibility of GUB constraints is always satised in the method. The balance of intensication and diversication is attained through a strategic oscillation scheme that explores varied depths on each side of the boundary in a consistent rhythm. Trial solutions are generated at critical events by a local search improvement, strengthened by a specialized tight oscillation (Glover, 2001, pers. comm.). The tabu search approaches of this dissertation represent a class of strategic oscillation methods that continue by alternating the constructive and destructive phases. A constructive phase in this dissertation corresponds to one that progressively becomes less feasible or more violated by setting variables equal to 1 or by swapping variables in some GUB sets, while a destructive phase corresponds to one that progressively becomes less violated or more feasible by setting variables equal to 0. The parameter span dened by Glover and Kochenberger (1996) is also used in the context of our heuristic. Within this framework, the tabu search heuristic is outlined as follows.

60

Initialization: Choose an initial solution x satisfying all GUB constraints. Constructive Phase: (Move from a feasible solution to an infeasible solution.) 1. While the current solution is feasible: if not all GUB constraints are tight, select a variable to add otherwise select a pair of variables in the same GUB set to swap. 2. Reset span s to 0. 3. While s su (su is a dened upper bound for s): (a) Select a variable to add as long as the proposed new solution doesn't violate any GUB constraints. (b) Increase the span counter by 1, i.e., s = s + 1. 4. Go to the Destructive Phase.

Destructive Phase: (Move from an infeasible solution to a feasible solution.) 1. While the current solution is infeasible, select a variable to drop. 2. Reset the span counter s to 0. 3. While s su and it is allowed to drop a variable: (a) Select a variable to drop. (b) Increase the span counter by 1, i.e., s = s + 1. 4. Go to the Constructive Phase. The constructive phase can be partitioned into two sub-phases including a feasibleconstructive phase and an infeasible-constructive phase. The heuristic progressively chooses variables to add during the constructive phase. An exception occurs when the current solution is feasible and all GUB constraints are tight. At this condition,

61 it is impossible to add a variable and still satisfy all the GUB constraints. Thus swap moves are used to proceed to the infeasible region. (In practice, this type of swap is not common when the resource is tight.) The destructive phase can also be partitioned into two sub-phases including an infeasible-destructive phase and a feasible-destructive phase. The heuristic progressively chooses variables to drop during the destructive phase. Upon nishing a constructive phase or a destructive phase, certain conditions will be checked to determine whether to end the preceding procedures. (A keyword is given inside the square bracket for each condition. The key words will be used in the next chapter when displaying computational results.)

1. NUM MOVE] The number of the overall moves (add, drop or swap) exceeds the overall limit.

2. NUM ITER] The number of iterations exceeds the iteration limit. An iteration corresponds to a pass of both a constructive phase and a destructive phase.

3. CYCLE] If the heuristic revisits a solution, this move generates a duplication. If a series of moves duplicate the solution history, then this series of moves very likely form a cycle of duplication. When the length of a cycle exceeds the given limit, the heuristic will be terminated.

4. DUP RATE] Solution duplication rate exceeds a given percentage. The duplication frequency

62 is computed by (number of solutions possibly revisited) 100%: (number of solutions generated)

(6.1)

The duplication of solutions can be detected eectively and eciently by using hash functions. Hash functions have been used within tabu search to avoid cycling. (See Woodru and Zemel 1993 Battiti and Tecchiolli 1994 Carlton and Barnes 1996.)

5. TIME] The run time limit has been reached.

6. GAP] The solution quality reaches a gap of desired precision (in comparison to the LP relaxation solution) or better.

7. TRAPPED] The heuristic has not made any progress (in terms of solution quality) for a number of iterations.

8. NO CHOICE] The heuristic runs out of choices of adding or swapping due to tabu memory limitations in the constructive phase, or runs out of choices of dropping due to tabu memory limitations in the destructive phase. This indicates the tabu rule is probably too strict for this run. E. Choice Rules Both Lagrangian relaxation and surrogate relaxation have been shown to be eective tools for solving a variety of integer programming problems. As shown in

63 Chapter V, the form of a general 0-1 MKP is:

z = max s.t.

X

j 2N

X


j 2N xj 2 f0 1g

j 2 N:

The most commonly used Lagrangian relaxation for the 0-1 MKP relaxes all of the knapsack constraints and has the following form. (For background on Lagrangian relaxation, see Georion 1974 and Fisher 1981.) X

(cj ;

zLR () = max

X

iaij )xj +

j 2N i2M xj 2 f0 1g j

s.t.

2 N

X

i2M

ibi

(6.2)

where is an m-dimensional non-negative vector and the Lagrangian dual for this relaxation is given by

zLD = min z (): LR 0

On the other hand, the surrogate relaxation for the 0-1 MKP forms a nonnegative linear combination of all the knapsack constraints as follows.

zSR( ) = max s.t.

X

j 2N

X

cj xj i

X

(6.3)

aij xj

i2M j 2N xj 2 f0 1g

X

i2M

i bi

(6.4)

j 2 N

where is an m-dimensional non-negative vector and the corresponding surrogate

64 dual is then given by

zSD = min z ( ): SR 0

Although Lagrangian relaxation is more widely known, the surrogate dual proposed by Glover (1965) achieves a better bound, which results directly from the fact that the former (when all knapsack constraints are relaxed as in our case) is a relaxation of the latter for any given choices of multipliers. That is,

zSD zLD for a maximization problem (see Greenberg and Pierskalla 1970). A good resource of additional properties of the surrogate dual can be found in Glover (1975). A most recent tutorial on surrogate constraint approaches can be found in Glover (2001) targeted for optimization in graphs. Both relaxations have been studied in this dissertation as choice rules. The rules are given in the next two subsections. 1. Lagrangian Relaxation Based Choice Rules As shown earlier in this section, we can form a Lagrangian relaxation of the 0-1 MKP by relaxing all knapsack constraints. GUB constraints and binary constraints are preserved and can be managed separately. The multiplier values are determined in three cases, adopted from the setting for surrogate constraint multipliers in Glover and Kochenberger (1996), depending on the feasibility status of the current solution and the phase of the search. In all cases,

bi = bi ;

X

0

fj jxj 2 1-setg

aij

65 is computed for each i 2 M . Thus bi indicates the remaining right-hand-side value after the current assignment for the i constraint. (The value is negative if the constraint is violated.) In the rst case, when the solution is feasible, the i for each constraint is always set to (bi ); . In the second case, when the solution is infeasible and the search is in the constructive phase, then for all i 2 M , 0

th

1

0

8 > >
0 0

i = > > :2 + jbi j if bi 0

0

0:

In the last case, when the solution is infeasible and the search is in the destructive phase, then for all i 2 M , 8 > >
P > :(jbi j + fj jxj 2 0

if bi 0 0

0-set

g aij

);1

if bi < 0:

(6.5)

0

The setting rules above focus on the inuence of the most tight or violated constraints and, therefore, encourage the search around critical solutions (i.e., solutions generated at critical events). When bi < 0 in Equation 6.5, i can be re-expressed as 0

X

j 2N

aij ; bi:

Setting variables to 0 is equivalent to setting their compliments to 1. To normalize each complimented constraint, just divide the complimented constraint by the complimented right-hand-side value as calculated in the re-expression. To choose a new assignment, the proposed change on the relaxed objective function value, by adding, dropping or swapping, is evaluated by the immediately preceding solution and multipliers. Once a move is chosen, the solution and multipliers are updated to form the basis for the next evaluation of eligible moves.

66 In the implementation of the Lagrangian relaxation approach, the objective function coecients are rst normalized by dividing through by the minimum. (Therefore, the minimum of the normalized objective coecient is 1.) Without loss of generality, assume cj in Equation 6.2 represents the normalized objective coecient for the variable xj . If the variable xj is added from the 0-set, then this move will increase the relaxed objective function value by

cj ;

X

i2M

i aij :

Similarly, if the variable xj is dropped from the 1-set, then this move will decrease the relaxed objective function value by the same amount. The constructive phase choice rule for this approach selects the variable xj to change from 0 to 1 in order to (

cj ;

maximize

X

i2M

!

)

iaij j xj = 0 :

(6.6)

The choice rule for the destructive phase selects the variable xj to change from 1 to 0 in order to (

minimize

cj ;

X

i2M

!

)

iaij j xj = 1 :

(6.7)

When adding is not permissible at the feasible-constructive phase, swap moves are performed by selecting the variable xj to change from 0 to 1 while selecting the variable xk to change from 1 to 0 in order to (

maximize

cj ; ck ;

X

i2M

!

i(aij ; aik ) j xj = 0 xk = 1

)

where xj and xk are in the same GUB set.

(6.8)

67 2. Surrogate Constraint Based Choice Rules To generate a surrogate constraint, a set of multipliers are rst computed by the rules specied in the last subsection, then each knapsack constraint is multiplied by its corresponding multiplier and the weighted sum becomes surrogate constraint. Let X

sj xj s

8j 2N

0

represent the surrogate constraint, where X

sj =

i2M

iaij

and

s = 0

X

i2M

ibi

as expressed in Equation 6.4. The ratio of the objective coecient and the surrogate constraint coecient reects its ratio of benet and cost to some degree. The choice rule for the constructive phase of this approach selects the variable xj to change from 0 to 1 in order to

maximize scj j xj = 0 : j

(6.9)

The choice rule for the destructive phase of this approach selects the variable xj to change from 1 to 0 in order to

minimize scj j xj = 1 : j

(6.10)

When it is not allowed to add a variable at the feasible-constructive phase, swap moves are chosen to move the solution from the feasibility boundary to the infeasible region by selecting the variable xj to change from 0 to 1 while selecting the variable

68

xk to change from 1 to 0 in order to

=sj j x = 0 x = 1 maximize ccj =s j k k k

(6.11)

where xj and xk are in the same GUB set. Preliminary results reported in Chapter VII indicate that the surrogate constraint approach is more promising than the Lagrangian relaxation approach as choice rules in our tabu search heuristic for this problem class. For this reason more eorts were devoted to studying how to strengthen the surrogate constraint approach as addressed in the next section. F. Normalization for Surrogate Constraint A normalization scheme developed by Glover (1998, pers. comm.) was implemented in order to strengthen the surrogate constraint. The normalization strategy diers based on representations (\ " and \ ") of 0-1 knapsack constraints. Coming from the \ " side, where variables are being set to 1 and all the knapsack constraints are satised, rst divide by the updated right-hand-side value to create a rst-level normalization, then multiply by the sum of the normalized lefthand-side coecients (for currently unassigned variables) to create the second-level normalization. Here the updated right-hand-side value reects the current solution by subtracting the current assignment from the original right-hand-side value. This approach reects how much the left-hand-side coecient potentially \contributes" to the right-hand-side value. (The right-hand-side value is 1 for every knapsack constraint after the rst level normalization. Given an equal right-hand-side value, the larger the sum of the left-hand-side coecients of a constraint, the more restrictive the constraint tends to be.)

69 Coming from the opposite side, where variables are being set to 0 and some (or all) of the knapsack constraints are infeasible, the approach implicitly replaces the variables by their complements to put the violated constraints in \ " form, therefore setting variables to 0 is treated as setting their complements to 1. The normalization is done directly by dividing the sum of the original left-hand-side coecients. (The same results will be obtained if one divides through by the updated right-hand-side value to create a rst level normalization rst, and then divides by the sum of the normalized left-hand-side coecients. Here the updated right-hand-side value is the same as the original right-hand-side value because complemented variables are used in the constraint. Dividing rather than multiplying is more appropriate in the \ " case because the smaller the sum of the left-hand-side coecients of a constraint, the more restrictive the constraint tends to be.) Lastly, second level normalization constraints are added to form a surrogate constraint. (If a knapsack constraint is not violated, the right-hand-side value will be non-positive for the complemented constraint, so it is redundant to include this constraint in forming a surrogate constraint.) Incidentally, the multiplier of the \ " case and the denominator of the \ " case (i.e., the sum of the left-hand-side coecients) can be raised to some power between 1 and 2. Glover (1999, pers. comm.) conjectures that = 1 and = 2 would be good values to use, and Zhang et al. (2000) nds = 1:2 and = 1:8 are good in an application of 0-1 MKP (without GUB constraints). The value of was set to 2:0 in the preliminary runs. The procedure to normalize each constraint at each sub-phase is described as follows:

Normalization at the feasible-constructive phase: The goal in this sub-phase is to set variables to 1 when the current solution is feasible. Simply apply the

70 approach for the \ " case.

Normalization at the infeasible-constructive phase: The goal in this sub-phase is to set variables to 1 when the current solution is infeasible. The variables that were forced to be 0 are not necessarily compelled to be 0 any more. We can think of enlarging the boundary to become feasible. The normalization is similar to what is done at the feasible-constructive phase except the righthand-side values of the \ " knapsack constraints are increased to let the current assignment become \feasible".

Normalization at the infeasible-destructive phase: The goal in this sub-phase is to set variables to 0 when the current solution is infeasible. Simply apply the approach for the \ " case.

Normalization at the feasible-destructive phase: The goal in this sub-phase is to set variables to 0 when the current solution is already feasible. Instead of enlarging the right-hand-side values, we can think of shrinking them so that at least one constraint will be \violated", then perform the normalization as designed for the infeasible-destructive phase. G. Memory Management and Oscillation Scheme 1. Penalty Adjusted Ratio The memory management method used is adapted from the scheme proposed by Glover and Kochenberger (1996). To inuence the search by recency information, the last t solutions obtained at critical events are recorded in a circular list. Denote the last i critical solution as x(last i), then tabuR, a recency tabu vector, can be calculated by summing up the last t critical solutions. In practice, each time a new th

71 critical solution is identied, the recency tabu vector can be updated by adding the last critical solution and subtracting the last t critical solution, that is, th

tabuR = tabuR + x(last 1) ; x(last t):

(6.12)

On the other hand, the long term frequency information is captured by summing up all the critical solutions collected so far. The resulting vector is referred to by tabuF . As mentioned earlier, the search adds variables from the 0-set to cross the feasibility boundary in the feasible-constructive phase, then the search adds more variables according to the current span in the infeasible-constructive phase. Next the search turns around to drop variables in the infeasible-destructive phase. The search recovers feasibility at some point in the dropping process and drops more according to the current span in the feasible-destructive phase. Eventually the search turns around to the following constructive phase and moves from the feasible region to the infeasible region. At these switch points, the tabu memory of the recency information and frequency information (tabuR and tabuF ) can aect the direction of the search. In general, if tabuRi > 0, then the i variable is currently on the tabu list, therefore, this variable is not allowed to be changed. To be more exible in utilizing tabu information, a penalty term is introduced when evaluating the ratio (the objective coecient/the surrogate constraint coecient) for each variable. Denote the ratio associated with the i variable as ri, and rmax as the maximum of the ratios of all relevant variables. (A variable is relevant if it is in the 0-set in the constructive phase or it is in the 1-set in the destructive phase.) Then, th

th

rmax tabuRi recency penalty scalar a penalty term associated with the recency information, can be subtracted from the original ri . Here the recency penalty scalar, indicating the penalty degree, is chosen in

72 the range of 0 to 1. For the results reported in the next chapter, the recency penalty scalar was set to 1. In addition, another penalty term associated with the frequency information can be subtracted from the original ri to penalize the frequently appeared variables in critical solutions. This value is calculated as

rmax tabuFi (the index of the current iteration); (frequency penalty scalar); 1

1

where the frequency penalty scalar was set to 100,000 by the choice specied in Glover and Kochenberger (1996). These penalty terms are applied in the rst k adds or drops immediately after a \turn around" to foster more diversity in the search process. The parameter k starts from 1, after 2t iteration (here t corresponds to the tabu list size of critical solutions), k is increased by 1. Continue in this fashion until k reaches kmax , the maximum value of k. Then k is decreased by 1 every 2t iterations until it equals to 1 again and the process repeats. Following the setting in Glover and Kochenberger (1996), kmax was set to 4. Using these penalty terms to substitute for the tabu status in tabuR will be referred to as using penalty rule. If the penalty rule is not used, the evaluation of moves will just be based upon the tabu status in tabuR. 2. GUB-size Adjusted Tenures To avoid reversing recent moves following a turn around, a move-based tenure is also implemented in addition to the critical event memory. When only the GUB constraints are considered, a variable in a smaller GUB set generally has a higher chance to be added or dropped. In contrast, a variable in a larger GUB set, generally has a lower chance to be added or dropped. Denote the number of variables in the i GUB set as gi and the maximum of all gi's as gmax, then the tenure of a variable th

73 in the i GUB set can be determined as follows: th

maximize the closest integer of ( g gi maximum tenure value) 1 max where the maximum tenure value is a given parameter. 3. Controlling Span The oscillation about the feasibility boundary is controlled by the span parameter (su as shown in Section D in this Chapter). In Glover and Kochenberger (1996), this parameter is xed at some value for a certain number of iterations and changed systematically as follows:

Increasing Span For span from 1 to p1: Allow p2 span iterations and then increase span by 1. For span from p1 + 1 to p2: Allow p2 iterations and then increase span by 1. When span exceeds p2, set it back to p2 and start to decrease span.

Decreasing Span For span from p2 to p1+1: Allow p2 iterations and then decrease span by 1. For span from p1 to 1: Allow p2 span iterations and then decrease span by 1. When span drops to 0, set it back to 1 and start to increase span. In our application, the number of GUB sets is not very large. A large value of span in the destructive phase is very likely to drop most of the variables currently in the 1-set. Thus, the following constructive phase will start from very similar regions. Due to this concern, the span is cut to one third of its originally designed size (rounded to the closest integer) in the destructive phase to increase the diversity of the search. H. Generating Trial Solutions Recall that critical events correspond to solutions generated by the constructive phase at the nal moment prior to becoming infeasible and by the destructive

74 phase at the rst moment after regaining feasibility. Critical solutions are those obtained immediately before and immediately after crossing the feasibility boundary. Note that the \feasibility-to-infeasibility" process at the constructive phase and the \infeasibility-to-feasibility" process at the destructive phase may not nd the best available solutions due to the choice rules utilized. A local search that ignores the critical-solution based tabu information of each variable can be launched from the critical solutions to look for alternatives that may be better. Glover and Kochenberger (1996) discuss a method to generate trial solutions. The steps are described as follows. As variables are chosen to be set to 1 in the constructive phase, an add move nally makes the solution infeasible. Instead of adding this variable, other variables in the 0-set are examined in order of decreasing objective function coecients to see whether setting a variable from the 0-set to 1 can satisfy all the knapsack constraints. This procedure generates the rst trial solution. The second trial solution can be generated from the critical solution obtained by the constructive phase after becoming infeasible by choosing a variable from the 1-set to drop in order to recover the feasibility. Variables from the 1-set are examined in order of increasing objective function coecients. Similarly, two other trial solutions can be generated at the critical event of the destructive phase as follows. As variables are chosen to be set to 0 in the destructive phase, a drop move eventually makes the solution feasible. Instead of dropping this variable, other variables in the 1-set are examined in order of increasing objective function coecients to nd whether a dierent drop variable will result in a better feasible solution. The last trial solutions can be generated from the critical solution obtained by the destructive phase after becoming feasible by choosing a variable from the 0-set to add in order to get a better feasible solution. Variables from the 0-set are examined

75 in order of decreasing objective function coecients. In the context of our application where swap can happen in some cases, an add move is considered at the critical solution obtained immediately prior to becoming infeasible and at the critical solution obtained immediately after recovering feasibility, while a drop move is considered at the critical solution obtained immediately after becoming infeasible and at the critical solution immediately before recovering feasibility. The trial solution choice obtained in this way will replace the best solution in the heuristic if it improves the best solution currently known, and the heuristic can continue either from the trial solution or from the critical solution obtained immediately after crossing the boundary. In the latter strategy, the best solution is updated but the search trajectory will not be changed by the trial solution process. In practice, the former strategy seems to perform slightly better than the latter strategy in general when applying this trial solution approach. However, the latter strategy preserves the search trajectory and therefore provides a more convenient basis when comparing dierent kinds of trial solution approaches. The trial solution procedures described above will be referred to \basic trial solution approach". In the next section, a more sophisticated trial solution approach is discussed. I. Tight-Oscillation Trial Solutions The basic trial solution approach described in Section H only uses the objective function information. The previous approach runs very fast since no extra calculation is needed, however, it does not include any reference to the feasibility change. Consequently, the selection by this choice rule may not be able to nd solutions with

76 very high quality. To achieve further improvements, a \specialized tight-oscillation" approach, suggested by Glover (2001, pers. comm.) that incorporates the change in both the objective function and feasibility in its choice rule was implemented. This approach looks at possibilities to add variables until the solution becomes infeasible and then looks at possibilities to drop variables until the solution recovers feasibility, then it repeats the previous step. If a resulting solution after crossing the feasibility boundary from either side has been visited before during the tight-oscillation process, then this method stops (and returns to the main heuristic described in Section D). A hashing tree can be built to store all the resulting solutions after crossing the boundary. To save memory usage, all other intermediate solutions in the tight-oscillation process are not stored in memory. (If a solution has been visited before in the intermediate steps of the tight-oscillation process, it will eventually lead to a resulting solution that has been visited before.) Two variations were implemented for deciding which variables to add (or to swap). The rst variation is referred to as a \fast" strategy which keeps adding (or swapping) variables until the solution becomes infeasible. At this moment, the step used in the basic trial solution approach is applied to see whether a substitute move can retain feasibility and improve the solution. If a better solution is identied in this step, the best solution will be updated. The tight oscillation process then continues from the infeasible solution obtained earlier. The second variation is referred to as a \complete" strategy that adds (or swaps) variables only when the resulting solution remains feasible. If none of the candidates for adding (or swapping) qualies, then the search eventually proceeds to the infeasible region. The second strategy requires more eort than the rst strategy but it might be able to reach better solutions.

77 Adapted from a perturbation-guided oscillation proposed by Glover (2000, pers. comm.) the choice rule for this tight oscillation includes the reference to both the objective function change and the feasibility change. Under this rule, when considering the addition of xj , the value of this move is calculated as

cj ; k c (infeasibility worsening by adding xj )

(6.13)

where cj is the objective coecient of xj , c is the average value of all objective coecients, and k is a chosen constant to adjust the weight put on \infeasibility worsening". The term \infeasibility worsening" refers to the amount by which a move causes a constraint to become more violated, summed over all the constraints. Similarly, when considering dropping xi , the value of this move is calculated as

;ci + k c (infeasibility improvement by dropping xi ):

(6.14)

The term \infeasibility improvement" refers to the amount by which a move causes a constraint to become less violated, but ignoring the amount by which a move causes a constraint to become more feasible, summed over all the constraints. In addition, when considering swapping xj (0 now) and xi (1 now) in the same GUB set, the value of this move is calculated as

cj ; ci + k c (infeasibility improvement by dropping xi) ; k c (infeasibility worsening by adding xj ): (6.15) As mentioned earlier, when all GUB constraints are tight but the solution is still feasible, swap moves are required to help the process cross the feasibility boundary. The following steps are used for implementing this choice rule.

Step 1: Dene bi as the right-hand-side value for constraint i after assigning the 0

78 current values to the variables. Let

ui = minf0 big: 0

Thus ui is negative and equals to bi only if the current solution is infeasible to constraint i ui is 0 otherwise. 0

Step 2: Consider the feasibility change on each constraint for a particular move (add, drop or swap). For example, if xk (currently 1) is set to 0 while xj (currently 0) is set to 1 (xk and xj are in the same GUB set), then let

vi = minf0 bi + aik ; aij g: 0

So vi is negative and equals to bi + aik ; aij only if the proposed solution is infeasible to constraint i vi is 0 otherwise. 0

Step 3: Let zi be the dierence of the infeasibility for constraint i, that is, zi = vi ; ui: If zi < 0, then the infeasibility of constraint i is worsened by the amount jzij, while if zi > 0, then the infeasibility of constraint i is improved by the amount zi.

Step 4: Sum over the infeasibility change for each knapsack constraint. Let netchange =

X

i2M

zi :

If netchange is negative, then the overall infeasibility is worsened, while if netchange is positive, then the overall infeasibility is improved.

79

Step 5: Finally calculate the weighted sum of infeasibility change and objective function change by multiplying infeasibility change by a multiple of the average of all objective coecients. Notice when evaluating which variables to drop (to recover feasibility), attention should be restricted to moves with positive infeasibility change (infeasibility improving). When evaluating what variables to add, the infeasibility change is always negative or 0, and this term penalizes the gain contributed by the dierence of the objective function values. When summing over an infeasibility change from all constraints, it is assumed that all constraints have been normalized by their righthand-side values. Other normalization schemes are also possible, such as the one used in the strong surrogate approach (Section F).

80 CHAPTER VII COMPUTATIONAL RESULTS FROM TABU SEARCH HEURISTIC APPROACHES This chapter demonstrates the computational results for a new set of test problems via the tabu search heuristic on a Silicon Graphics Power Challenge with six 194 MHz R10000 processor and 1 giga byte of memory (only one processor was used when solving the problems using the tabu search heuristic). The operating system was IRIX 64 Release 6.5. The results in Chapter IV show that the problem instances for the right-handside value of 5 are generally harder than those with right-hand-side values of 10 and 15. Therefore, the same three asset allocation problems based on the three dierent engagement scenarios at right-hand-side values of 5 were used to form a new test set. Recall from Chapter IV, each problem can be distinguished by the number of knapsack constraints and the number of GUB sets. (Problem 1 consisted of ten knapsack constraints and twenty GUB sets, Problem 2 consisted of twenty knapsack constraints and forty GUB sets, and Problem 3 consisted of thirty knapsack constraints and sixty GUB sets.) New problem instances with 10% to 100% in increments of 10% of the original available variables were varied for each problem. The right-hand-side value of each knapsack constraint was xed at 5 (a very tight level) for all the thirty instances. For convenience, the ten instances of Problem 1 are referred to by the set

fP1a, P1b, , P1jg where a stands for 10%, b stands for 20%, etc. Similarly, the ten instances of Problems

81 2 and 3 are referred to by the sets

fP2a, P2b, , P2jg and fP3a, P3b, , P3jg respectively. Numbers of variables of these thirty instances are summarized in Table XIII. TABLE XIII. Size of Thirty Problem Instances Problem 1 (10 Knapsack Constraints and 20 GUB Sets) Ins. P1a P1b P1c P1d P1e P1f P1g P1h P1i Per. 10% 20% 30% 40% 50% 60% 70% 80% 90% #Vars. 407 806 1207 1605 2000 2406 2804 3205 3604 Problem 2 (20 Knapsack Constraints and 40 GUB Sets) Ins. P2a P2b P2c P2d P2e P2f P2g P2h P2i Per. 10% 20% 30% 40% 50% 60% 70% 80% 90% #Vars. 848 1676 2514 3343 4168 5004 5842 6671 7508 Problem 3 (30 Knapsack Constraints and 60 GUB Sets) Ins. P3a P3b P3c P3d P3e P3f P3g P3h P3i Per. 10% 20% 30% 40% 50% 60% 70% 80% 90% #Vars. 1563 3093 4637 6168 7697 9242 10785 12317 13859 Ins.: name of the problem instance Per.: percentage of the corresponding original problem #Vars.: number of variables

P1j 100% 3994 P2j 100% 8321 P3j 100% 15365

All the runs stopped when any of the terminating conditions illustrated in Chapter VI appeared. The setting of parameters for preliminary runs are as follows.

82 1. Parameters related to terminating conditions (see Section D in Chapter VI) NUM ITER]: h iterations encountered where h equals to 10 number of variables 1

1

NUM MOVE]: h moves encountered where h equals to 100 number of variables 2

2

CYCLE]: detecting 100 consecutive duplicated solutions DUP RATE]: reaching 50% of all solutions in the solution history TIME]: elapsed run time more than 20 minutes GAP]: 1% TRAPPED]: no progress for h iterations where h equals to 3 times number of variables 3

3

2. Parameters related to tabu memory management and oscillation scheme (see Section G in Chapter VI)

tabu rule: active penalty rule: inactive but may be used when there is no choice available using the tabu rule

maximum tabu tenure value after each move: 4 recency penalty scalar = 0.1 frequency penalty scalar = 100,000 kmax = 4, the maximum of k (checking tabu status within k moves immediately after the turn around points)

span control: p1 = 3, p2 = 7

83

tabu list size (t specied in Equation 6.12) = 4 3. Other Parameters

initial solution: every variable is 0 strong surrogate normalization power (): 2.0 A. Preliminary Results Based on Choice Rules Using the Lagrangian relaxation choice rules and the surrogate constraint choice rules in the heuristic, a series of preliminary runs were made to compare the performance of these two approaches on Problems 1, 2 and 3. The results are summarized in Tables XIV to XVI. In Table XIV the column labeled Problem Instance shows the name of the problem instance as dened earlier. The columns labeled Gap indicate the closing gap in percentage of the surrogate constraint and Lagrangian relaxation approaches. The columns labeled Dup. display the duplication rate in percentage of solutions as computed by Equation 6.1. The columns labeled Condition show the keyword of terminating condition of each run. Tables XIV to XVI show the surrogate constraint approach achieved a better solution in 28 out of all 30 instances except for Instances P1h and P1i. The average gaps of the 30 instances are 17.44% and 33.54% by surrogate constraint and Lagrangian relaxation, respectively. The average duplication rates of the 30 instances are 0.2% and 28.8% by surrogate constraint and Lagrangian relaxation, respectively. Observe that the duplication rates of the Lagrangian relaxation approach on all thirty instances, except for three of them, exceeded 10% while the duplication rates of the surrogate constraint approach were less than 2%. This helps to explain why the surrogate constraint approach achieved better solutions in most of these instances. Recall in

84

TABLE XIV. Performance Comparison between Simple Surrogate and Lagragian Relaxation Approaches of Problem 1 Problem Surrogate Constraint Lagrangian Relaxation Instance Gap(%) Dup.(%) Condition Gap(%) Dup.(%) Condition P1a 10.31 1.34 NUM MOVE 14.65 22.27 CYCLE P1b 7.56 1.47 TIME 13.31 36.86 CYCLE P1c 11.44 0.73 TIME 18.92 41.58 CYCLE P1d 10.27 1.61 TIME 11.50 18.96 CYCLE P1e 13.48 0.21 TIME 15.33 29.70 CYCLE P1f 14.06 0.08 TIME 14.87 30.73 CYCLE P1g 14.35 0.06 TIME 16.92 32.95 CYCLE P1h 15.57 0.19 TIME 11.02 38.36 CYCLE P1i 15.94 0.18 TIME 12.60 33.89 CYCLE P1j 11.98 0.11 TIME 14.20 49.51 CYCLE run-time limit was set to 20 minutes

85

TABLE XV. Performance Comparison between Simple Surrogate and Lagragian Relaxation Approaches of Problem 2 Problem Surrogate Constraint Lagrangian Relaxation Instance Gap(%) Dup.(%) Condition Gap(%) Dup.(%) Condition P2a 12.86 0.08 TIME 15.98 16.73 CYCLE P2b 15.07 0.00 TIME 29.73 23.54 CYCLE P2c 17.08 0.00 TIME 35.44 25.16 CYCLE P2d 14.41 0.00 TIME 30.68 27.89 CYCLE P2e 14.75 0.00 TIME 40.31 27.87 TIME P2f 20.40 0.00 TIME 40.40 15.69 TIME P2g 19.64 0.00 TIME 25.73 30.12 TIME P2h 15.74 0.00 TIME 45.31 52.07 TIME P2i 18.93 0.00 TIME 49.96 20.06 TIME P2j 19.09 0.00 TIME 44.78 40.71 TIME run-time limit was set to 20 minutes

86

TABLE XVI. Performance Comparison between Simple Surrogate and Lagragian Relaxation Approaches of Problem 3 Problem Surrogate Constraint Lagrangian Relaxation Instance Gap(%) Dup.(%) Condition Gap(%) Dup.(%) Condition P3a 14.56 0.00 TIME 31.84 25.30 CYCLE P3b 18.27 0.00 TIME 36.29 10.14 TIME P3c 20.22 0.00 TIME 92.24 0.00 NO CHOICE P3d 19.07 0.00 TIME 52.98 50.00 TIME P3e 22.33 0.00 TIME 46.19 2.94 TIME P3f 25.03 0.00 TIME 49.81 11.05 TIME P3g 20.02 0.00 TIME 52.84 42.42 TIME P3h 37.86 0.00 TIME 47.16 4.44 TIME P3i 26.31 0.00 TIME 46.42 46.04 TIME P3j 26.65 0.00 TIME 48.74 56.89 TIME run-time limit was set to 20 minutes

87 Chapter VI the same set of multipliers are used in both approaches. The scale of the objective coecients does not change the selection by using the surrogate constraint choice rules (Equations 6.9 to 6.11), but does have an impact on the selection by using the Lagrangian relaxation choice rules (Equations 6.6 to 6.8). Three dierent trial scalars (0.1, 10, and 100) were applied to the Lagrangian relaxation approach by multiplying the normalized objective coecients. The scalar of 0.1 generated the best results among these three experiments. The results show that using the new scalar value of 0.1 instead of 1 reduced the average gap of these 30 instances from 33.54% to 29.61% and reduced the duplication rate from 28.80% to 12.86%. However, this improved duplication rate was still too high in comparison with the results (0.2%) obtained from the surrogate constraint approach. Although it may be possible to nd other scalars that reduce the duplication rate more and achieve smaller closing gaps than the three scalars tried for the Lagrangian relaxation approach, this may require considerable eort. It is easier to use the surrogate constraint approach for this problem class. The nding motivated us to concentrate on surrogate constraints in the latest development for the tabu search heuristic. B. Preliminary Results of Strong Surrogate Constraint Approach To see whether the surrogate constraint obtained from the multipliers given by the paper by Glover and Kochenberger (1996) can be strengthened, preliminary runs were conducted to compare the \strong surrogate constraint" approach with the former one. The former surrogate constraint will be referred to as \simple surrogate constraint". Tables XVII to XIX compare the gaps obtained from these two approaches on all thirty problem instances.

88

TABLE XVII. Performance Comparison among Dienent Approaches Using Surrogate Constraints and Trial Solutions of Problem 1 (1) (2) (3) (4) (5) (6) Problem Gap (%) Gap (%) Gap (%) Improvement Improvement of of of (3) from from Simple Strong with Trial (2) to (3) (3) to (4) Instance Surrogate Surrogate Solutions (%) (%) P1a 10.31 9.41 6.21 8.73 34.01 P1b 7.56 8.48 5.49 -12.17 35.26 P1c 11.44 6.33 4.70 44.67 25.75 P1d 10.27 7.62 4.64 25.80 39.11 P1e 13.48 10.96 6.17 18.69 43.70 P1f 14.06 9.59 4.03 31.79 57.98 P1g 14.35 9.38 5.26 34.63 43.92 P1h 15.57 8.04 4.25 48.36 47.14 P1i 15.94 9.00 3.73 43.54 58.56 P1j 11.98 12.44 3.84 -3.84 69.13 Run-time limit was set to 20 minutes.

89

TABLE XVIII. Performance Comparison among Dienent Approaches Using Surrogate Constraints and Trial Solutions of Problem 2 (1) (2) (3) (4) (5) (6) Problem Gap (%) Gap (%) Gap (%) Improvement Improvement of of of (3) from from Simple Strong with Trial (2) to (3) (3) to (4) Instance Surrogate Surrogate Solutions (%) (%) P2a 12.86 8.61 9.45 33.05 -9.76 P2b 15.07 12.31 9.36 18.31 23.96 P2c 17.08 12.54 7.44 26.58 40.67 P2d 14.41 15.18 7.64 -5.34 49.67 P2e 14.75 12.69 7.45 13.97 41.29 P2f 20.40 13.89 10.42 31.91 24.98 P2g 19.64 15.34 9.59 21.89 37.48 P2h 15.74 15.86 10.80 -0.76 31.90 P2i 18.93 13.14 13.08 30.59 0.46 P2j 19.09 15.49 9.92 18.86 35.96 Run-time limit was set to 20 minutes.

90

TABLE XIX. Performance Comparison among Dienent Approaches Using Surrogate Constraints and Trial Solutions of Problem 3 (1) (2) (3) (4) (5) (6) Problem Gap (%) Gap (%) Gap (%) Improvement Improvement of of of (3) from from Simple Strong with Trial (2) to (3) (3) to (4) Instance Surrogate Surrogate Solutions (%) (%) P3a 14.56 13.10 12.23 10.03 6.64 P3b 18.27 18.46 13.44 -1.04 27.19 P3c 20.22 19.29 12.93 4.60 32.97 P3d 19.07 17.90 12.08 6.14 32.51 P3e 22.33 18.94 10.02 15.18 47.10 P3f 25.03 20.34 9.69 18.74 52.36 P3g 20.02 19.47 11.32 2.75 41.86 P3h 37.86 18.45 15.48 51.27 16.10 P3i 26.31 17.47 13.02 33.60 25.47 P3j 26.65 19.95 11.78 25.14 40.95 Run-time limit was set to 20 minutes.

91 In Table XVII, Column (2) shows the closing gap of each of these 30 instances by simple surrogate approach. Column (3) shows the closing gap of each of these 30 instances by strong surrogate approach. Column (5) displays the improvement from the simple surrogate approach to the strong surrogate approach by calculating Gap of Simple Surrogate ; Gap of Strong Surrogate 100%: Gap of Simple Surrogate (A negative value indicates that the simple surrogate constraint achieved a smaller gap.) The results show that the strong surrogate approach achieves better gaps in 25 out of the thirty instances. Based on all the thirty instances, the average gap of all these 30 instances for the simple surrogate approach was 17.44% while the average gap of all these 30 instances for the strong surrogate approach was only 13.66%. The average improvement using the strong surrogate approach versus the simple surrogate approach was 19.86%. The results validate the eectiveness of using the strong surrogate approach. C. Preliminary Results of Basic Trial Solution Approach To see the impact of the simple trial solution approach in critical events, all instances were tested when using the strong surrogate constraint. Tables XVII to XIX also compare the results obtained from the heuristic without and with the basic trial solutions when using the strong surrogate constraint to guide the choice rules. In Table XVII, Column (4) shows the closing gap of each of these 30 instances by strong surrogate approach with the basic trial solutions. Column (6) displays the improvement from the strong surrogate approach without the basic trial solutions to the one with the basic trial solutions. The resulting gaps with trial solutions included for all the thirty instances were smaller except for Problem P2a. The average gap

92 of the thirty instances dropped from 13.66% to 8.85%. Moreover, the gap of the worst case (Instance P3h) was lower than 16%. The results show that exploring trial solutions in critical events can be very helpful to nd better targets. D. Rening Experiments So far in this chapter, Section A demonstrates the advantages of using the surrogate constraint approach over the Lagrangian relaxation approach. Section B shows a good normalization can make the surrogate constraint method much stronger. Section C further veries the merits of using trial solution in critical events. For further improvements, eorts were made to increase the eciency of the computer programs and adjust the power value () dynamically between 1.0 to 2.0 in the normalization scheme in case of cycling. In the preliminary runs, () was set to 2.0. By varying the power value from 1.0 to 2.0 with an increment of 0.1, the value of 1.0 generated the best results among the eleven tests with a gap of 12.62% in average. The average gaps in twenty minutes of the 30 problem instances under dierent power values are shown in Table XX. An value of 1.0 was taken in the nal results of dierent schemes as presented after this section. The strong surrogate constraint approach was re-tested using the improved codes. In addition, the basic trial solution approach during critical events was also re-tested using = 1:0. Tables XXI to XXIII summarize the results of the strong surrogate approach ( = 1:0) without and with basic trial solutions (non-overwriting and overwriting). The average closing gaps of the thirty instances using strong surrogate approach and basic trial solution approach (non-overwriting) are 12.62% and 7.63%, respectively. Overwriting the solution in the main heuristic when the trial solution is better

93

TABLE XX. Comparison of Results by Dierent Power Values in the Strong Surrogate Approach Power Value () 1.0 1.1 1.2 1.3 1.4 1.5 Average Gap (%) 12.62 12.77 13.00 13.62 13.30 13.55 Power Value () 1.6 1.7 1.8 1.9 2.0 Average Gap (%) 13.76 14.48 13.93 14.51 14.53

-

Run-time limit was set to 20 minutes. No trial solutions

than the best solution known, is also helpful in majority of problem instances, resulting in a smaller average closing gap of 6.75%. E. Results of Tight Oscillation Experiments were conducted to implement the tight oscillation process described in Section I of Chapter VI. Evaluation from some pilot runs shows this process has the ability to detect very good solutions. The tight oscillation process keeps going until a resulting solution of crossing the feasibility boundary is a duplication of a previous solution in this process. Because this process can be very time consuming, it will not be worthwhile to launch this process too frequently. If the basic trial solution doesn't provide a very good trial solution, then running this tight oscillation process might be a waste of time. On the other hand, if a trial solution in fact is a \better solution", then it is certainly worth trying to explore this trial solution further. However, if attention is just limited to those \better solutions", then the tight oscillation process will not be utilized very often. Our goal is to invoke this process 10% to 25% of time (i.e., visits to the basic

94

TABLE XXI. Performance Comparison between Strong Surrogate and Basic Trial Solution Approaches of Problem 1 Gap (%) of Problem Strong Basic Trial Solution Instance Surrogate with Strong Surrogate Approach non-overwriting overwriting P1a 6.46 4.32 3.84 P1b 6.56 2.25 2.25 P1c 5.12 4.59 2.78 P1d 6.94 2.71 2.85 P1e 10.19 4.19 5.33 P1f 8.30 3.14 2.41 P1g 8.34 3.48 3.50 P1h 9.63 3.37 2.75 P1i 10.15 3.52 3.69 P1j 8.47 3.56 3.51 Run-time limit was set to 20 minutes. Power ( ) was set to 1.0.

95

TABLE XXII. Performance Comparison between Strong Surrogate and Basic Trial Solution Approaches of Problem 2 Gap (%) of Problem Strong Basic Trial Solution Instance Surrogate with Strong Surrogate Approach non-overwriting overwriting P2a 10.86 4.67 6.45 P2b 11.26 7.15 7.89 P2c 11.41 8.42 7.33 P2d 12.35 8.52 9.06 P2e 14.94 7.74 8.99 P2f 11.36 6.12 4.79 P2g 14.20 7.41 8.18 P2h 14.72 8.58 6.78 P2i 14.38 8.57 8.93 P2j 15.00 9.07 8.25 Run-time limit was set to 20 minutes. Power ( ) was set to 1.0.

96

TABLE XXIII. Performance Comparison between Strong Surrogate and Basic Trial Solution Approaches of Problem 3 Gap (%) of Problem Strong Basic Trial Solution Instance Surrogate with Strong Surrogate Approach non-overwriting overwriting P3a 13.62 8.16 9.99 P3b 13.11 11.27 11.82 P3c 14.69 10.65 9.74 P3d 17.28 11.16 9.83 P3e 18.51 12.54 8.69 P3f 16.21 13.47 8.01 P3g 17.91 12.07 7.09 P3h 18.34 12.57 5.74 P3i 17.95 13.52 8.99 P3j 20.23 12.00 12.95 Run-time limit was set to 20 minutes. Power ( ) was set to 1.0.

97 trial solution approach). It is necessary, therefore, to set a standard that evaluates whether a trial solution qualies to launch this process. In our application, the objective value of the optimal solution of the LP relaxation (which is also an upperbound of the objective function) approximates the objective value of the optimal solution very well (when the right-hand-side in not in the range of 1 or 2). Thus if a trial solution has an objective value very close to the upper-bound, it might be an \acceptable starting solution". The standards of being an \acceptable starting solution" were set to require the objective value of a solution to be at least r times the upper-bound of the objective value and be at least r times the objective value of the best solution known. Through try and error, an r value of 80% and an r value of 90% were found, in general, to invoke this process at the desired frequency. These standards might be still too restrictive or too loose in some cases. Due to this concern, the standards were dynamically adjusted to help the process reach the desired visiting frequency. (Decrease the values of the standard-parameter when the visiting frequency is too low and increase the values of the corresponding standard-parameter when the visiting frequency is too high.) As mentioned earlier, two variations of the constructive phase in the tight oscillation were considered. Tables XXIV to XXVI summarize the results of running the \complete" and the \fast" strategies with dierent time limits (one minute, twenty minutes, and one hour). The average gap has been signicantly reduced to 3.14% (20 minutes) by the complete strategy and 3.65% (20 minutes) by the fast strategy. Moreover, both strategies were able to obtain very good solution in the rst minute (4.41% and 5.14%, respectively). The ability to get a good solution quickly is a very nice feature of the tight oscillation process. When the process was run for one hour, the average closing gaps were further reduced to 2.89% and 3.34%, respectively. While both strategies were able to produce very good solutions, better results were obtained lp

best solution

lp

best solution

98 by the complete strategy in most of the hard problems (Problem 1 is considered easy while Problems 2 and 3 are considered hard when solving them in CPLEX.) An interesting point is that the complete approach appeared to have more moves in each phase (constructive phase or destructive phase) in the hard problems (5-6 moves) while it only had 2-3 moves in the easy problems. The fast approach generally had 2-3 moves in each phase for both hard and easy problem classes. Therefore, the oscillation by the \complete" strategy had more opportunity to reach better solutions. This helps to explain why the complete approach performed better than the fast approach on the hard problem class. Experiments were also conducted to see whether overwriting the solution in the main heuristic when the tight oscillation process nds a better solution can boost the performance. No signicant dierence was found between the "overwriting" and the "non-overwriting" versions. The constant k, which multiplies the product of the average of objective coecients and the infeasibility change, was varied over the range 0.5, 1.0, 2.0, , 5.0. No signicant change was found in these variations. However, by setting k to 0.25 the results were improved slightly (dierence of 0.2% to 0.3%). The results reported in Tables XXIV to XXVI were based on k = 0:25. The same set of problem instances were also solved by using CPLEX version 6.6 on the same platform. The average closing gap obtained from CPLEX were 2.75%, 1.93% and 1.82% when running one minute, twenty minutes and one hour respectively. Comparison between the \complete" approach and CPLEX is summarized in Table XXVII.

99

TABLE XXIV. Performance of Tight Oscillation Using the \Complete" and the \Fast" Strategies of Problem 1 Gap (%) of Problem Complete Strategy Fast Strategy Instance One Twenty One One Twenty One minute minutes hour minute minutes hour P1a 4.24 2.26 2.26 4.24 2.26 2.26 P1b 3.71 2.25 2.25 3.71 2.25 2.25 P1c 2.77 2.25 2.25 2.77 2.25 2.25 P1d 3.13 1.98 1.38 3.27 1.98 1.98 P1e 3.05 3.05 2.49 3.05 2.94 2.94 P1f 3.20 2.13 2.13 3.20 2.15 2.15 P1g 3.30 1.99 1.99 3.30 1.99 1.99 P1h 3.02 1.83 1.83 3.65 1.10 1.10 P1i 2.76 2.57 1.84 3.97 2.52 2.52 P1j 3.11 2.23 1.90 3.11 2.13 2.05

100

TABLE XXV. Performance of Tight Oscillation Using the \Complete" and the \Fast" Strategies of Problem 2 Gap (%) of Problem Complete Strategy Fast Strategy Instance One Twenty One One Twenty One minute minutes hour minute minutes hour P2a 5.75 3.48 3.48 5.00 3.48 3.48 P2b 5.47 4.11 3.58 6.34 4.29 4.29 P2c 4.75 3.43 3.43 5.30 3.40 3.40 P2d 5.91 3.44 3.44 6.33 4.27 3.99 P2e 4.62 3.22 2.92 5.09 4.11 3.78 P2f 5.68 3.02 3.02 5.52 4.27 3.75 P2g 4.56 2.31 2.31 5.60 3.67 3.61 P2h 4.63 2.84 2.69 5.70 3.92 3.65 P2i 3.56 2.89 2.85 5.51 3.90 3.72 P2j 4.58 4.06 2.81 4.57 3.47 2.44

101

TABLE XXVI. Performance of Tight Oscillation Using the \Complete" and the \Fast" Strategies of Problem 3 Gap (%) of Problem Complete Strategy Fast Strategy Instance One Twenty One One Twenty One minute minutes hour minute minutes hour P3a 5.65 4.46 4.46 6.89 6.59 5.29 P3b 4.44 4.44 4.44 7.49 4.96 4.96 P3c 5.87 4.28 4.06 6.41 5.45 5.45 P3d 6.19 3.65 2.78 7.13 5.32 4.80 P3e 6.00 3.44 3.44 6.61 5.86 4.28 P3f 5.13 3.90 3.40 5.88 4.51 4.19 P3g 5.60 3.52 3.35 6.20 4.56 3.47 P3h 4.00 4.00 3.11 6.45 4.39 3.44 P3i 3.95 3.40 3.34 5.77 3.30 3.30 P3j 3.72 3.70 3.53 6.19 4.29 3.30

TABLE XXVII. Comparison between Complete Tight Oscillation and CPLEX Gap (%) of Method One Minute Twenty Minutes One Hour CPLEX Version 6.6 2.75% 1.93% 1.82% Complete Tight Oscillation 4.41% 3.14% 2.89%

102 CHAPTER VIII CONCLUSIONS AND FUTURE RESEARCH A. Conclusions In this research, the computational tractability of solving large asset allocation problems was addressed. The problems were rst modeled without suppression involved and then were generalized to include suppression of defenders. Integer programming formulations were developed and a broad variety of problem instances (semi-real problems and random problems) were generated to test the computational diculty of the problems by using CPLEX. The results demonstrated that in most instances, except when the right-hand-side value is very small, the original LP relaxation of the asset allocation problem (w/o suppression) is a very good approximation of the optimal solution. In those instances where optimality is elusive, feasible solutions very close to optimality can be generated by using CPLEX version 6.6 within one minute. While CPLEX version 6.6 is very ecient in solving this class of problems, it was much harder using CPLEX version 4 to get to the 1% closing gap between the best bound available and the best solution found for some problem instances. (When this research was started, CPLEX version 4 was available.) In January 1998, when CPLEX version 5.0 became available to us, the results were much better than previously obtained from CPLEX version 4. It was still dicult however, to get to the 1% gap for some problem instances. This motivated an investigation of heuristic methods to solve this class of problem| multi-dimensional multiple-choice knapsack problems. In this problem class, the only available heuristic, to our knowledge, based on the Generalized Lagrange Multiplier Method, was found to be ineective in solving

103 our applications. Eorts were made to investigate a similar problem class| multidimensional knapsack problems, which has much more literature available. Metaheuristics have become very popular in solving hard combinatorial problems. Among dierent meta-heuristics, tabu search was chosen for solving the multi-dimensional multiple-choice knapsack problems because of its wide spread success on dicult problems. Critical event tabu search provides the basic structure of the heuristic search method developed in this research. Lagrange relaxation and surrogate constraint information were used to form choice rules. The advantage of using surrogate constraint information versus using Lagrange relaxation in this problem class was demonstrated in Chapter VII. A constraint normalization scheme was implemented to strengthen the surrogate constraint (see Chapters VI and VII). The surrogate constraint choice rule may not be able to reach the best solution when the search crosses the feasibility boundary. Dierent complexity levels of trial solution approaches were investigated during critical events to intensify the search. The basic trial solution approach using simple local swaps was rst studied. This boosted the performance of the heuristic. The tight oscillation process was further developed. Computational results show that the search method with the tight oscillation process is capable of guiding the search to better solutions. Moreover, this method has the ability to nd good solutions very quickly, which is a very strong feature of the tight oscillation process. Using CPLEX version 6.6, the average closing gaps (between the best solution found and the optimal solution of the LP relaxation) for the thirty instances studied were 2.75%, 1.93% and 1.82% when running for one minute, 20 minutes, and one hour, respectively. Our tabu search heuristic results (closing gaps of 4.41%, 3.14%, and 2.89% in average when running for one minute, 20 minutes, and one hour,

104 respectively) were able to get very close to the state-of-the-art commercial software. Moreover, there are still many possibilities to further enhance the tabu search method as is pointed out in the \Future Research Directions" section. B. Future Research Directions We have demonstrated the eectiveness of using both a basic-level trial solution approach and an advanced-level trial solution approach at critical events. In the tight oscillation process, swap moves can be considered in addition to add moves in the constructive phase even if not all GUB constraints are tight. It is also possible to nd a better value for k in Equations 6.13 to 6.15 in Chapter VI. Dynamically changing the value of k might also yield good improvements. When the duplication frequency of resulting solutions is higher than expected, increasing the weight on the change of infeasibility is worth researching. On the other hand, if the duplication frequency of resulting solutions is not high but the solution quality is not as good as expected, decreasing the weight on the change of infeasibility should be worthwhile. There are many other ways to search good trial solutions. For example, a perturbation-guided oscillation (Glover, 2000, pers. comm.) can be used to enlarge the feasibility boundary and then seek to restore feasibility. Through this oscillation approach, potential solutions have a greater chance of being chosen. A local swap can be launched after restoring feasibility. Another promising approach, Shifting Landscape Methods proposed by Glover (2001), can combine the strength of the tabu search heuristic and CPLEX. This method would involve identifying some sub-problems by the heuristic rst, then these would be solved by CPLEX. Then the solution obtained from CPLEX would be used to identify new sub-problems for CPLEX to solve. We did not have enough time to implement all these alternatives. But it would

105 be very interesting if these possibilities can be explored.

106 REFERENCES

Aarts, E. H. L. and Lenstra, J. K. (1997). Local Search in Combinatorial Optimization. Wiley, Chichester, United Kingdom. Aboudi, R. and J$ornsten, K. (1992). Tabu search for general zero-one integer programs using the pivot and complement heuristic. ORSA Journal on Computing, 6(1):82{93. Balas, E. (1975). Facets of knapsack polytope. Mathematical Programming, 8:146{ 164. Balas, E. and Martin, C. H. (1980). Pivot and complement|a heuristic for zero-one programming. Management Science, 26:86{96. Balas, E. and Zemel, E. (1978). Facets of knapsack polytope from minimal covers. SIAM Journal Applied Mathematics, 34:119{148. Battiti, R. and Tecchiolli, G. (1992). Parallel biased search for combinatorial optimization: Genetic algorithms and tabu. Microprocessors and Microsystems, 16:351{367. Battiti, R. and Tecchiolli, G. (1994). The reactive tabu search. ORSA Journal on Computing, 6(2):126{140. Battiti, R. and Tecchiolli, G. (1995). Local search with memory: Benchmarking RTS. Operations Research Spektrum, 17:67{86. Boroujerdi, A. (1994). Joint Routing and Persistent Data Structures. PhD thesis, University of New Mexico, Albuquerque, NM.

107 Boroujerdi, A., Dong, C., Ma, Q., and Moret, B. M. E. (1993). Joint routing in networks. NETFLO93: Network Optimization Theory and Practice. Centro Studi Cappucini, San Miniato, Italy. Boyd, E. A. (1993). Generating Fenchel cutting planes for knapsack polyhedra. SIAM Journal Optimization, 3:734{750. Carlton, W. B. and Barnes, J. W. (1996). A note on hashing functions and tabu search algorithms. European Journal of Operational Research, 95:237{239. Cheong, C. K. (1985). Survey of investigations into the missile allocation problem. Master's thesis, Naval Postgraduate School, Monterey, CA. Cho, C.-C. (2000). Optimal weapon allocation with target-dependant kill probabilities. Presented at the Spring INFORMS National Meeting in Salt Lake City, UT. Chu, P. C. and Beasley, J. E. (1998). A genetic algorithm for the multidimensional knapsack problem. Journal of Heuristics, 4:63{86. CPLEX 6.6 (1999). Using the CPLEX Callable Library. ILOG Inc. CPLEX Division, Incline Village, NV. Crowder, H., Johnson, E. L., and Padberg, M. W. (1983). Solving large-scale zero-one linear programming problems. Operations Research, 31:803{834. Dammeyer, F. and Voss, S. (1993). Dynamic tabu list management using the reverse elimination method. Annals of Operations Research, 41:31{46. Dyer, M. E. and Walker, J. (1998). Dominance in multi-dimensional multiple-choice knapsack problems. Asia Pacic Journal of Operational Research, 15(2):159{168.

108 Eckler, A. R. and Burr, S. A. (1972). Mathematical models of target coverage and missile allocation. Technical Report DTIC: AD-A953517, Military Operations Research Society, Alexandria, VA. Everett III, H. (1963). Generalized Lagrange multiplier method for solving problems of optimum allocation of resources. Operations Research, 11:399{417. Fisher, M. L. (1981). The Lagrangian relaxation method for solving integer programming problems. Management Science, 27(1):1{18. Garey, M. R. and Johnson, D. S. (1979). Computers and Intractability: A Guide to the Theory of NP-Completeness. Freeman, San Francisco. Georion, A. M. (1974). Lagrangian relaxation for integer programming. Mathematical Programming Study, 2:82{114. Glover, F. (1968). Surrogate constraints. Operations Research, 16:741{749. Glover, F. (1975). Surrogate constraint duality in mathematical programming. Operations Research, 23(3):434{451. Glover, F. (1986). Future paths for integer programming and links to articial intelligence. Computers and Operations Research, 19:533{549. Glover, F. (1989). Tabu search | part I. ORSA Journal on Computing, 1(3):190{206. Glover, F. (1990). Tabu search | part II. ORSA Journal on Computing, 2(1):4{32. Glover, F. (1995). Tabu search fundamentals and uses. Technical report, Graduate School of Business, University of Colorado, Boulder, CO. Condensed version published in Mathematical Programming: State of the Art, Birge and Murty (eds), pp.64{92, 1994.

109 Glover, F. (2001a). Tabu search and shifting landscape methods. Hearin Center for Enterprise Science, School of Business Administration, The University of Mississippi, University, MS. Glover, F. (2001b). Tutorial on surrogate constraint approaches for optimization in graphs. Hearin Center for Enterprise Science, School of Business Administration, The University of Mississippi, University, MS. Glover, F. and Kochenberger, G. A. (1996). Critical event tabu search for multidimensional knapsack problems. In Osman, I. H. and Kelly, J. P., editors, Metaheuristics: Theory and Applications, pages 407{427. Kluwer Academic Publishers, Boston. Glover, F. and Laguna, M. (1998). Tabu Search. Kluwer Academic Publishers, Boston. Glover, F., Taillard, E., and de Werra, D. (1993). A user's guide to tabu search. Annals of Operations Research, 41:3{28. Greenberg, H. J. and Pierskalla, W. P. (1970). Surrogate mathematical programming. Operations Research, 18:924{939. Hammer, P. L., Johnson, E. L., and Peled, U. N. (1975). Facets of regular 0-1 polytopes. Mathematical Programming, 8:179{206. Hana, S. and Freville, A. (1998). An ecient tabu search approach for the 0-1 multidimensional knapsack problem. European Journal of Operational Research, 106(2{3):659{675. Hansen, P. (1986). The steepest ascent mildest descent heuristic for combinatorial programming. presented at the Congress on Numerical Methods in Combinatorial Optimization, Capri, Italy.

110 Hosein, P. A. (1989). A Class of Dynamic Nonlinear Resource Allocation Problems. PhD thesis, MIT, Cambridge, MA. Hosein, P. A. and Athans, M. (1990). Some analytical results for the dynamic weapontarget allocation problem. Technical Report DTIC: AD-A219281, MIT, Cambridge, MA. Jakobovits, R. H. (1995). Centralized and distributed resource optimization for strike warfare planning. Presented at the Real Time Strike Force Planning Workshop at George Mason University, Fairfax, VA and the Naval Research Laboratory in Washington, DC. Kirkpatrick, S., Jr., C. D. G., and Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220(4598):671{680. Laporte, G. and Osman, I. H. (1995). Metaheuristics in combinatorial optimization. Annals of Operations Research, 60. Lee, S. H. (1995). Route optimization model for strike aircraft. Master's thesis, Naval Postgraduate School, Monterey, CA. Li, V. C., Boyd, E. A., and Elkins, D. A. (1997). Solving real time strike force asset allocation. Working paper, Texas A&M University. Lin, E. Y. H. (1994). Multiple choice programming: A state-of-the-art review. International Transactions in Operational Research, 1:409{421. Lin, E. Y. H. (1998). A bibliographical survey on some well-known non-standard knapsack problems. INFOR, 36(4):274{317.

111 Lokketangen, A. and Glover, F. (1996). Probabilistic move selection in tabu search for 0/1 mixed integer programming problems. In Osman, I. H. and Kelly, J. P., editors, Metaheuristics: Theory and Applications, pages 467{488. Kluwer Academic Publishers, Boston. Lokketangen, A. and Glover, F. (1998). Solving zero-one mixed integer programming problems using tabu search. European Journal of Operational Research, 106(2{ 3):624{658. Lokketangen, A., J$ornsten, K., and Storoy, S. (1994). Tabu search within a pivot and complement framework. International Transactions of Operations Research, 1(3):305{316. Magazine, M. J. and Chern, M. S. (1984). A note on approximation schemes for multidimensional knapsack problems. Mathematics of Operations Research, 9:244{247. Magazine, M. J. and Oguz, O. (1981). A full polynomial approximation algorithm for the 0-1 knapsack problem. European Journal of Operational Research, 8:270{273. Magazine, M. J. and Oguz, O. (1984). A heuristic algorithm for the multidimensional 0-1 knapsack problem. European Journal of Operational Research, 16:319{326. Martello, S. and Toth, P. (1990). Knapsack Problems: Algorithms and Computer Implementations. John Wiley & Sons, Inc., Chichester, United Kingdom. Matlin, S. (1970). A review of the literature on the missile allocation problem. Operations Research, 18:334{373. Moser, M., Jokanovic, D. P., and Shiratori, N. (1997). An algorithm for the multidimensional multiple-choice knapsack problem. IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences, E80-A(3):582{589.

112 Nemhauser, G. L. and Trotter, L. E. (1974). Properties of vertex packing and independence system polyhedra. Mathematical Programming, 6:48{61. Nemhauser, G. L. and Wolsey, L. A. (1988). Integer and Combinatorial Optimization. Wiley and Sons, New York. Osman, I. H. and Kelly, J. P. (1996). Metaheuristics: An overview. In Osman, I. H. and Kelly, J. P., editors, Metaheuristics: Theory and Applications, pages 1{21. Kluwer Academic Publishers, Boston. Padberg, M. W. (1973). On the facial structure of set packing polyhedra. Mathematical Programming, 5:199{215. Padberg, M. W. (1979). Covering, packing, and knapsack problems. Annals of Discrete Mathematics, 4:265{287. Padberg, M. W. (1980). (1,k)-congurations and facets for packing problems. Mathematical Programming, 18:94{99. Salkin, H. M. and de Kluyver, C. A. (1975). The knapsack problems: A survey. Naval Research Logistic Quarterly, 22(1):127{144. Trotter, L. E. (1975). A class of facet producing graphs for vertex packing polyhedra. Discrete Mathematics, 12:373{388. Volgenant, A. and Zoon, J. A. (1990). An improved heuristic for multidimensional 0-1 knapsack problems. Journal of the Operational Research Society, 41(10):963{970. Wolsey, L. A. (1976). Further facet generating procedures for vertex packing polytopes. Mathematical Programming, 11:158{163.

113 Woodru, D. L. and Zemel, E. (1993). Hashing vectors for tabu search. Annals of Operations Research, 41:123{138. Zhang, G. Q., Glover, F., and Kochenberger, G. (2000). Critical event tabu search method for multi-diminution knapsack problem. Presented at the Spring INFORMS National Meeting in Salt Lake City, UT. Zuniga, M. and Gorman, P. (1995). Threat site overight modeling for strike route optimization. In Proceedings of the First International Symposium on Command and Control Research and Technology, Washington, DC. National Defense University. Zuniga, M., Uhlmann, J. K., and Hofmann, J. B. (1994). The independence joint routing problem: Description and algorithm approach. Naval Research Laboratory, Washington, DC.

114 APPENDIX A DESCRIBING MODEL 1 IN MATRIX NOTATION Model 2 is not ecient to display in matrix's form, while it is more convenient and concise to express Model 1 in matrix's notation as presented in Li, Boyd and Elkins (1997). For consistency with Model 2, we introduced Model 1 in Section A of Chapter II in summation form. For simplicity, we also translate Model 1 in matrix's form here. Notice all symbols dened below are independent of those discussed in Chapter II

S : the set of strike force assets C : the set of all asset classes T : the set of all targets P : the set of all possible packages under consideration A : a matrix of size jC j jP j R : a matrix of size jT j jP j b : a column vector of size jC j, with bi = number of assets available in asset class i c : a row vector of size jP j, with cj = value associated with package j x : a column vector of size jP j, with xj = 1 if the j th package is assigned to some target, xj = 0 otherwise Each row of A corresponds to an asset class in the set C , and each row of R corresponds to a target in T . A and R have the same number of columns, with each column

115 corresponding to a potential package that is possibly assigned to target t. The entry aij of A represents the number of assets in class i contributed to package j . The constraints Ax b guarantee that whatever packages are chosen, the overall demand for each asset class will not exceed the supply. The entry rij of R equals 1 if package j is designed for target i, and 0 if not. The constraints Rx 1, where 1 is a column vector of 1's, guarantee that no target will be assigned to more than one package. The model can then be stated as follows. max s.t.

cx Ax b Rx 1 xj 2 f0 1g 8j 2 P:

116 VITA Vincent Chi-Wei Li received the B.B.A. degree in transportation engineering and management from National Chiao-Tung University, Taiwan, in 1987. In 1992, he started his mater's degree in industrial engineering at Texas A&M University and recived the M.S. degree in 1994. His elds of interest include tabu search, integer programming, linear programming, optimization, operations management, simulation, and economics. He joined American Airlines in summer 2001, working on applications of revenue management. His permanant mailing address is 2573 Hall Johnson Rd., Apt 1118, Grapevine, TX 76051. He can be reached at the following email address: [email protected].

solving strike force asset allocation problems: an ...

solving strike force asset allocation problems: an ...

Suggest Documents

Solving Railway Track Allocation Problems

An application of hidden Markov models to asset allocation problems

an asset allocation puzzle - CiteSeerX

An Asset Allocation Puzzle - CiteSeerX

SOLVING MULTI-CRITERIA ALLOCATION PROBLEMS - Core

Towards the Real Time Solution of Strike Force Asset

An algebraic method for solving central force problems

Asset Allocation

An Evolutionary Approach to Multiperiod Asset Allocation

Asset Allocation and Long-Term Returns: An

Asset Allocation Strategies:

asset allocation - Diaz Invest

Regime-Based Asset Allocation

asset allocation - Diaz Invest

Dynamic Asset Allocation

Asset Allocation and Derivatives

Regime-Based Asset Allocation

Calculating Asset Allocation - CiteSeerX

academy registration - Miami Strike Force

academy registration - Miami Strike Force

strike force - ACLU of Arizona

strike force - ACLU of Arizona

Solving Large-Scale Allocation Problems with Partially ... - CiteSeerX

Solving resource allocation problems in cognitive radio networks: a ...