A survey of recent advances in discrete input parameter discrete-event ...

IIE Transactions (2004) 36, 591–600 C “IIE” Copyright  ISSN: 0740-817X print / 1545-8830 online DOI: 10.1080/07408170490438726

A survey of recent advances in discrete input parameter discrete-event simulation optimization JAMES R. SWISHER1 , PAUL D. HYDEN2 , SHELDON H. JACOBSON3,∗ and LEE W. SCHRUBEN4 1

Mary Washington Hospital, 1001 Sam Perry Boulevard, Fredericksburg, VA 22401, USA Department of Mathematical Sciences, Clemson University, O-326 Martin Hall, Clemson, SC 29634-0975, USA 3 Department of Mechanical and Industrial Engineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801-2906, USA E-mail: [email protected] 4 Department of Industrial Engineering and Operations Research, University of California (Berkeley), Berkeley, CA 94720, USA 2

Received June 2000 and accepted June 2003

Discrete-event simulation optimization is a problem of significant interest to practitioners interested in extracting useful information about an actual (or yet to be designed) system that can be modeled using discrete-event simulation. This paper presents a survey of the literature on discrete-event simulation optimization published in recent years (1988 to the present), with a particular focus on discrete input parameter optimization. The discrete input parameter case differentiates techniques appropriate for small and for large numbers of feasible input parameter values. Examples of applications that illustrate these methods are also discussed.

1. Introduction Simulation optimization provides a structured approach to determine optimal input parameter values, where optimal is measured by a function of the output variables (steady state or transient) associated with a simulation model. Several excellent surveys have been written on this topic. Swisher et al. (2003) provide an overview of discrete simulation optimization techniques using ranking, selection and multiple comparison procedures. Azadivar (1999) provides a survey of issues specific to simulation optimization. Andradottir (1998a, 1998b) also presents a review of simulation optimization techniques, focusing on both gradient estimation techniques (for continuous input parameters) and random search methods (for discrete input parameters). Carson and Maria (1997) present a general summary of simulation optimization. Fu (1994a, 1994b) provides a comprehensive review of simulation optimization and simulation gradient estimation techniques. Vysypkov et al. (1994) classify and analyze different situations in simulation model optimization and suggest appropriate search algorithms. Gaivoronski (1992) offers a survey of recent results on the optimization of stochastic discrete-event dynamic systems. Sargent (1991) identifies and discusses some of the research issues in the metamodeling of simulation models as they relate to the goal of the study, including optimization and the decisions that must be made by analysts. Safizadeh ∗

Corresponding author

C 2004 “IIE” 0740-817X 

(1990) contributes a general survey of simulation optimization techniques and procedures. Park (1990) provides an overview of simulation optimization techniques, including a discussion of methods appropriate to uni-modal and multimodal objective functions. Glynn (1989) discusses research issues associated with optimizing simulated systems, including convergence rates for different gradient estimators and Stochastic Approximation (SA) algorithms. The objective of this paper is to survey the simulation optimization literature since 1988, with a particular focus on discrete input parameter optimization. This paper is a companion to that of Jacobson and Schruben (1989) which covers the pre-1989 literature. At that time, most simulation optimization procedures were path-searchbased, involving some type of gradient estimation technique on continuous input parameters (e.g., response surface methodology, perturbation analysis) imbedded in a SA algorithm. Other techniques that were reported included pattern search methods (e.g., Hooke and Jeeves, Nelder-Mead simplex method), random methods and integral methods. The updated survey in this paper covers several new advances that have surfaced through the course of the past decade, including multiple comparisons with the best and metaheuristics such as tabu search and simulated annealing. Note that related issues such as gradient estimation or sensitivity analysis that are not specific to discrete input parameter simulation optimization are not included in this survey. The paper is organized as follows: Section 2 provides a description of the problem. Section 3 looks at simulation

592 optimization techniques designed for discrete input parameters. Section 4 discusses simulation optimization software modules and add-ons that have been developed for commercial simulation software packages. Section 5 presents concluding comments, as well as future directions and opportunities with simulation optimization.

Swisher et al. lent tutorial reviews of these two techniques. A more detailed treatment of these topics can be found in Bechhofer et al. (1995) and Hsu (1996). If the set  is infinite or very large, then techniques such as ordinal optimization (Ho, Sreenivas, and Vakili, 1992), simulated annealing (Fleischer, 1995), tabu search (Glover and Laguna, 1997) and genetic algorithms (Liepins and Hilliard, 1989) have been adapted for the simulation environment.

2. Problem description 3.1. Small number of feasible solutions Consider a discrete-event simulation model with p deterministic input parameters ψ ≡ (ψ1 , ψ2 , . . . , ψp ) and q stochastic output variables Y ≡ (Y1 , Y2 , . . . , Yq ), where Y is a function of ψ (i.e., Y = Y (ψ)). Suppose that the input parameters are defined over a feasible region . Define a real function of Y , C(Y ), that combines the q output variables into a single stochastic output variable. The goal is to determine values for ψ such that F(ψ) , the simulation response function, is optimized. Typically, F(ψ) ≡ E[C(Y (ψ))]. The p deterministic input parameters ψ can be either continuous or discrete (or both). For now, assume that the p input parameters are either all continuous or all discrete. If the p input parameters are continuous, assume that F(ψ) is continuously differentiable, where ψ is defined over the feasible region . If the p input parameters are discrete, the feasible region  can be finite (and small), finite (and large) or countably infinite (the last two classifications will be grouped together). The simulation optimization techniques that have been applied depend on whether the input parameters are continuous or discrete; the focus of this paper will be on the discrete case. The challenge associated with determining ψ ∗ = arg optψ∈ F(ψ) is that F(ψ) cannot be observed directly, but rather, must be estimated. This may require multiple simulation run replications or long simulation runs, with F(ψ) ˆ estimated by F(ψ) using Y i (X) as the ith output from n simulation replications. The stochastic nature of the output from a simulation run complicates the optimization problem. Note that this survey focuses on single-response optimization; papers that look at multiple-response techniques are not considered (see Evans et al. (1991) for a description of techniques that can be used for the multi-criteria simulation optimization problem).

3. Discrete input parameter methods Several techniques have been developed for simulation optimization when the input parameter values (i.e., the size of ) is discrete (Goldsman et al., 1991). If the set  is finite and small, ranking and selection (Goldsman and Nelson, 1998a) and multiple comparisons procedures (in particular, multiple comparisons with the best as adapted by Hsu and Nelson (1988) for the simulation environment) are appropriate. Goldsman and Nelson (1994, 1998b) give excel-

Ranking and Selection (R&S) focuses on selecting the optimal input parameter values over a finite set  = {ψ 1 , ψ 2 , . . ., ψ k }, k < +∞, where k is “small” (i.e., two to 200). The objective is to determine which of the k input parameter values minimizes F(ψ). By applying a two-stage procedure (Dudewicz and Dalal, 1975; Rinott, 1978), the output from simulation runs at the k input parameter values can be used to determine the most likely input parameter values that minimize F(ψ). By defining differences in F(ψ) that are less than a user-specified value δ > 0 to be insignificant, we can ensure the probability of making the correct selection P∗ by choosing the lengths of our simulation runs carefully. In general, as P∗ approaches one, or alternatively, as δ approaches zero, the lengths of the k simulation runs must approach infinity. Procedures of this type are referred to as indifference-zone R&S procedures. To apply these procedures, the simulation runs must be independently seeded to ensure that the simulation outputs from each run are independent. Koenig and Law (1985) extend the indifference-zone approach for use as a screening procedure. They present a method for selecting a subset of size m (user specified) of the k systems so that with a probability of at least P∗ , the selected subset will contain the best system. Gray and Goldsman (1988) provide an application of indifference-zone R&S for choosing the best airspace configuration for a major European airport. Chen et al. (1996), Chen et al. (1997), Chen et al. (1998), and Lin et al. (1999) address the problem of optimal computing budget allocation for discrete-event simulation through an extension of indifference-zone R&S. Hyden and Schruben (1999) introduce the idea of an optimal allocation path, a series of budget allocations that yield the optimal probabilities of choosing the best system at all budget levels in a simultaneous simulation environment. Morrice et al. (1998) and Morrice et al. (1999) propose an indifference zone R&S procedure that allows multiple performance measures through the use of a multiple attribute utility function. A second type of R&S procedure, known as subset selection, aims to produce a subset of random size that contains the best system with probability P∗ without specification of an indifference zone (i.e., δ = 0). Originally these procedures saw little usage, since they required equal and known variances among alternatives, although Sullivan and Wilson (1989) have made subset selection more feasible with a procedure that allows for unknown and unequal variances.

593

Simulation optimization survey Like R&S, Multiple Comparison Procedures (MCPs) attempt to identify the optimal input parameter values over a finite set  = {ψ 1 ,ψ 2 ,. . . ,ψ k }, k < +∞. MCPs approach the optimization problem as a statistical inference problem and, unlike R&S procedures, do not guarantee a decision. Three main classes of MCPs are used in practice: (i) all pairwise Multiple Comparisons approach (MCA) (Tukey, 1953); (ii) Multiple Comparisons with the Best (MCB) (Hsu, 1984; Hsu and Nelson, 1988); and (iii) Multiple Comparisons with a Control (MCC) (Dunnett, 1955). The most popular approach among these is MCB. In particular, MCB looks at F(ψ j ) − opti =j F(ψ i ), j = 1, 2, . . . , k, to determine that j ∗ with F(ψ j∗ ) − opti =j∗ F(ψ i ) > 0. Simultaneous confidence intervals for F(ψ j ) − opti =j F(ψ i ), j = 1, 2, . . . , k, can be used to determine j ∗ , by looking for the confidence interval with the lower confidence limit (rather than the upper confidence limit) that is zero. To apply this procedure, the simulation runs must be independently seeded and the simulation output must be normally distributed (or averaged so that the estimators used are approximately normally distributed). Yang and Nelson (1989, 1991) present modifications to the MCB procedure (and MCA and MCC) that incorporate two variance reduction techniques (common random numbers and control variates). Their results suggest that using variance reduction can lead to correct decisions with higher probabilities. Goldsman and Nelson (1990) present a MCB procedure for steady-state simulation experiments. They also present results on how the batch size can impact on the probability of a correct selection. Nakayama (1995) generalizes these results on applying MCB using batch means in steady-state simulations. Matejcik and Nelson (1993) establish a fundamental connection between indifference-zone procedures and MCB. They present three procedures that incorporate MCB based on whether independent or common random numbers are used to obtain the simulation outputs. Matejcik and Nelson (1995) extend their earlier work by showing that most indifference-zone procedures can simultaneously provide MCB confidence intervals. They also present a two-stage MCB procedure, including guidelines on the number of batches for the first stage. Bofinger and Lewis (1992) also present two-stage procedures for MCC. Nelson and Matejcik (1995) show how common random numbers can be incorporated into a combined R&S-MCB procedure. Swisher and Jacobson (2002) illustrate this procedure on a clinic design healthcare simulation model. Yuan and Nelson (1993) discuss MCB procedures for steady-state simulations, where the simulation outputs are assumed to follow an auto-regressive process with normally distributed residuals. Nakayama (1996, 1997a) presents a single-stage MCB procedure that is asymptotically valid for steady-state simulations, hence extending the domain of applicability of previous MCB procedures. Nakayama (1997b, 2000) presents a two-stage MCB procedure using common random numbers for steady-state simulations and shows that it is asymptotically valid. Nelson et al.

(2001) combine the indifference-zone, subset selection and MCP methodologies to perform screening of poor candidates and then selection of the best when the number of competitors is “large.” Kolonja et al. (1993) give an application of MCB using common random numbers to a truck dispatching system simulation model. Damerdji et al. (1996) formulate a procedure for selecting the best system for transient simulations with known variances. Damerdji and Nakayama (1996, 1999) present a two-stage procedure for multiple comparisons with a control in steady-state simulations. Chick (1997) offers a Bayesian methodology for selecting the best simulated system. Inoue and Chick (1998) compare Bayesian and frequentist approaches for selecting the best system. Chick and Inoue (1998, 1999) extend Chick’s (1997) work to the study of sampling costs and the value of information arguments to improve the computational efficiency of identifying the best system. Retrospective optimization uses several simulation output runs to generate a performance function. This performance function is then optimized using standard deterministic techniques. Healy and Schruben (1991) illustrate retrospective optimization on a yield management simulation model to determine optimal production lot sizes, on a tandem queuing network simulation model to determine optimal buffer allocations and on an inventory management simulation model to determine optimal inventory policies. Fu and Healy (1992) present a comparison of retrospective optimization with a perturbation analysis (gradient-based) SA algorithm applied to (s,S) inventory systems. They conclude that the two approaches are so different (and are best applied over different horizons), that no meaningful comparisons can be made. Fu and Healy (1997) extend their earlier work and develop a hybrid gradientbased, retrospective algorithm that attempts to build on the strengths and mitigate the shortcomings of each approach. Healy (1994) presents a simulation optimization technique, where the idea is to retrospectively solve a related deterministic optimization problem with respect to realizations of the stochastic effects as if the outcomes of all uncertainties were known in advance. In particular, Healy addresses situations where it makes sense to define the best solution as the one that is most likely to produce a desirable outcome. In contrast to retrospective techniques, Hurrion (1997) presents an interesting example of simulation optimization using a visual interactive simulation combined with a neural network metamodel. The simulation model first trains the neural network metamodel, then the metamodel is used to find an approximate optimal solution. This solution is then used as the starting point for a more precise search. Alternatively, Nasereddin and Mollaghasemi (1999) give a neural network approach specifically tailored for discreteevent simulation in designing systems. 3.2. Large number of feasible solutions When the set of possible input parameter values is discrete, but very large, different methods than those presented

594 in Section 3.1 must be applied. Ordinal optimization (Ho, Sreenivas, and Vakili, 1992; Ho, 1994) focuses on finding good solutions, rather than trying to find the very best solution (i.e., goal softening) (Lee et al., 1999). In doing this, ordinal optimization reduces the search for an optimal solution from sampling over a very large set of solutions to sampling over a smaller, more manageable set of good solutions. Ho and Deng (1994) show that goal softening cannot only be effective in search spaces of manageable cardinality, but that it can be effective for problems with in excess of 10 000 alternatives. Dai (1995) and Xie (1997) address the convergence properties of ordinal optimization. Xie (1997) shows that for a regenerative system, the probability of obtaining a desired solution using ordinal optimization converges at an exponential rate while the variance of the performance measures converge at a rate of O(1/t 2 ), where t is the simulation time. Deng et al. (1992) study the impact of correlation on ordinal optimization. They show, both theoretically and experimentally, that correlation between the output data results in ordinal optimization selecting a larger number of good solutions in the selected set. Lau and Ho (1997) examine how the choice of subset selection rules affects the alignment probability (i.e., the probability of the intersection between the “good enough” subset and the selected subset). They provide recommendations for the subset size given certain system parameters. Chen (1996) establishes two simple methods for establishing a lower bound on the probability that the selected subset contains at least one good design and the probability that the best of the selected subsets is very close to the true best design. Deng and Ho (1997) introduce an iterative ordinal optimization procedure, analogous to traditional hill climbing procedures, that moves from one subset or search representation to another (rather than between individual solutions in a solution space). Ho and Larson (1995) introduce a new approach to rare-event simulation using ordinal optimization. The performance criterion’s dependence on the rare events is relaxed so that it depends on more frequent attempts. This goal softening produces a considerable reduction in the computational cost, but the user must be willing to accept some loss of precision. Cassandras et al. (1998) examine a class of resource allocation problems that can be formulated as discrete optimization problems. They address the stochastic version of the problem with the use of ordinal estimates to derive an efficient solution scheme. Chen (1995) and Chen et al. (1996) present an extension of ordinal optimization for smartly allocating computing budgets for discrete-event simulation. Some of their numerical testing suggests a factor of 10 speedup in simulation time when using their computing budget allocation algorithm. Ho and Cassandras (1993) present an overview of parallel processing in the optimization of stochastic discrete-event systems with an emphasis on using ordinal optimization. Cassandras and Pan (1995a) compare ordinal optimization, a discrete stochastic comparison algorithm, and a stochastic descent algorithm for solving the buffer allocation problem. Yang et al. (1997)

Swisher et al. review the fundamentals of ordinal optimization and provide two examples of the technique applied to manufacturing systems. Xie (1994) presents an application of ordinal optimization to a token partition problem for stochastic timed event graphs. Ganz and Wang (1994, 1995) apply ordinal optimization to the wavelength assignment problem in a WDM optical network. Wieselthier et al. (1995) apply ordinal optimization and the standard clock simulation technique to the voice-call admission-control problem in integrated voice/data multihop radio networks. General search strategies such as simulated annealing (Eglese, 1990; Fleischer, 1995), genetic algorithms (Liepins and Hilliard, 1989; Muhlenbein, 1997), and tabu search (Glover and Laguna, 1997) have been adapted for the stochastic environment associated with discrete-event simulation optimization. Simulated annealing mimics the annealing process for crystalline solids, where a solid is cooled very slowly from an elevated temperature, with the hope of relaxing towards a low-energy (objective function value) state. Genetic algorithms emulate the evolutionary behavior of biological systems to create subsequent generations that guide the search towards optimal/near-optimal solutions. Tabu search uses memory to guide the search towards optimal/near-optimal solutions, by dynamically managing a list of forbidden moves. There are numerous variations of tabu search that have been developed, including probabilistic tabu search, aspiration criteria and also candidate list strategies, among others (Glover and Laguna, 1997). Manz et al. (1989) present a simulated annealing algorithm applied to a simulation model of an automated manufacturing system. Haddock and Mittenhall (1992) discuss how to apply simulated annealing to discrete-event simulation models. Baretto et al. (1999) discuss linear move and exchange move optimization which is based on simulated annealing and apply their algorithm to a steel works plant. Zeng and Wu (1993) present a strategy for introducing perturbation analysis techniques into a simulated anneal¨ ing algorithm for simulation optimization. Yucesan and Jacobson (1996) present a simulated annealing algorithm to address the simulation search problem ACCESSIBILITY ¨ (Jacobson and Yucesan, 1999). Stuckman et al. (1991) compare a Bayesian/sampling algorithm, simulated annealing and genetic algorithms on their applicability and effectiveness to globally optimize simulations. Brady and McGarvey (1998) compare the use of simulated annealing, tabu search, genetic algorithms and a frequency-based heuristic for optimizing staffing levels in a simulation model of a pharmaceutical manufacturing laboratory. Dummler (1999) uses simulation in conjunction with genetic algorithms to improve cluster tool performance in semiconductor manufacturing. Faccenda and Tenga (1992) apply a genetic algorithm to a simulation model of a process plant to determine the optimal design of the plant. Tompkins and Azadivar (1995) utilize a genetic algorithm for a class of manufacturing system simulation models, where the algorithm is capable of optimizing over qualitative input

595

Simulation optimization survey parameters. Azadivar and Shu (1998) use a genetic algorithm search to choose the optimal maintenance policy in a manufacturing environment where the policy parameters are both quantitative and qualitative. Glover et al. (1996) apply tabu search to several simulation models. Hall et al. (1996) apply evolutionary strategies to optimize a pull production system simulation model. Hall and Bowden (1996) combine evolutionary strategies and the Hooke and Jeeves algorithm to optimize a manufacturing simulation model. Hall and Bowden (1997) compare evolutionary strategies and tabu search with the Nelder-Mead method for simulation optimization. For smooth convex response surfaces, they observe that the Nelder-Mead method obtains better solutions. These articles illustrate how general search strategies conceived for discrete optimization problems have been successfully adapted for simulation optimization. Andradottir (1996) uses a random walk approach to develop a simulation optimization algorithm over a large discrete set of input parameter values. Alrefaei and Andradottir (1999) present a modified simulated annealing algorithm that can be applied to discrete-event simulations. They also provide conditions under which the algorithm converges almost surely to a global optimum, with the cooling schedule kept fixed. Alrefaei and Andradottir (2001) give a modified stochastic ruler algorithm (Yan and Mukai, 1992) and show that it converges almost surely to a global optimum. Cassandras and Strickland (1989) and Cassandras (1993) display a technique termed “rapid learning” that aims at enumerating all possible sample paths under different input parameter values of the model based on the observed sample path under nominal input parameter values. Rapid learning relies on two necessary conditions: (i) observability, which asserts that every state observed in the nominal path is always richer in terms of feasible events than the states observed in constructed paths; and (ii) constructability, which requires that the lifetime of the events has the same distribution as their residual lives. Cassandras and Pan (1995b) provide an application of rapid learning to traffic smoothing. ¨ Yucesan and Jacobson (1995, 1997) show that verification of the observability condition is a NP-hard search problem and thereby encourage the development of heuristic procedures to validate the applicability of rapid learning. ´ Shi and Olafsson (1997) present the nested partitions method for simulation optimization. This method combines partitioning, random sampling, a selection of a promising index and backtracking to create a Markov chain ´ that converges to a global optimum. Olafsson and Shi (1998, 1999) show that the Markov chain generated by nested partitions converges geometrically fast to the true stationary distribution and use these results to derive a ´ stopping criterion for the method. Olafsson (1999) links this work with iterative ranking and selection methods. Shi et al. (1999) discuss the nested partitions method as applied to a simultaneous simulation environment. Morito and Lee (1997) present a local search algorithm to deter-

mine optimal dispatching priorities in a job shop simulation model. The growing attention to simulation optimization over a large set of discrete input parameter values, coupled with the increased popularity of search strategies such as simulated annealing, tabu search and genetic algorithms, has provided a rich avenue of research that crosses both discrete-event simulation and discrete optimization. Moreover, such interaction is likely to provide new algorithm breakthroughs impacting both areas. As computing power becomes even less expensive, these search strategies should continue to find new, more interesting applications in the simulation framework.

4. Software The commercial simulation software development community has begun to recognize the value of simulation optimization modules and add-ons to their simulation software packages. Benson (1997), Heflin and Harrell (1998a, 1998b) and Price and Harrell (1999a, 1999b) discuss the ProModel optimization suite, named SimRunner. SimRunner is capable of running three types of projects: (i) statistical advantage; (ii) design of experiments; and (iii) optimization. Statistical advantage helps the user select a warm-up length, run length and replication count. Design of experiments creates a factorial design of experiments and tells the user whether changing a given input factor affects the objective function and the significance of the effects. Optimization varies multiple input parameters to find a combination that provides the best objective function value. Kalasky (1996) makes use of ProModel’s SimRunner optimizer to schedule production at a plant for a supply chain problem. Kalasky optimizes this model using a single performance measure (total cost), but notes that SimRunner allows a user to define a multi-variable weighted objective function. Sadowski and Bapat (1999) describe the general capabilities of Arena’s built-in optimization system, OptQuest. OptQuest allows the analyst to bound the search space for the input parameters of interest, to set a maximum search length and to return the input parameter combinations that meet the analyst’s criteria. Glover et al. (1999) discuss OptQuest in the broader context of recent advances in simulation optimization. Carson (1996) describes how AutoStat, an add-on package for the simulation software Automod, allows users to take advantage of several statistical techniques, including R&S. While Arena does not include a built-in package for optimization in its product suite, it does allow users to build templates to imbed optimization problems. Baker (1997) describes such a template that solves a truck loading problem using a knapsack approach. Savory (1995) describes the use of Mathematica to produce response surface plots to determine sets of optimal input parameter values. Rawles (1998) describes the Witness optimizer which allows the user to define an objective function comprised of

596 one or more input parameter values and output variables. Each input parameter has a range of possible values or a discrete set of values it may assume. It also allows linear combinations of input parameters to form constraints. The user may choose from several optimization methods: (i) all combinations; (ii) min/mid/max; (iii) hill climb; (iv) random solutions; or (v) adaptive thermostatistical simulated annealing. The all-combinations method runs all possible input parameter value combinations to determine the optimum. The min/mid/max method runs three evaluations based upon the minimum, midpoint and maximum values of all input parameter values. The hill climb method generates random input parameter values (at each iteration) that are accepted (rejected) if these values are of higher (lower) quality than the previous iteration values. The random solutions method generates random combinations of input parameter values. The adaptive thermostatistical simulated annealing is based upon simulated annealing. This technique is less dependent upon the starting input parameter values than the hill climb method and is less likely to choose a local optimum instead of a global optimum. The thermostatistical SA search parameters can be set and adjusted by the user. For simulation optimization software modules and addons to gain lasting credibility among both researchers and practitioners, it may be necessary to provide a benchmark framework by which all such software tools can be compared and evaluated. Bowden and Hall (1998) point to the need for a unified strategy for simulation optimization and propose six domains as the foundations for all automated simulation optimization tools. They suggest that simulation optimization tools should address problem formulation (i.e., construction of the objective function), methods (i.e., optimization methods used), classification (i.e., how the method may be classified or compared to other methods), strategy and tactics (i.e., how to best make use of computational resources for increasing solution accuracy), intelligence (i.e., the solver’s ability to select the proper strategic approach) and interfaces (i.e., the interface between the optimizer and the user, and the interface between the optimizer and the simulation model). Bowden and Hall believe that the adoption of a unified view will help to improve the next generation of simulation optimization tools through a focus on the problem as a whole.

5. Conclusions Research and applications in the field of simulation optimization has advanced significantly in the past decade. The 1980s were dominated by gradient estimation research over continuous input parameters. The 1990s and on to the 2000s have seen a shift towards discrete sets of input parameter values. The unification of R&S and multiple comparison procedures has been a significant advancement for those problems with discrete input parameters and a small number of feasible solutions. Simultaneous R&S-MCB proce-

Swisher et al. dures will likely dominate the literature in this field in the next decade. Moreover, the cross-fertilization between discrete optimization heuristics and discrete-event simulation has provided a rich avenue for research breakthroughs and new hybrids in the field of optimization. All these events have served to foster a climate conducive to new research di¨ rections for simulation optimization. For example, Yucesan et al. (1998) describe a framework for carrying out simulation optimization via a distributed simulation on the internet. Hill (1998) addresses the increasing popularity of heuristics to solve discrete optimization problems. Due to the increased popularity of heuristics, there is an inevitable need of a standardized test for empirical evaluations as described by Hill (1998) and Reilly (1998). As more robust optimization algorithms are incorporated into simulation software packages, an increasing number of practitioners will be able to take advantage of sophisticated optimization procedures that previously may have been unavailable to them. Simulation enjoys the unique position of possessing an active and vibrant research community (mostly in academia and research labs) interacting with a dedicated and application-driven software development community. This environment provides a fertile ground ripe for technology transfer and other exchanges between the theoretical and the applied foci present in these two communities. Research that generalizes the domain of application of all these techniques, as well as the incorporation of variance reduction and other efficiency enhancement techniques, will serve to broaden the application and appeal of simulation optimization procedures. Such enhancements may also serve to build bridges, for example, between R&S/MCB procedures and the techniques used for discrete input parameters with a large number of feasible solutions. The next decade will most certainly provide many new breakthroughs that build upon such research foundations for the next generation of simulation software tools and simulation practitioners.

Acknowledgements The second author was supported by a National Science Foundation Graduate Research Fellowship, as well as fellowships through the Semiconductor Research Corporation and the Olin Foundation. The third author was supported in part by the Air Force Office of Scientific Research (F49620-01-1-0007) and the National Science Foundation (DMI-9907980). The fourth author was supported in part through a joint SRC (FJ-490) and NSF (DMI-9713549) research project in semiconductor operations modeling.

References Alrefaei, M.H. and Andradottir, S. (1999) A simulated annealing algorithm with constant temperature for discrete stochastic optimization. Management Science, 45(5), 748–764.

Simulation optimization survey Alrefaei, M.H. and Andradottir, S. (2001) A modification of the stochastic ruler method for discrete stochastic optimization. European Journal of Operational Research, 133(1), 160–182. Andradottir, S. (1996) A global search method for discrete stochastic optimization. SIAM Journal on Optimization, 6(2), 513–530. Andradottir, S. (1998a) A review of simulation optimization techniques, in Proceedings of the 1998 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 151–158. Andradottir, S. (1998b) Simulation optimization, in Handbook on Simulation, Banks, J. (ed.), Wiley, New York, NY, pp. 307–333. Azadivar, F. (1999) Simulation optimization methodologies, in Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 93–100. Azadivar, F. and Shu, J.V. (1998) Use of simulation in optimization of maintenance policies, in Proceedings of the 1998 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 1061–1067. Baker, G.S. (1997) Taking the work out of simulation modeling: an application of technology integration, in Proceedings of the 1997 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 1345– 1351. Barretto, M.R.P., Chwif, L., Eldabi, T. and Paul, R.J. (1999) Simulation optimization with the linear move and exchange move optimization algorithm, in Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 806–814. Bechhofer, R.E., Santer, T.J. and Goldsman, D.M. (1995) Design and analysis of experiments for statistical selection, screening, and multiple comparisons, Wiley, New York, NY. Benson, D. (1997) Simulation modeling and optimization using ProModel, in Proceedings of the 1997 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 587–593. Bofinger, E. and Lewis, G.J. (1992) Two-stage procedures for multiple comparisons with a control. American Journal of Mathematical and Management Sciences, 12, 253–275. Bowden, R.O. and Hall, J.D. (1998) Simulation optimization research and development, in Proceedings of the 1998 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 1693–1698. Brady, T. and McGarvey, B. (1998) Heuristic optimization using computer simulation: a study of staffing levels in a pharmaceutical manufacturing laboratory, in Proceedings of the 1998 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 1423–1428. Carson, J.S. (1996) AutoStatTM : output statistical analysis for AutoModTM users, in Proceedings of the 1996 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 492–499. Carson, Y. and Maria, A. (1997) Simulation optimization: methods and applications, in Proceedings of the 1997 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 118–126. Cassandras, C.G. (1993) Rapid Learning Techniques for Discrete Event Systems: Modeling and Performance Analysis, Irwin, Homewood, IL. Cassandras, C.G., Dai, L. and Panayiotou, C.G. (1998) Ordinal optimization for a class of deterministic and stochastic discrete resource allocation problems. IEEE Transactions on Automatic Control, 43(7), 881–900. Cassandras, C.G. and Pan, J. (1995a) Some new approaches to discrete stochastic optimization problems, in Proceedings of the IEEE Conference on Decision and Control, 1, IEEE, Piscataway, NJ, pp. 573– 578. Cassandras, C.G. and Pan, J. (1995b) Parallel sample path generation for discrete event systems and the traffic smoothing problem. Journal of Discrete Event Dynamic Systems, 5(2/3), 187–217. Cassandras, C.G. and Strickland, S.G. (1989) Sample path properties of timed discrete event systems. Proceedings of the IEEE, 77, 59– 71. Chen, C.H. (1995) An effective approach to smartly allocate computing budget for discrete event simulation, in Proceedings of the IEEE Conference on Decision and Control, 3, IEEE, Piscataway, NJ, pp. 2598– 2603.

597 Chen, C.H. (1996) A lower bound for the correct subset-selection probability and its application to discrete-event systems simulations. IEEE Transactions on Automatic Control, 41(8), 1227–1231. Chen, C.H., Chen, H.C. and Dai, L. (1996) A gradient approach for smartly allocating computing budget for discrete event simulation, in Proceedings of the 1996 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 398–405. ¨ Chen, H.C., Chen, C.H., Dai, L. and Yucesan, E. (1997) New development of optimal computing budget allocation for discrete event simulation, in Proceedings of the 1997 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 334–341. ¨ Chen, C.H., Yuan, Y., Chen, H.C., Yucesan, E. and Dai, L. (1998) Computing budget allocation for simulation experiments with different system structures, in Proceedings of the 1998 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 735–741. Chick, S.E. (1997) Selecting the best system: a decision-theoretic approach, in Proceedings of the 1997 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 326–333. Chick, S.E. and Inoue, K. (1998) Sequential allocations that reduce risk for multiple comparisons, in Proceedings of the 1998 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 669–676. Chick, S. and Inoue, K. (1999) A decision-theoretic approach to screening and selection with common random numbers, in Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 603– 610. Dai, L. (1995) Convergence properties of ordinal comparison in the simulation of discrete event dynamic systems, in Proceedings of the IEEE Conference on Decision and Control, 3, IEEE, Piscataway, NJ, pp. 2604–2609. Damerdji, H., Glynn, P.W., Nakayama, M.K. and Wilson, J.R. (1996) Selecting the best system in transient simulations with known variances, in Proceedings of the 1996 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 281–286. Damerdji, H. and Nakayama, M.K. (1996) Two-stage procedures for multiple comparisons with a control in steady-state simulations, in Proceedings of the 1996 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 372–375. Damerdji, H. and Nakayama, M.K. (1999) Two-stage multiplecomparison procedures for steady-state simulations. ACM Transactions on Modeling and Computer Simulations, 9, 1–30. Deng, M. and Ho, Y.C. (1997) Iterative ordinal optimization and its application, in Proceedings of the IEEE Conference on Decision and Control, 4, IEEE, Discataway, NJ, pp. 3562–3567. Deng, M., Ho, Y.C. and Hu, J.Q. (1992) Effect of correlated estimation errors in ordinal optimization, in Proceedings of the 1992 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 466–474. Dudewicz, E.J. and Dalal, S.R. (1975) Allocation of observations in ranking and selection with unequal variances. The Indian Journal of Statistics, 37B, 28–78. Dummler, M.A. (1999) Using simulation and genetic algorithms to improve cluster tool performance, in Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 875–879. Dunnett, C.W. (1955) A multiple comparisons procedure for comparing several treatments with a control. Journal of the American Statistical Association, 78, 965–971. Eglese, R.W. (1990) Simulated annealing: a tool for operational research. European Journal of Operational Research, 46, 271–281. Evans, G.W., Stuckman, B. and Mollaghasemi, M. (1991) Multicriteria optimization of simulation models, in Proceedings of the 1991 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 894–900. Faccenda, J.F. and Tenga, R.F. (1992) A combined simulation/optimization approach to process plant design, in Proceedings of the 1992 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 1256–1261. Fleischer, M.A. (1995) Simulated annealing: past, present, and future, in Proceedings of the 1995 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 155–161.

598 Fu, M. (1994a) Optimization via simulation: a review. Annals of Operations Research, 53, 199–247. Fu, M. (1994b) A tutorial review of techniques for simulation optimization, in Proceedings of the 1994 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 149–156. Fu, M. and Healy, K. (1992) Simulation optimization of (s,S) inventory systems, in Proceedings of the 1992 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 506–514. Fu, M. and Healy, K. (1997) Techniques for optimization via simulation: an experimental study on an (s, S) inventory system. IIE Transactions, 29, 191–199. Gaivoronski, A.A. (1992) Optimization of stochastic discrete event dynamic systems: a survey of some recent Lecture Notes in Economics and Mathematical Systems, Springer, Laxemburg, Austria, 374, 24– 44. Ganz, A. and Wang, X. (1994) Efficient algorithm for virtual topology design in multihop lightwave networks. IEEE-ACM Transactions on Networking, 2(3), 217–225. Ganz, A. and Wang, X. (1995) Virtual topology design of shared channel lightwave networks. Fiber and Integrated Optics, 14(1), 53–69. Glover F., Kelly J.P. and Laguna, M.L. (1996) New advances and applications of combining simulation and optimization, in Proceedings of the 1996 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 144–152. Glover, F., Kelly, J.P. and Laguna, M. (1999) New advances for wedding optimization and simulation, in Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 255–260. Glover, F. and Laguna, M. (1997) Tabu Search, Kluwer, Norwell, MA. Glynn, P.W. (1989) Optimization of stochastic systems via simulation, in Proceedings of the 1989 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 90–105. Goldsman, L. and Nelson, B.L. (1990) Batch-size effects on simulation optimization using multiple comparisons, in Proceedings of the 1990 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 288–293. Goldsman, D. and Nelson, B.L. (1994) Ranking, selection and multiple comparisons in computer simulation, in Proceedings of the 1994 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 192– 199. Goldsman, D. and Nelson, B.L. (1998a) Comparing systems via simulation. in Handbook on Simulation, Banks, J. (ed.), Wiley, New York, NY, ch. 8. Goldsman, D. and Nelson, B.L. (1998b) Statistical screening, selection and multiple comparison procedures in computer simulation, in Proceedings of the 1998 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 159–166. Goldsman, D., Nelson, B.L. and Schmesier, B. (1991) Methods for selecting the best system, in Proceedings of the 1991 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 177–186. Gray, D. and Goldsman, D. (1988) Indifference-zone selection procedures for choosing the best airspace configuration, in Proceedings of the 1988 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 445– 450. Haddock, J. and Mittenhall, J. (1992) Simulation optimization using simulated annealing. Computers and Industrial Engineering, 22, 387– 395. Hall, J.D. and Bowden, R.O. (1996) Simulation optimization for a manufacturing problem, The Society for Computer Simulation, Huntsville, Alabama, pp. 135–140. Hall, J.D. and Bowden, R.O. (1997) Simulation optimization by direct search: a comparative study. Institute of Industrial Engineerings, Miami Beach, Florida, pp. 298–303. Hall, J.D., Bowden, R.O. and Usher, J.M. (1996) Using evolutionary strategies and simulation to optimize a pull production system. Journal of Materials Processing Technology, 61, 47–52. Healy, K.J. (1994) Identifying policies that are most likely to produce a desirable outcome, in Proceedings of the 1994 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 387–391.

Swisher et al. Healy, K.J. and Schruben, L.W. (1991) Retrospective simulation response optimization, in Proceedings of the 1991 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 901–906. Heflin, D.L. and Harrell, C.R. (1998a) Healthcare simulation modeling and optimization using ProModel, in Proceedings of the 1998 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 185–189. Heflin, D.L. and Harrell, C.R. (1998b) Simulation modeling and optimization using ProModel, in Proceedings of the 1998 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 191–197. Hill, R.R. (1998) An analytical comparison of optimization problem generation methodologies, in Proceedings of the 1998 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 609–615. Ho, Y.C. (1994) Overview of ordinal optimization, in Proceedings of the IEEE Conference on Decision and Control, IEEE, Piscataway, NJ, pp. 1975–1977. Ho, Y.C. and Cassandras, C.G. (1993) Parallel computation in the design and stochastic optimization of discrete event systems, in Proceedings of the IEEE Conference on Decision and Control, IEEE, Piscataway, NJ, pp. 2199–2204. Ho, Y.C. and Deng, M. (1994) The problem of large search space in stochastic optimization, in Proceedings of the IEEE Conference on Decision and Control, IEEE, Piscataway, NJ, pp. 1470–1475. Ho, Y.C. and Larson, M.E. (1995) Ordinal optimization approach to rare event probability problems. Discrete Event Dynamic Systems: Theory and Applications, 5(2–3), 281–301. Ho, Y.C., Sreenivas, R. and Vakili, P. (1992) Ordinal optimization of discrete event dynamic systems. Discrete Event Dynamical Systems, 2(2), 61–88. Hsu, J.C. (1984) Constrained simultaneous confidence intervals for multiple comparisons with the best. Annals of Statistics, 12, 1136–1144. Hsu, J.C. (1996) Multiple Comparisons: Theory and Methods, Chapman & Hall, London, England. Hsu, J.C. and Nelson, B.L. (1988) Optimization over a finite number of system designs with one-stage sampling and multiple comparisons with the best, in Proceedings of the 1988 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 451–457. Hurrion, R.D. (1997) An example of simulation optimisation using a neural network metamodel: finding the optimum number of kanbans in a manufacturing system. Journal of the Operational Research Society, 48(11), 1105–1112. Hyden, P. and Schruben, L.W. (1999) Designing simultaneous simulation experiments, in Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 389–394. Inoue, K. and Chick, S.E. (1998) Comparison of Bayesian and frequentist assessments of uncertainty for selecting the best system, in Proceedings of the 1998 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 727–734. Jacobson, S.H. and Schruben, L.W. (1989) A review of techniques for simulation optimization. Operations Research Letters, 8, 1–9. ¨ Jacobson, S.H. and Yucesan, E. (1999) On the complexity of verifying structural properties of discrete event simulation models. Operations Research, 47(3), 476–481. Kalasky, D.R. (1996) Simulation-based supply-chain optimization for consumer products, in Proceedings of the 1996 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 1373–1378. Kleijnen, J.P.C. (1998) Experimental design for sensitivity analysis, optimization, and validation of simulation models, in Handbook on Simulation, Banks, J. (ed.), Wiley, New York, NY, ch. 6. Koenig, L.W. and Law, A.M. (1985) A procedure for selecting a subset of size m containing the l-best-of-k independent normal populations, with applications to simulation. Communications in Statistics, B14, 719–734. Kolojna, B., Kalasky, D.R. and Mutmansky, J.M. (1993) Optimization of dispatching criteria for open-pit truck haulage system design using multiple comparisons with the best and common random numbers, in Proceedings of the 1993 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 393–401.

Simulation optimization survey Lau, T.W.E. and Ho, Y.C. (1997) Universal alignment probabilities and subset selection for ordinal optimization. Journal of Optimization Theory and Applications, 93(3), 455–489. Lee, L.H., Lau, T.W.E. and Ho, Y.C. (1999) Explanation of goal softening in ordinal optimization. IEEE Transactions on Automatic Control, 44(1), 94–99. Liepins, G.E. and Hilliard, M.R. (1989) Genetic algorithms: foundations and applications. Annals of Operations Research, 21, 31–58. Lin, J., Chen, H.-C. and Chen, C.H. (1999) An asymptotic allocation for simultaneous simulation experiments, in Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 359–366. Manz, E.M., Haddock, J. and Mittenhall, J. (1989) Optimization of an automated manufacturing system simulation model using simulated annealing, in Proceedings of the 1989 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 390–395. Matejcik, F.J. and Nelson, B.L. (1993) Simultaneous ranking, selection and multiple comparisons for simulation, in Proceedings of the 1993 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 386–392. Matejcik, F.J. and Nelson, B.L. (1995) Two-stage multiple comparisons with the best for computer simulation. Operations Research, 43(4), 633–640. Morito, S. and Lee, K.H. (1997) Efficient simulation/optimization of dispatching priority with ’Fake’ processing times, in Proceedings of the 1997 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 872–879. Morrice, D.J., Butler, J. and Mullarkey, P.W. (1998) An approach to ranking and selection for multiple performance measures, in Proceedings of the 1998 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 719–725. Morrice, D.J., Butler, J., Mullarkey, P. and Gavireni, S. (1999) Sensitivity analysis in ranking and selection for multiple performance measures, in Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 618–624. Muhlenbein, H. (1997) Genetic algorithms, in Local Search in Combinatorial Optimization, Aarts, E. and Lenstra, J.K. (eds.), pp. 137– 172. Nakayama, M.K. (1995) Selecting the best system in steady-state simulations using batch means, in Proceedings of the 1995 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 362–366. Nakayama, M.K. (1996) Multiple comparisons with the best in steadystate simulations, in Proceedings of the 2nd International Workshop on Mathematical Methods in Stochastic Simulation and Experimental Design, Publishing House of Saint Petersburg University, Saint Petersburg, Russia. Ermakov, S.M. and Melas, V.B. (eds.), pp. 230– 235. Nakayama, M.K. (1997a) Multiple-comparison procedures for steadystate simulations. Annals of Statistics, 25, 2433–2450. Nakayama, M.K. (1997b) Using common random numbers in two-stage procedures for multiple comparisons with the best for steady-state simulations, in Proceedings of the 11th European Simulation Multiconference, The Society for Computer Simulation International, Istanbul, Turkey. Kaylan, A.R. and Lehmann, A. (eds.), pp. 155– 159. Nakayama, M.K. (2000) Multiple comparisons with the best using common random numbers in steady-state simulations. Journal of Statistical Planning and Inference, 85, 37–48. Nasereddin, M. and Mollaghasemi, M. (1999) The development of a methodology for the use of neural networks and simulation modeling in system design, in Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 537–542. Nelson, B.L. and Matejcik, F.J. (1995) Using common random numbers for indifference-zone selection and multiple comparisons in simulation. Management Science, 41(12) 1935–1945. Nelson, B.L., Swann, J., Goldsman, D. and Song, W. (2001) Simple procedures for selecting the best system when the number of alternatives is large. Operations Research, 49, 950–963.

599 ´ Olafsson, S. (1999) Iterative ranking and selection for large-scale optimization, in Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 479–485. ´ Olafsson, S. and Shi, L. (1998) An integrated framework for deterministic and stochastic optimization, in Proceedings of the 1998 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 743–750. ´ Olaffson, S. and Shi, L. (1999) Optimization via adaptive sampling and regenerative simulation, in Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 666–672. Park, J.W. (1990) Optimization techniques using computer simulation. Korea Information Science Society Review, 8(1), 37–47. Price, R. and Harrell, C.R. (1999a) Simulation modeling and optimization using PROMODEL, in Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 208–214. Price, R. and Harrell, C.R. (1999b) Healthcare simulation modeling and optimization using MEDMODEL, in Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 215–219.  R Rawles, I. (1998) The Witness toolbox—a tutorial, in Proceedings of the 1998 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 223–226. Reilly, C.H. (1998) Properties of synthetic optimization problems, in Proceedings of the 1998 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 617–621. Rinott, Y. (1978) On two-stage selection procedures and related probability inequalities. Communications in Statistics, A7, 799–811. Sadowski, D. and Bapat, V. (1999) The arena product family: enterprise modeling solutions, in Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 159–166. Safizadeh, M.H. (1990) Optimization in simulation: current issues and the future outlook. Naval Research Logistics Quarterly, 37, 807–825. Sargent, R.G. (1991) Research issues in metamodeling, in Proceedings of the 1991 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 888–893. Savory, P.A. (1995) Using mathematica to aid simulation analysis, in Proceedings of the 1995 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 1324–1328. ´ Shi, L. and Olafsson, S. (1997) An integrated framework for deterministic and stochastic optimization, in Proceedings of the 1997 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 358–365. ¨ Shi, L., Chen, C.H. and Yucesan, E. (1999) Simultaneous simulation experiments and nested partition for discrete resource allocation in supply chain management, in Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 395–401. Stuckman, B., Evans, G. and Mollaghasemi, M. (1991) Comparison of global search methods for design optimization using simulation, in Proceedings of the 1991 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 937–944. Sullivan, D.W. and Wilson, J.R. (1989) Restricted subset selection procedures for simulation. Operations Research, 37, 52–71. Swisher, J.R. and Jacobson, S.H. (2002) Evaluating the design of a family practice healthcare clinic using discrete-event simulation. Health Care Management Science, 5(2), 75–88. ¨ Swisher, J.R., Jacobson, S.H. and Yucesan E. (2003) Discrete-event simulation optimization using ranking, selection, and multiple comparison procedures: a survey. ACM Transactions on Modeling and Computer Simulation, 13(2), 134–154. Tompkins, G. and Azadivar, F. (1995) Genetic algorithms in optimizing simulated systems, in Proceedings of the 1995 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 757–762. Tukey, J.W. (1953) The problem of multiple comparisons. Unpublished manuscript. Visipkov, V., Merkuryev, Yu. and Rastrigin, L. (1994) Optimization of discrete system simulation models (survey). Automatic Control and Computer Sciences. Allerton Press, Inc., 28(4), 10–20. Wieslethier, J.E., Barnhart, C.M. and Ephremides, A. (1995) Standard clock simulation and ordinal optimization applied to admission

600 control in integrated communication networks. Discrete Event Dynamic Systems: Theory and Applications, 5(2–3), 243–280. Xie, X. (1994) An ordinal optimization approach to a token partition problem for stochastic timed event graphs, in Proceedings of the 1994 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 581–588. Xie, X. (1997) Dynamics and convergence rate of ordinal comparison of stochastic discrete-event systems. IEEE Transactions on Automatic Control, 42(4), 586–590. Yan, D. and Mukai, H. (1992) Stochastic discrete optimization. SIAM Journal on Control and Optimization, 30, 594–612. Yang, M.S., Loohay, L. and Ho, Y.C. (1997) On stochastic optimization and its applications to manufacturing. Mathematics of Stochastic Manufacturing Systems, Proceedings of the 1997 AMS-SIAM Summer Seminar in Applied Mathematics, Yin, G. and Zhang, Q. (eds.), American Mathematical Society, Williamsburg, VA, pp. 317–331. Yang, W.N. and Nelson, B.L. (1989) Optimization using common random numbers, control variates and multiple comparisons with the best, in Proceedings of the 1989 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 444–449. Yang, W.N. and Nelson, B.L. (1991) Using common random numbers and control variates in multiple-comparison procedures. Operations Research, 39(4), 583–591. Yuan, M. and Nelson, B.L. (1993) Multiple comparisons with the best for steady-state simulation. ACM Transactions on Modeling and Computer Simulation, 3(1), 66–79. ¨ Yucesan, E., Chen, C.H. and Lee, I. (1998) Web-based simulation experiments, in Proceedings of the 1998 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 1649–1654. ¨ Yucesan, E. and Jacobson, S.H. (1995) On the simultaneous construction of sample paths, in Proceedings of the 1995 Winter Simulation Conference, IEEE, Piscataway, NJ, pp. 357–361. ¨ Yucesan, E. and Jacobson, S.H. (1996) Computational issues for ACCESSIBILITY in discrete event simulation. ACM Transactions on Modeling and Computer Simulation, 6(1), 53–75. ¨ Yucesan, E. and Jacobson, S.H. (1997) Complexity of rapid learning in discrete event simulation. IIE Transactions, 29(9), 783–790. Zeng, J. and Wu, J. (1993) DEDS (discrete event dynamic systems) simulation-optimization algorithm using simulated-annealing combined with perturbation analysis. Zidonghua Xuebao Acta Automatica Sinica, 19(6), 728–731.

Biographies James R. Swisher is Director of Operations Design & Analysis for Mary Washington Hospital (MWH) in Fredericksburg, Virginia where he established and oversees MWH’s industrial engineering function. Prior to joining MWH, he worked as Director of Analysis and Design for Lotus Biochemical Corporation. He has also served in the Management Engineering Unit and as Director of Call Center Operations for Biological &

Swisher et al. Popular Culture, Inc. (Biopop) and as an Industrial Engineer and Analyst for the Department of Defense. Mr. Swisher’s research interests lie in the application of operations research techniques to real-world problems, particularly in the fields of healthcare, safety, transportation, and logistics. He holds a B.S. and a M.S. from Virginia Tech in Industrial & Systems Engineering/Operations Research and a MBA from Mary Washington College. Mr. Swisher is a member of the Institute for Operations Research and the Management Sciences (INFORMS), the Institute of Industrial Engineers (IIE), and Alpha Pi Mu. He can be reached via e-mail at [email protected]. Paul Hyden is an Assistant Professor in the Department of Mathematical Sciences at Clemson University. He completed his B.S., M.S., and Ph.D. at Cornell University’s School of Operations Research and Industrial Engineering. His work in the area of operations research focuses on simulation methodology, applied optimization, and stochastics. He is a member of INFORMS and an active participant in the Winter Simulation Conference. His email address is [email protected]. Sheldon H. Jacobson is a Professor, Willett Faculty Scholar and Director of the Simulation and Optimization Laboratory in the Department of Mechanical and Industrial Engineering at the University of Illinois. He has a B.Sc and M.Sc (both in Mathematics) from McGill University, and a M.S. and Ph.D. (both in Operations Research and Industrial Engineering) from Cornell University. His research interests broadly span theory, computation and applications in the field of operations research. He was a co-winner of the 1998 Institute of Industrial Engineers Operations Research Division Application Award, the 2002 Aviation Security International Research Award and the 2003 Best Paper Award in IIE Transactions (focused issue on Operations Engineering). He was also named a 2003 Guggenheim Fellow by the John Simon Guggenheim Memorial Foundation. His research has been supported by the Federal Aviation Administration, the National Science Foundation and the Air Force Office of Scientific Research. He is a member of the Institute for Operations Research and the Management Sciences (INFORMS), the Institute of Industrial Engineers (IIE), the American Society for Engineering Education (ASEE), and the Society for Industrial and Applied Mathematics (SIAM). Lee Schruben is Chairman of the Department of Industrial Engineering and Operations Research and Chancellor’s Professor at the University of California, Berkeley. Prior to joining the faculty at Berkeley, he was on the Operations Research and Industrial Engineering faculty at Cornell where he held the A. Schultz Professorship in Engineering. Professor Schruben’s research interests are on simulation modeling and analysis methodologies with a broad range of applications including semiconductor manufacturing, biopharmaceutical logistics, banking, food services, medicine and golf course design and operation. He can be reached via e-mail at [email protected]. Contributed by the Simulation Department