Observation Scheduling Simulation Framework

Observation Scheduling Simulation Framework: Design and First Results. Stephen N. Fraser and Iain A. Steele Astrophysics Research Institute, LJMU, Twelve Quays House, Egerton Wharf, Birkenhead, UK ABSTRACT This paper describes a modular component architecture for the construction of observation schedulers along with a simulation framework with which schedulers can be tested under a variety of environmental scenarios. We discuss a series of basic efficiency and quality metrics which can be used to measure the value of schedules. Results are presented from a series of simulations using this framework in which a set of observation scheduling paradigms ranging from on-demand despatching to a short-horizon look-ahead scheduler are tested under a series of increasingly challenging environmental conditions. Keywords: Simulation, Telescopes, Optimization, Scheduling

1. INTRODUCTION Previous studies of scheduling, both in the realm of observation scheduling1, 2 and in other very disparate realms where a considerable variety of scheduling paradigms have been used,3–8 suggest that it should be possible to devise a framework consisting of a set of fundamental components common to most schedulers along with a set of extendable interface specifications from which plugin implementations can be provided to allow many different observation schedulers to be built. Additionally it was felt that it would be useful to extend this framework to include components with which to build a simulation environment in which to test these schedulers. This paper describes a software architecture for constructing observation schedulers. In Sect. 2 we describe a series of complexity and quality metrics to evaluate scheduler performance. The architectural layers and software components from which schedulers can be built and a simulation environment with which they can be tested are described in Sect. 3. In Sect. 4 we describe an extendable set of scoring heuristics which can be used to evaluate groups of observations. Some example schedulers are described in Sect. 5 along with the results of some preliminary experiments performed to test the architecture. Suggestions for further research are presented in Sect. 6.

2. METRICS In order to compare schedulers a set of metrics are required to guage the effectiveness of the algorithms under different circumstances.

2.1 Problem Complexity Metrics Problem Complexity Metrics (PCM) describe the difficulty of a particular scheduling problem and provide an independant variable with which to asses the performance of different schedulers.. It might be found for example that a particular scheduler which performs well on an easy problem is out-performed by an alternative scheduler when faced with difficult problems. In addition to their use for assessing the performance of schedulers under test, it is likely that an advanced scheduler could itself make use of these metrics by looking ahead to see where hotspots (difficult scheduling periods) are likely to occur and take these into account in its decision making, thus introducing an element of feedback into the process. Further author information: (Send correspondence to S.N.F) S.N.F.: E-mail: [email protected], Telephone: +44 (0) 151 231 2903

Advanced Software and Control for Astronomy II, edited by Alan Bridger, Nicole M. Radziwill Proc. of SPIE Vol. 7019, 70190M, (2008) · 0277-786X/08/$18 · doi: 10.1117/12.789369

Proc. of SPIE Vol. 7019 70190M-1 2008 SPIE Digital Library -- Subscriber Archive Copy

2.1.1 Contention profile Cc This is the time evolving profile of the number of observation groups which could be scheduled according to their explicit timing constraints. Additional refinements include convolving with the probability of the time actually being available based on likely weather and technical downtime forecasts. The average contention C¯c over the course of a night gives an estimate of how overloaded the schedule could potentially be. 2.1.2 Demand profile Cd A given group of observations has potentially multiple observing windows when it should be attempted. During any given window the group can be considered to have a demand D on the time within that window. If the group g has an estimated execution time X(g) and its window of opportunity ∗ is φ(g, t) then the group’s demand over that window is D = X(g)/(X(g) + φ(g, t)). If we add up the demand contributions of all the groups which are enabled at any given time we should have a measure of how much demand is placed on that instant. If this aggregate demand exceeds unity then it is likely that some of the groups will not be observed i.e. the requirement for time is greater than the actual time available. 2.1.3 Load Cl The load is a fairly simple metric which describes the ratio of total potentially executable time in a given observing night to the length of the night LN (or astronomical night LA ). Load is calculated using the sum of execution times for each executable window of each group that can be executed during the night. It includes all windows of all groups whether they must be executed that night or not. An urgency weighted load Cul can also be calculated which weights each execution time by the reciprocal of the number of remaining nights in which the window could be observed. Critical load Ccl is a pruned version of the normal load where only groups which must be executed that particular night are included.

2.2 Schedule Quality Metrics Schedule Quality Metrics (SQM) are used to compare the results of using different schedulers or scheduling policies on a given scheduling problem. These are needed to determine which scheduler performs best. There are a number of simple measures that can be used independantly though complex tradeoffs between these make a more generic measure more useful. • Optimal Height metric QOH measures how close to the optimum height the group is observed in its execution window. The optimum height is basically the highest elevation attained by the group during the feasible observation window - in a given window this may be at the end (rising target), start (setting target) or somewhere in the middle of the observing window (transiting target). Due to the non-linear relationship between elevation and airmass it may be better to measure the ratio of execution-time airmass aactual to best achievable airmass aopt as an Optimal Airmass benefit metric QBA . QBA =

1
b(aactual ) LS b(aopt )

(1)

g∈S

where a = sec z is the airmass of group’s target at zenith distance z, b(a) represents the benefit (image quality advantage) for airmass a and LS is the sequence length of the schedule S. . • Priority metric QP X measures the total priority score achieved i.e just sums up the priorites of the groups selected weighted by the amount of time spent performing the group on the basis that e.g. an hour of high priority observing is better than 2 lots of 10 minutes preforming low priority groups. The final value can be normalized by dividing by the length of night (or astronomical night). ∗

this is the window during which it may legally start and may consist of a number of distinct time-separated sub-intervals

Proc. of SPIE Vol. 7019 70190M-2

QP X =

1
X(g, t)p(g) LN

(2)

g∈S

where p(g) is the priority-based value of group g and X(g, t) is the estimated execution time for group g at execution instant t. • Target demand metric QT D measures how urgent the selected groups are with reference to the night’s demand profile Cd (Sect. 2.1.2). QT D =

1
fD (g, t) LN

(3)

g∈S

where fD (g, t) represents the value of the demand heuristic (see Sect. 4) for group g at execution instant t. • Execution target metric QXT is a measure of the amount of the night during which groups are actively being observed. A low value is effectively an indication of slack time. QXT =

1
X(g, t) LN

(4)

g∈S

where X(g, t) is the estimated execution time for group g at execution instant t. • Urgency (remaining nights) metric QRN counts a decreasing function (for convenience 1/n) of the number of nights remaining n for any groups in an observability window Φ(g, t) at the time of selection/execution. It effectively measures how urgent the window was in terms of the number of future chances of executing the group other than tonight. e.g. If a group could be executed tonight or on 3 future nights this yields a value for RN of 4 and thus 1/4 as its urgency. QRN =


g∈S

1 R(g, t, Φ(g, t))

(5)

where R(g, t, φ) counts the number of remaining nights for a feasiblity window φ(g, t) of group g at execution instant t - i.e. it counts the number of feasible sub-intervals of Φ with maximum of 1 per night. • Yield tracking metric QY T measures the ratio of achieved to potential data product to date. A value around 1 indicates that all potential observation windows are being executed, a low value indicates that potential windows are being missed. This turns out to be a very expensive and complex metric to derive. • Score-based utility QSU measures the heuristic score (value) attached to a group by the scheduler in selecting a group - this is a primitive (but easily calculated) version of the combined utility metric QUU .2

3. SCHEDULER COMPONENT ARCHITECTURE The Scheduler Component Architecture (SCA) provides implementations of a number of fundamental modules which together form a framework from which many different scheduling engines can be constructed. In addition a number of tools are provided to allow PCMs and SQMs to be generated during the execution of a scheduler. Fig. 1 shows some of the component layers provided by the SCA.

Proc. of SPIE Vol. 7019 70190M-3

Proc. of SPIE Vol. 7019 70190M-4

Fundamental Components Layer

Stochastic Timing Model Weather Scenario Generator

Environ Scenario Generator

Environ Prediction Model E(t)

Weather Prediction Model W(t)

Time Model t

Prediction Layer

Simulator Components Layer

History Model) H(g,t)

Charge Accounting Model

Accounting Model A(g,t)

Composite Components Layer

Metric Generator

Updater

SimCtrlApp

Sky Monitoring System

Weather Monitoring System

UTC System Clock

External Feeds

Time Signal Generator

Figure 1. Scheduler Component Architecture consists of a set of fundamental modules and plugin implementations of higher functional modules.

Phase2 Model P = {G}

Execution Feasibility Model Φ(g,t,e,h)

Scheduling Engine

Selection Heuristics

Despatcher

Executor

Timing Constraint Calculator

Execution Timing Model ) X(g,t)

Scoring Model

Interface Layer

Execution Layer

3.1 Fundamental Components Layer 3.1.1 Phase2 II model This is the information entered by the observers or on their behalf by automated agents which describes the content and constraints of the observing programs - i.e what to do and when. The information can be broken into the following categories:• Observation specifications - details of the observations; target selection, acquisition and tracking information, instrument selection and configuration, number and length of exposures, mosaic/dither offset patterns. • Timing constraints - when and how often to perform groups of observations. • Observing constraints- limitations on the conditions under which the observations may be taken. • Observing preferences - quantification of the observer’s tradeoffs between competing effects in determining the value of performing observations under particular conditions or at specific times. • Quality requirements - limitations on the amount, regularity and quality of data taken. 3.1.2 History model The history model H(g, t) provides a record of all the execution histories of groups from the phase2 model. 3.1.3 Accounting model The accounting model A(g, t) provides information on the allocation and use of resources by groups, proposals and TAGs. The simple accounting model provides access only to the accumulated balance of the various accounts to date. A more advanced accounting model envisoned for the future deployed system will allow the history of transactions to be followed as an audit trail.

3.2 Composite Components Layer 3.2.1 Execution timing model The execution timing model X(g, t) provides an answer to the question “how long will it take to execute group g ?” by modelling the function of the executor environment (telescope’s robotic control system or simulation control application). Execution is a stochastic process. Knowledge must be provided of the manner (degree of parallelism, sequencing order) in which the execution system will decompose the tasks which make up the observation group along with some knowledge of the uncertainty in individual execution times of these tasks as performed by the telescope’s lower level control systems and instrument systems. It is important this estimate is good - an over-estimate could lead to a potential candidate group being rejected for no good reason, an under-estimate could lead to selection of an unsuitable candidate. The model must take into account a large number of factors including timing information on:• slewing rate in each axis, rotator mode and angle changes, rotator cardinal pointing requirements. • instrument filter defocussing. • fold mirror move/deployment when instrument changes. • instrument configuration change (filter wheel, grating etc). • instrument calibration before/after image, lamp calibrations. • configuration of acquisition instrument when required. • acquisition by acquisition instrument onto spectrograph slit. • autoguider acquisition.

Proc. of SPIE Vol. 7019 70190M-5

• aperture offsets. • readout (per instrument) which can also depend on binning and windowing. • data write-to-disc time (per instrument) which may involve network latency. • recovery and retry delays for failed tasks. 3.2.2 Execution feasibility model The feasibility model Φ(g, t, e, h) provides an answer to the question “can group g be executed at time t given environment e and execution history h subject to its observing and timing constraints ?”.

3.3 Scheduling Engine Component Layer 3.3.1 Selection heuristic Selection heuristics (Sect. 4) are used to select single groups (or sequences of groups) for execution using the scoring/utility metrics provided by the scoring model. Implementations include:• Best score heuristic - selects the group with the highest ranked score. • Biased score heuristic9, 10 - selects a group with probability according to a function of its score. • Random heuristic - selects a group at random (for baseline testing). 3.3.2 Scoring model The scoing model S(g, t, e, a, h) evaluates the contributions of various metrics (Sect. 4) to answer the question “what is the value/reward for performing group g at time t under environment e assuming history h and account balance a ?”. 3.3.3 Candidate generator Uses the execution feasibility model Φ(g, t, e, h) and scoring model S(g, t, e, a, h) to generate a set of the groups which can potentially be executed at time t. 3.3.4 Cost accounting model When observers prepare their proposals to go before the TAG for time and resource allocation the cost or charging model C(g, t, e) is used to calculate the amount of time the groups should take to execute. The standard figures for slewing, acquisition, readout and instrument reconfiguration defined in the charging model are published on the telescope website for observers to calculate their resource requirements and these same values are used to calculate the nominal cost of performing a group. In reality a group may take more or less time than this but this is the resource usage which is charged to the relevant account. It is expected that access to this model will be integrated into a future user GUI by way of a webservice endpoint.

3.4 Interface Components Layer 3.4.1 Despatcher This plugin component is the execution application’s entry point for requesting observations from the scheduler. 3.4.2 Updater This represents a plugin component which allows the executor to supply observation completion statistics back to the fundamental component models (e.g. to let the history model know that a group was successfully executed).

Proc. of SPIE Vol. 7019 70190M-6

3.5 Executive Components Layer 3.5.1 Executor This represents the external application which requires the services of the scheduler. This will either be the telescope’s robotic control system, a simulation control application or a user tool.

3.6 Environmental Prediction Components Layer 3.6.1 Environmental prediction model This plugin component provides estimates of the likely environmental (atmospheric) conditions at future times. The variables potentially available include; seeing, photometricity/extinction and sky brightness. Implementations can range from annual or seasonal climatological statistics through to local predictive schemes. 3.6.2 Weather prediction model A plugin module which provides estimates of likely future weather. Basically all that is required of this module is whether the weather will be good (when observing is possible) or bad (where observing is not possible). Implementations can be as simple or complex as required. 3.6.3 Time model This module provides a time signal. In an operational situation this is simply the system clock. In a simulated environment it is provided by the simulation controller’s time signal generator component.

3.7 Simulation Framework Components Layer The components in this layer are used to build a simulation environment incoporating time-varying and random effects (weather, sky condition, timing) with which to test any schedulers built using SCA. 3.7.1 Stochastic execution timing model A group’s actual execution time will vary due to uncertainty in various mechanical and software operations (often outwith the control of the execution controller). Variations occur in the times taken for target acquisition, instrument re-configuration, slewing etc. The stochastic execution timing model ξ(g, t, e) represents the execution controller’s model of group execution timing. A simulation control application uses this model to decide how long a group will actually take (rather than how long it is expected to take as provided by X(g, t). 3.7.2 Environmental scenario generator (ESG) Implementations of an ESG allow varying sky conditions to be modelled. The data can be fed into the environmental prediction model. We are not generally bothered about the actual details of the conditions just the general classification of conditions into extinction as photometric/non-photometric and seeing into one of 3 or 4 pre-defined bands. The stability of the environmental conditions is represented by a parameter ∆E which indicates the average length of time the environment model stays in a given state. 3.7.3 Weather scenario generator (WSG) Scenarios generated by a WSG implementation can be fed as inputs to the weather prediction model. As with environmental scenarios we only need to know if the weather is good or bad not the precise details of each variable, though this level of detail might be needed for prediction modelling. The stability of the weather is represented by a parameter ∆W which indicates the average length of time the weather model stays in a given state. 3.7.4 Time signal generator Simulation controllers must provide an implementation of this module to allow the controller to synchronize with the scheduler, this is especially important where the scheduler runs as a seperate thread/process. Some scheduling operations take a finite time to execute (e.g. certain metrics are extremely costly to generate) and it would otherwise be possible for the simulator and scheduler to drift in time.

Proc. of SPIE Vol. 7019 70190M-7

4. HEURISTIC SCORING METRICS (HSM) These are the metrics used by the scheduler to determine the utility (value) of potential observation group candidates from the pool of available groups. Some of these metrics are peculiar to despatch shchedulers, others are more generally applicable. The following HSMs have been identified but is by no means an exhaustive list. • Height metric fh simply measures the elevation of the group’s target at the time of potential observation. Variations on this metric can inlcude taking the mean or peak elevation over the duration of the group. If a group has multiple targets various averages can be used such as taking mean elevation of each target at some point in the execution. More complex averages could take into account the actual times at which each target will be observed. • Airmass metric fair is similar to fh except that the airmass is used rather than elevation thus taking into account the nonlinear nature of the relationship between sky-quality and elevation. • Transit metric ftrans is an attempt to take into account the unfair advantage provided by the two previous metrics fh and fair . These force a bias towards targets whose declination is close to the latitude of the observatory. This metric measures the quality of observing at a given time by taking the current target elevation as a fraction of its best elevation. • Optimal elevation foh and optimal airmass foa metrics attempt to redress this problem by taking the ratio of current elevation to best possible elevation (or airmass) in the group’s feasibility window. • Slew metric fslew attempts to penalize groups for which a long axis slew (including rotator) from the current telescope position is required. • Sky condition matching metrics fsee (also fphot and fsky ) are designed to match a group’s sky condition requirements to the actual (or predicted) conditions at the time of execution. This is intended to ensure that groups which do not require particularly good conditions do not take an excessive share of good conditions. • Lunar condition matching metric fmoon is similar to the above but attempts to prevent groups which can use bright time from taking an excessive shar of dark time. • Priority metric fp is designed to ensure that groups of higher scientific priority get a better share of time. • Resource allocation metric fa measures the use of various resources by a group or its containing proposal. This is typically the use of time from the proposal’s total semester allowance. It can be used either to help distribute time fairly between proposals or to force completion of proposals which have already been started. • Demand metric fdmd uses the group’s demand for time (Sect. 2.1.2) as an indication of the urgency of performing the group. The idea behind this is to select the groups which have the most critical requirment for a given time period. • Urgency metric furg is similar to the demand metric but measures the number of additional chances (in terms of alternative nights on which the group could be observed if not tonight. • Yield tracking metric fyt tracking deficit in data product yield - needs past and future yield estimates and yield to-date to allow effect of any deficit to be deduced.

5. EXPERIMENTAL RESULTS 5.1 Basic Despatch Scheduler The basic despatch scheduler (BDS) uses a weighted metric scoring model and best ranked selection heuristic. All candidate groups are scored and ranked, the best candidate at the instant being selected without reference to any future potential.

Proc. of SPIE Vol. 7019 70190M-8

Comparison of effect of scheduling horizion on QSU with time 100 95 90 85

QSU

80 75 70 65 60 QLAS H=0.5 QLAS H=1.0 QLAS H=2.0 BDS (H=0)

55 50 10-May

17-May

24-May

31-May

07-Jun

14-Jun Time

21-Jun

28-Jun

05-Jul

12-Jul

19-Jul

Figure 2. Effect of horizon length on QSU metric under unstable environment ∆E = 0.5 hours. BDS exceeds all QLAS horizons during second half but QLAS performs marginally better in earlier half of scenario.

Comparison of effect of length of look-ahead horizon H on QSU 95 BDS QLAS H=0.5h QLAS H=1.0h QLAS H=2.0h BDS Average

90

Σ QSU

85

80

75

70

65 0

0.5

1

1.5 2 2.5 3 Environment model stability parameter ∆ E [hours]

3.5

4

Figure 3. Effect of look-ahead horizon length H on QSU metric under a range of environment model stability parameter ∆E ranging from 0.25 to 4 hours. BDS provides fairly stable baseline with average QSU = 74.09. QLAS shows reduced effectiveness (lower QSU ) under unstable conditions (low ∆E) for all tested horizon lengths. As environment becomes more stable (∆E increasing) the longer horizon QLAS becomes increasingly more effective - i.e. the optimization effect.

Proc. of SPIE Vol. 7019 70190M-9

Comparison of effect of length of sequence count Ns on QSU 88

∆ E = 1h ∆ E = 2h ∆ E = 4h BDS Average

86 84 82

Σ QSU

80 78 76 74 72 70 68 10

100

1000

10000

Sample sequence count

Figure 4. Effect of sequence count Ns on QSU under a range of environment model stability parameter ∆E. QSU is seen to increase with Ns in line with expectation as more of the search space of potential solutions is explored but with diminishing effect at high Ns .

5.2 Look-Ahead Scheduler Look-ahead schedulers work on the assumption that the environmental (and weather) conditions will remain stable for a given period of time (the parameter ∆E in an environment scenario represents the length of this stable period). By suitable selection of the length of look-ahead horizon it is hoped that a better degree of optimization can be achieved than by a BDS. Identification of the size of this horizon is clearly important. Choosing a value too large means that the schedule generated is likely to break (reducing the optimizing effect). Selection of too short a horizon wastes potential optimization opportunities.

5.3 Quantum Look-Ahead Scheduler QLAS is an implementation of a look-ahead scheduler in which lists of potential candidates are generated at fixed intervals ∆τq for upto a given horizon length H in the future. A number of sequences Ns are generated at random from these candidate lists and evaluated. The best (highest value) sequence is selected and then executed until it either completes or breaks e.g. due to changing environmental conditions.

5.4 Experimental setup The two schedulers already discussed (BDS and QLAS) were compared under varying environmental scenarios. Seperate tests were performed to determine the effects of varying the look-ahead horizon H and sequence count Ns . The same scoring model was used for both BDS and QLAS. Single environmental scenarios were used based on environmental stability parameter ∆E ranging from 0.5 to 4 hours. The Phase II model (MD1) was used. This produces a medium load and has a high percentage of observations requiring dark moon conditions resulting in pronounced monthly variations in most quality metrics (see dips around 20th May and 18th June in Fig. 2). The stochastic execution timing model was swapped for a fixed timing model. Statistical variation is thus not well handled due to lack of time available to run simulations so no error bars are shown.

Proc. of SPIE Vol. 7019 70190M-10

5.5 Initial comparison An initial test was made using a highly unstable environment (∆E = 0.5 hours). Fig. 2 shows the results of a series of runs using BDS and QLAS with H values of 0.5, 1 and 2 hours. As can be seen under these (severe) conditions BDS is as or more effective then QLAS. It was also found that BDS performs consistently better at filling the available time.

5.6 Comparison of effect of varying horizon H A set of horizon lengths ranging from 0.5 to 4 hours were tested yielding the results displayed in Fig. 3. As can be seen, increasing H yields some overall improvement over the BDS though clearly this is not constant, i.e. some of the time BDS beats all QLAS examples, especially at lower ∆E as might be expected. Study of the amount of available execution time actually used shows somewhat unexpectedly that QLAS tends to be less efficient overall than BDS - this is accounted for by the fact that BDS always tries to select something to do, QLAS will sit idle for short periods in order to await a more lucrative observation. There are clearly some dangers in this approach, especially if the conditions should change during the wait thus leaving the awaited high-value observation no longer available, this then becomes wasted time.

5.7 Comparison of effect of changing sequence count Ns In order to examine the effect of varying the sequence count in QLAS, a series of simulations were run with Ns varying from 10 through to 10000 in logarithmic progression and with fixed values of H = 1 hour and ∆τq = 60 secs. The experiments were performed under 3 different environmental scenarios with ∆E selected at 1,2 and 4 hours along with a baseline comparison performed using BDS. The results are given in Fig. 4 and show that the schedule quality improves significantly as we move to higher Ns values though with diminishing effect. At low Ns BDS always beats QLAS. These results are more or less as expected. The QLAS is looking for the best sequence from a potentially very large number of possible sequences. With small Ns it stands a good chance of missing the highest scoring sequences. As Ns increases we are searching more of the space of possible solutions.

6. FUTURE WORK The preliminary simulations already described provide an indication of some of the effects that can be investigated but there is the need to perform many more simulation runs than are illustrated here in order to determine the extent of random variation. The main requirment for this is the availability of sufficient CPU time. Several different new types of scheduler will be investigated. • Minimum energy scheduler (MES) assigns a repulsive force to each executable group window determined by its importance, this being based on scientific priority and other factors. The time varying utility measure is treated like an energy well such that stronger groups repel weaker groups away from their prefered location (in time). A particularly weak group may be forced completely out of its well and thus not be executed, otherwise it is simply pushed aside to a less favorable time slot. The overall schedule is determined by dropping all the groups into the well and allowing the system energy to minimize. • Bidding agent scheduler (BAS) assigns time dependant utilities to each group represented by an agent. Each agent has funds assigned to it based on combinations of scientific priority and TAG’s assigned time allocation under various atmospheric conditions. The agents then take part in bidding rounds for their preferred timeslots. Sufficient funds are assigned to groups to allow each program to complete but additional funds can be assigned to allow users to boost their bids at certain key times. These booster funds cannot be topped up but allow the users to have some influence on the scheduler outcome.

Proc. of SPIE Vol. 7019 70190M-11

REFERENCES [1] Fraser, S. N., “Robotic telescope scheduling: The Liverpool Telescope experience,” in [Optimizing Scientific Return for Astronomy through Information Technologies, Proceedings of SPIE ], Quinn, P. J. and Bridger, A., eds., 5493, 331–340, SPIE (sep 2004). [2] Fraser, S. N., “Scheduling for Robonet-1 homogenous telescope network,” Astronomische Nachrichten 327(8), 779–787 (2006). [3] Wellman, M. P., MacKie-Mason, J. K., Reeves, D. M., and Swaminathan, S., “Exploring bidding strategies for market-based scheduling,” in [EC ’03: Proceedings of the 4th ACM conference on Electronic commerce], 115–124, ACM Press, New York, NY, USA (2003). [4] Zhang, W. and Dietterich, T. G., “A reinforcement learning approach to job-shop scheduling,” in [Proceedings of the International Joint Conference on Artificial Intellience], (1995). [5] Popova, E. and Morton, D., “Adaptive stochastic manpower scheduling,” in [WSC ’98: Proceedings of the 30th conference on Winter simulation], 661–668, IEEE Computer Society Press, Los Alamitos, CA, USA (1998). [6] Policella, N., Smith, S. F., Cesta, A., and Oddi, A., “Steps toward computing flexible schedules,” in [Proceedings of Online-2003 Workshop CP 2003], (2003). [7] Kramer, L. and Smith, S., “Maximizing flexibility: A retraction heuristic for oversubscribed scheduling problems,” in [Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence], (August 2003). [8] Hart, E. and Ross, P., “An immune system approach to scheduling in changing environments.,” in [Proceedings of the Genetic and Evolutionary Computation Conference], 1559–1566 (1999). [9] Bresina, J., “Heuristic-biased stochastic sampling,” in [Proceedings of the AAAI-96], 271–278 (1996). [10] Cicirello, V. and Smith, S., “Amplification of search performance through randomization of heuristics,” in [Principles and Practice of Constraint Programming: 8th International Conference Proceedings], 124–138, Springer-Verlag (September 2002).

Proc. of SPIE Vol. 7019 70190M-12