Unrelated Parallel Machine Scheduling Using Local ...

Matkl.

Corn@.

Pergamon

Modelling Vol. 20, No. 2, pp. 41-52, 1994 Copyright@1994 Elsevier Science Ltd Printed in Great Britain. All rights reserved

0895-7177(94)00103-O

Unrelated

0895-7177/94 $7.00 + 0.00

Parallel Machine Scheduling Using Local Search

C. A. GLASS, C. N. POTTS AND P. SHADE Faculty of Mathematical Studies, University of Southhampton Southhampton, SO9 5NH, U.K. Abstract-Simulated annealing and taboo search are well-established local search methods for obtaining approximate solutions to a variety of combinatorial optimization problems. More recently, genetic algorithms have also been applied. However, there are few studies which compare the relative performance of these different methods on the same problem. In this paper, these techniques are applied to the problem of scheduling jobs on unrelated parallel machines to minimize the maximum completion time. Results of extensive computational tests indicate that the quality of solutions generated by a genetic algorithm is poor. However, a hybrid method in which descent is incorporated into the genetic algorithm is comparable in performance with simulated annealing and taboo search. Keywords-Scheduling, search, Genetic algorithm.

Unrelated parallel machines, Local search, Simulated annealing, Taboo

1. INTRODUCTION This

paper

unrelated

considers parallel

a problem

machines.

in which

Each job

n jobs

are to be scheduled

is to be assigned

without

to one of the machines

preemption

on m

and, at any time,

. . , n) becomes available for processing at time zero and requires a positive integer processing time pzj if it is assigned to machine i The objective is to schedule the jobs so that the maximum completion time, (i = l,...,m). or makespan M, is minimized. Essentially, M represents the total processing time on the most each machine

can process

at most

one job.

Job j (j = 1,.

heavily loaded machine. Karp [l] shows that the problem is NP-hard for the special case of identical machines where the processing time of each job does not depend on the machine to which it is assigned. A branch and bound algorithm is proposed by van de Velde [2]. However, its computational requirements become excessive when there are more than 50 jobs and 5 machines. There has also been much research on the worst-case analysis of heuristics for unrelated parallel machine scheduling [3-91. The best available worst-case performance bounds are for heuristics that are based on linear programming [7,8]. In spite of their theoretical guarantees, Hariri and Potts [4] show that, for randomly generated problems, a simply designed descent (or iterative improvement) algorithm out-performs these linear programming based methods in terms of both solution quality and computation time. Research on local search in scheduling is quite extensive, especially for the flow shop [lo-151 and the job shop [16-231. However, applications to parallel machine scheduling are scarce. Furthermore, there are relatively few computational studies that compare different local search methods on the same scheduling problems. In this paper, we help to fill this gap in the literature by giving an extensive computational comparison of local search methods for unrelated parallel machine scheduling. Our starting point is the descent algorithm of Hariri and Potts [4]. A major disadvantage of descent is that it terminates with a local optimum which may deviate significantly from the true

41

C. A. GLASS et al.

42

optimum. Simulated annealing and taboo search are related techniques which aim to overcome this drawback. Reviews of the methodology and applications of simulated annealing and taboo search are given by Eglese [24] and van Laarhoven and Aarts [25], and by Glover [26-281 and Hertz and de Werra [29], respectively. Another class of local search methods is formed by genetic algorithms, an introduction to which is given by Goldberg [30] and Liepins and Hilliard [31]. We perform extensive computational annealing, taboo search and genetic The remaining

sections

tests to compare algorithms.

of the paper

are organized

the performance

as follows.

of descent,

In Section

2, our application

descent, simulated annealing and taboo search algorithms to unrelated parallel ing is described, while Section 3 gives corresponding applications of a genetic genetic descent algorithm which is a hybrid method. Some of the features and 3 are fixed on the basis of initial experiments. Moreover, simulated and genetic algorithms require numerical values of some parameters to first gives details of experimental design and recommended parameter comparative computational contained in Section 5.

results

are presented

and analyzed.

described in Sections 2 annealing, taboo search be specified. Section 4 settings, and then our

Some concluding

Initialization. Choose an initial feasible solution S to be the current solution and compute its objective function value.

.

Neighbour generation. Select a (feasible) neighbour S’ of the current solution S and compute its objective function value.

.

Acceptance test. Test whether to accept the move from 5’ to S’. If the move is accepted, then S’ replaces S as the current solution; otherwise, S is retained as the current solution.

.

Termination test. Test whether the algorithm should terminate. If it terminates, output an appropriately chosen solution; otherwise, return to the neighbour generation step.

2. NEIGHBOURHOOD 2.1. Structure of Neighbourhood

SEARCH

of

machine schedulalgorithm, and a

.

Figure 1. Outline of a general neighbourhood

simulated

remarks

are

search algorithm.

ALGORITHMS

Search

Before discussing the application of neighbourhood search to our unrelated parallel machine scheduling problem, we give an overview of the approach. Suppose that an objective function is to be minimized over a finite set of feasible solutions. The structure of a neighbourhood search algorithm is shown in Figure 1. A major decision when applying this type of algorithm is the choice of neighbourhood structure. However, there are also other features of the algorithm to be decided upon. In the initialization step, for example, an initial solution may be generated randomly, or by applying a simple heuristic method. Neighbours of the current solution may be selected randomly (with or without The acceptance test, which is usually replacement) or by some systematic search procedure. based on the objective function values for 5’ and S’, either accepts or rejects the move from S to S’ or, in some cases, adopts a ‘wait and see’ approach. In the latter case, several neighbours may be generated before a decision is made on whether the best of them is accepted. Finally, the termination test may allow the algorithm to run either until some convergence condition is satisfied, or until a prespecified limit on computation time or numbers of iterations is met. In some cases, the current solution S at termination is not the best one found during the search. Thus, whenever a superior solution is found, it is stored, so that the best solution generated by the algorithm can be output when the algorithm terminates.

Unrelated Parallel Machine Scheduling 2.2.

Application

First,

of Neighbourhood

43

Search

we discuss our choice of neighbourhood

for the problem

of scheduling

unrelated

parallel

machines. Recall that the objective in our problem is to minimize the makespan. Therefore, the order in which jobs are processed on a machine is immaterial, and a solution is specified by an assignment of jobs to machines. For any solution S, let Ji(S) denote the set of jobs assigned be the corresponding machine load and to machine i (i = 1,. . . ,m), let Pi(S) = cjeJ,(sJp~j We consider two types of neighbours let M(S) = maxi,l,...,m {P,(S)} denote the makespan. solution S. First, for the reassignment neighbourhood, we select any machine i P,(S) = n/r(S), ch oose some job j in Ji(S), and select some machine h (h # i). A S’ of the current solution S is formed by moving job j from machine i to machine h.

of the current for which neighbour

Thus, Jh(S’) = &(S)U{j}, Second, for the interchange

&(S’) = Ji(S)-{j} and Jg(S’) = Jg(S) (g = 1,. . . ,m; g # h; g # i). neighbourhood, we also select any machine i such that Pi(S) = M(S),

choose some job j in Ji(S) and select some machine h (h # i). Additionally, some job k in Jh(S) is chosen. A neighbour S’ is formed by interchanging jobs j and k so that job j moves from machine i to machine h and job k moves from h to i. Therefore, Jh(S’) = Jh(S) U {j} - {k}, Ji(S’) = Jz(S) U {k} - {j} and J,(S’) = Jg(S) (g = 1,. . . ,m; g # h; g # i). We adopt a combined reassignment and interchange neighbourhood which contains all reassignment and interchange neighbours. In neighbourhood search, the chance of reducing the makespan is increased if there is a decrease in the number of machines having total processing time equal to the makespan. Thus, it is useful to associate with any solution S a secondary criterion N(S), which is the number of machines i for which PZ(S) = M(S), and to define the vector V(S) = (M(S), N(S)). For any two solutions S and S’, we write V(S) < V(S’)

if

M(S)

5 M(S’),

or if

M(S)

= M(S’)

and

N(S)

5 N(S’),

in which case V(S) is lexicographically not greater than V(S’). In a similar way, V(S) < V(S’), if V(S) is lexicographically (strictly) smaller than V(S’). In some of our neighbourhood search algorithms, we base our acceptance test on V(S) and V(S’), rather than M(S) and M(S’). We now discuss the computational complexity of evaluating a neighbour S’ of the current solution S. A heap data structure is used to store PI(S), . . . , P,(S), and the frequency of occurrence in S of each machine load is also stored. To compute Ph(S’) and P,(S’) (assuming that S’ is the neighbour obtained from S by reassigning or interchanging jobs between machines h and i) from Ph(S) and Pi(S) and to test whether V(S’) < V(S) (or V(S’) 5 V(S)), or whether M(S’) < M(S) (or M(S’) I M(S)) re q uires constant time. However, to perform the necessary updating of the heap when the move from S to S’ is accepted, and to update the machine load frequencies, requires O(logm) time. This efficient comparison of V(S’) and V(S) and of IM(S’) and M(S) in the acceptance test increases the chances that the neighbourhood search algorithm finds a near-optimal solution using reasonable computational resources. The following subsections describe the structure of our descent, simulated annealing and taboo search algorithms. In descent, a move to a neighbour is accepted if it achieves a reduction in the objective function value. The search proceeds until a local minimum is reached, which is detected when no improvement in the objective function value is possible. Simulated annealing and taboo search algorithms are both variants of descent which attempt to avoid termination at a local minimum, rather than a global minimum. Simulated annealing adopts a probabilistic approach, whereas taboo search is deterministic. In the descent, simulated annealing and taboo search algorithms, an initial solution is obtained by applying the earliest completion time (ECT) h euristic of Ibarra and Kim [6]. At each stage, the ECT heuristic chooses an unscheduled job so that, when it is scheduled on the machine that can complete it earliest, its completion time is as small as possible. This chosen job is then scheduled on the machine that can complete it earliest, and the process is repeated until a complete schedule

C. A. GLASS et al.

44

is obtained. The use of a heuristic solution to initialize a neighbourhood search algorithm is a commonly used device for reducing computation time. The descent, simulated annealing and taboo search algorithms can now be completely specified by the neighbour generation mechanism, and the acceptance and termination tests. 2.3. Descent The descent algorithm which we describe in this section is essentially the method proposed by Hariri and Potts [4]. It provides a benchmark against which to compare our other algorithms. In the neighbour generation step, a systematic search is performed. More precisely, consider a list containing all reassignment neighbours and all interchange neighbours of the current solution S. The neighbour 5” is chosen to be the first entry in the list which has not been selected previously while S is the current solution. The list is not constructed explicitly: knowledge of the mechanism by which the previous neighbour is generated from S, enables generation of the next neighbour. Our acceptance test sets S’ to be the new current solution if V(S’) < V(S); otherwise S is retained as the current solution. It may be viewed as a first decrease (or first improvement) strategy, as opposed to a best decrease (or best improvement) strategy in which the whole neighbourhood is searched before selecting a move to the best neighbour. Finally, the algorithm terminates when V(S) 5 V(S’), for all neighbours S’ of the current solution S, i.e., when the current solution S is a local minimum. 2.4. Simulated

Annealing

The major difference between simulated annealing and descent is in the acceptance rule. In simulated annealing, a neighbour is accepted with a certain probability if its objective function value is worse than that of the current solution. This section describes our implementation of simulated annealing for the problem of scheduling unrelated parallel machines. In accordance with most implementations of simulated annealing, a random generation of neighbours (with replacement) is adopted. Specifically, a random choice is made between generating a reassignment or interchange neighbour; the probability of generating each type of neighbour is 0.5. Having decided on the type, the neighbour S’ is generated by the following procedure. First, a machine i for which P%(S) = M(S) is fixed (if there is a choice, machine i is chosen arbitrarily). Then, a random selection of some job j in Ji(S), of some machine h, where h # i, and, in the case of an interchange neighbour, of some job Ic in Jh(S) yields the required neighbour. The acceptance test in simulated annealing is as follows. Moves which decrease the makespan, or leave it unchanged, are always accepted. Equivalently, we accept S’ as the new current solution if A 5 0, where A = M(S’) - M(S). Furthermore, to allow the search to continue from a local optimum, moves that increase the makespan are accepted with a probability which becomes smaller when the size of this increase is greater. More precisely, S’ is accepted with probability e -A/T if A > 0, where T is a parameter called the temperature which changes during the course of the algorithm. Typically, T is relatively large in the initial stages of the algorithm so that many increases in the objective function are accepted, and then T decreases until it is close to zero in the final stages, at which point the procedure resembles descent. It is necessary to define a cooling schedule which specifies the temperature at each stage of the algorithm. In the scheme of Lundy and Mees [32], a single iteration (the generation and evaluation of one schedule) is performed at each of the K temperatures Tl, . . , TK, which are related by Tk Tk+i = -1 l+PTk where /3 is a constant. Once K, Tl and TK are fixed, the value of fl is given by Ti - TK ' =

(K - ~)T~TK’

Unrelated Parallel Machine Scheduling Initial

experiments

obtained stant

indicate

ature),

geometric

obtained

[33]. Thus,

By implication,

by adopting

schedule

a constant

superior

(for which Tk+l

results

= CyTk, where the con-

are performed

temperature

to those

at each temper-

throughout,

as proposed

by

we adopt the scheme of Lundy and Mees.

the algorithm

we set K so that

tests,

cooling

in the range 0 < cx < 1, and several iterations

and to those

Connolly tional

that the Lundy and Mees scheme produces

by the well-known

CLis chosen

45

terminates

K iterations

when

the computation

are performed.

time is approximately

In our computa-

equal to a specified

time

limit. 2.5.

Taboo

Taboo

Search

search

differs from descent

the best neighbour value.

Taboo

follows.

search

uses a taboo

Unless precautions

after a short sequence

Storing

the time

required

values, rather

taboo

values.

aspiration First,

we give details

parallel

the attribute permutation indicate

function

A(M(S))

=

makespan

M(S)

values A(M) is accepted Having our taboo algorithm:

function

of the taboo

solutions.

the proposal

irrespective

of its taboo

current -

is rejected

solution l),

the taboo algorithm.

the neighbour

The neighbour

M(S)

solution;

move to S’ is rejected.

however,

if V(S)

is to solution

M.

S”

a vector-valued to V) accepted

S’, then we define

from a solution

Thereafter,

we reset A(M(S)) step is identical

whenever

- 1). of

with that in our descent

If S’ is taboo

if V(S’)

a move

the specification

search of the combined

Otherwise,

with

we set initial

= (M(S’),N(S’)

we can complete

test is as follows.

2 V(S),

respect

of S,

with makespan

I n our implementation,

function,

generation

The acceptance

we adopt

(with

move to a solution

S’ of S is found by a systematic

neighbourhood.

then any possible

the new current

list and aspiration

machine

S’, which is a neighbour

Consequently,

2 A(M(S)). < A(M(S)),

in the

of initial experiments

move from a solution

If the best

with makespan

V( S”)

Results

the list to

makespan

list stores the previous

If the solution

status.

and any taboo

unless

[14] for minimizing

S’ from S.

accepted

with V.

the

a move from S to S’ is accepted,

as our attribute.

as follows.

unrelated

list contains

the oldest entry if this causes

= ( M, m) for each possible value of the makespan

and interchange

irrespective

of scheduling

thus, our taboo

Whenever

while generating

from any previously

from S to S’ for which V(S’) search

the

A, which is defined on objective

to that in which the taboo

is motivated

(M(S’),N(S’)

defined

with the aim of reaching

is no worse than the value of the

list:

of Taillard

we use makespan

A, which is compatible

move from any previous

of storing

consequently,

may be lost if the value of its

list which is used in our problem

are reallocated

resulting

it is allowed,

aspiration

saves on

advantage

are taboo;

then the move is eligible for acceptance,

the length

is superior

function

than that

is an entry

solutions,

list. Such a loss can be avoided by allowing the

value of a neighbour

L current

flow-shop problem,

Our aspiration is better

A(M(S)),

of the taboo

L. Following

of jobs that

few iterations.

is taboo.

that this approach

assignment

solutions

a good solution

value of S is added to the list, replacing

longer than

A more subtle

generated

to an aspiration

solution,

is regenerated which contains

for the previous

is that more solutions

taboo

function

values of the previous

become

M(S),

according

Let L denote

machines.

attribute

in the current

for the current

the neighbour

is taboo.

Occasionally,

solution

than complete

function

for which is as

if the value of this attribute

list, rather

solutions,

away from recently

If the objective

function

of whether

a solution

than complete

list to be overridden

function

values in the taboo

is contained

the motivation

solutions

rejected,

a move to

in the objective

a taboo list is constructed

of the current

is normally

of a global optimum.

chosen attribute

is encountered,

it is possible that a previous current

to check whether

is driven further

neighbourhood

function,

To avoid such cycling,

chosen attribute

attribute

attribute the search

are taken,

is taboo, and therefore

in this list.

when a local optimum

list and an aspiration

of moves.

values of some suitably A neighbour

in that,

is made, even though this may cause a deterioration

< V(S),

reassignment and V(S’)

>

then S’ becomes

then a ‘wait and see’ approach

is adopted

C. A. GLASS et al.

46

with S’ becoming a candidate. Under the ‘wait and see’ scheme, once all neighbours of the current solution S have been considered, a move is made from S to the best (with respect to V) candidate neighbour. In the early work in taboo search [34], it is suggested that a move to the best candidate neighbour should always be made, irrespective of whether the move decreases or increase the objective function. This strategy can be view as one of best decrease/smallest increase, However, to save computation time, our test immediately accepts a move to a better neighbour and only searches the complete neighbourhood for the best candidate neighbour when no better neighbour is detected. It can therefore be described as a first decrease/smallest increase strategy. Our termination test is based on a limit on computation time. .

Initialization. Choose an initial population containing s solutions to be the current population. Compute the objective function value for each solution in the population.

.

Evaluation. tion.

.

Reproduction. Create a mating pool by selecting members of the current population. For each selection, a solution is chosen with a probability that is proportional to its fitness.

.

Crossover. Select a pair of elements from the mating pool, neither of which have been chosen previously. Perform a crossover on them with probability P CKl86. If a crossover is not undertaken, the original pair form the input into the mutation step; otherwise, the two offspring are input.

.

Mutation. Perform a mutation on each element of both solutions output from the crossover step with (small) probability Pmut. The resulting solutions are inserted into the new population and their objective function values are computed. If the new population is not complete, return to the crossover step.

.

Termination test. Test whether the algorithm should terminate. If it terminates, output an appropriately chosen solution; otherwise, set the current population to be the new population and return to the evaluation step.

Apply a fitness function to each member of the current popula-

Figure 2. Outline of a standard genetic algorithm.

3. 3.1. Structure

of Genetic

GENETIC

ALGORITHMS

Algorithms

We first provide an overview of the mechanics of genetic algorithms. A prerequisite for applying this type of algorithm is a method to represent any solution as a string, which is usually a finite sequence of characters from a finite alphabet. Also, a fitness function is required, which is a mapping from the solutions to the real numbers with the property that a ‘good’ solutions have a high fitness value. An initial population of solutions is created; in the absence of any problem-specific knowledge, this population is randomly generated. Three types of operations are used to create a new population. First, reproduction, an operation that is motivated by a survival of the fittest philosophy, deletes the weakest members of the population (those with a low fitness or, equivalently, a poor objective function value) and replaces them with replicates of the stronger members to form a mating pool. Second, certain pairs of solutions from the mating pool are selected for crossover. More precisely, two parents combine to produce two offspring, each containing some characteristics of each parent. The hope is that one of the offspring will inherit the desirable features of each parent to produce a good quality solution, while the other offspring which contains the weak features of both parents will be removed later by reproduction. Before placing any offspring or solutions from the mating pool into the new population, a mutation operation is applied to perturb some of these solutions. Although the number of perturbations is usually small, the mutation operator helps to diversify the population. Having formed the new

Unrelated Parallel Machine Scheduling population, the reproduction, crossover and mutation operations are repeated. the fairly standard genetic algorithm described above is outlined in Figure 2. 3.2.

47 The structure of

A Genetic Algorithm

In this section, we explain how a genetic algorithm is applied to our problem of scheduling unrelated parallel machines. In particular, the representation of solutions as strings, the choice of initial population, fitness function, crossover and mutation mechanisms, and termination

test

are discussed. For unrelated parallel machine scheduling, there is a natural representation

of a solution as a

string. Each string is of length n, and the element in position j (j = 1,. . . , n) is the machine to which job j is assigned. Each population in our genetic algorithm has the same size s. To provide a fair comparison with the neighbourhood search algorithms of Section 2 for which the ECT heuristic provides the initial solution, the initial population is seeded with two copies of the ECT solution. The remainder of the population is filled with randomly generated solutions; each element of these strings is randomly selected from the set (1, . . . , m}. The mating pool also has constant size s. At each iteration, members of the current population are selected for the mating pool on the basis of their fitness (function values). Since the probability of selection is proportional to fitness, ‘good’ solutions should be assigned a high fitness, whereas ‘bad’ ones should have a low fitness. Moreover, the difference between these fitness values needs to be carefully chosen. If it is too small, the chance of a better solution surviving to future generations is not appreciably greater than that for poorer solutions. On the other hand, a large differential causes future generations to be swamped with a few relatively good solutions, which results in the population converging quickly to a possibly suboptimal solution. To compute a possible fitness function, we first evaluate the difference between the makespan for the solution 5’ under consideration and for the poorest solution 3 in the current population, to obtain f(S) = M(S) - M(S). S everal other fitness functions which increase the chance that a ‘good’ solution is picked for the mating pool are tested in initial experiments. These include (f(S))2 and maxif - k,O}, where k is set so that the fitness function has a zero value for all but the better solutions in the current population, such as the top ten or those with above average fitness. The function (f(S))2 performs best in initial tests and is therefore adopted as the fitness function in our algorithm. Our algorithm uses a standard two-point crossover to produce two offspring from two parent solutions. Having randomly chosen two points in a string, the sub-strings between the crossover points are interchanged. Figure 3 depicts the crossover operation for an instance with six jobs and three machines. Although the machine assignments of successively numbered jobs are likely to appear in the same crossover segments, it is not clear if this is a disadvantage or whether it can be exploited. Parents 213 2 111

1

213311

311

2

332122

3

312

Offspring

Figure 3. A two-point crossover. If an element of a string is selected for mutation, then the machine assignment of the corresponding job is changed by incrementing it by one, or, in the case that the current assignment is m, by resetting to 1. Mutation is applied with the aim of diversifying the population so that it does not prematurely converge to multiple copies of one solution. Choosing suitable settings for the fitness function and the mutation probability Pmut, allows control over the speed of convergence of the population.

C. A. GLASSetal.

48 We adopt

the usual convention

time limit.

Additionally,

in the current

population

in the final population. that

have the same makespan.

when the algorithm

the computational

in constant

time.

corresponding

pool can be generated constant time,

solutions

is, therefore,

It is possible

to compute

a crossover

operation

into

groups,

]fi]

are stored

requires

O(s(m

each containing

of jobs

intact,

load contributions the machine 3.3.

Descent

The poor performance improvements

the genetic

Our genetic exceptions. function

based

genetic

mating

The time complexity

in

O(sn)

the makespan

of all

for the each

that is obtained

is stored.

of two groups Moreover, O(mfi)

from

the jobs are partitioned

The contribution

algorithm

such hybridization.

to the machine

loads

Each

keeps

crossover

for which the new machine

to compute

the makespan

from

time.

of every population. algorithm

as the genetic

algorithm

of Section

algorithm

of

them.

2.3 to each new solution

of Section

3.2, but with two

we adopt

terminates

3.2

the function

a constant

pool with an equal probability.

the algorithm

that

of authors

of Section

Informally,

experimentation,

for the mating

suggests

A number

and the local search descends

from initial

test is not used:

search.

We modify the genetic

applies the descent

on evidence

in our initial experiments

neighbourhood

to each solution

each solution

termination

To compute

population

time.

by incorporating

algorithm

which selects

secondary

in O(mfi)

It has the same structure

First,

for each member search procedure

step requires

of a new solution

jobs.

exception

is to find the ‘valleys’

descent

that is generated.

[fi]

of the current

of the standard

algorithm

algorithm

+ n)) t ime.

all

Algorithm

have suggested a descent

Once

is evaluated

on a pair of solutions

The mutation

for mutation.

for each group requires

may be achieved

[16,30,35,36] by applying

population

A bisection

is performed

as linked lists.

the makespan

at most

can be computed

A Genetic

population.

crossover

O(s(m

with the possible

load contributions

is stored so

each new population.

in the current

To achieve this saving in computation,

for each group of jobs in each solution all groups

by the algorithm

+ n + logs)).

more efficiently

if m < fi.

found is not

number on O(log s) time, so the complete

Each

of every string is eligible

in the new population

new population

of generating

from the current time.

that strings

since each element

generated

for each solution

to the random

in O(slogs)

time provided

if all solutions

however, requires the use of one random number

pool to select a solution

finds the member

terminates

In some cases, the best solution

currently

complexity

are known, the fitness function Reproduction,

the algorithm

terminates.

makespans

of the mating

test which is based on a computation

test is employed:

Thus, the best solution

it can be output

We now discuss

of using a termination

a secondary

fitness

Second,

the

when a limit on computation

time is met. 4. 4.1.

Experimental

of jobs, sample

was created

of a problem

to compare

To ensure considered problems distribution

was generated

a varied sample and five problems

in the processing each problem

generated sizes.

of the algorithms.

by the number

of machines

times between

with 5, 10,20 and 50 machines,

within

for each of 12 problem

The processing

There

and/or

machines.

A

and 20, 50 and 100 jobs.

size, four different

for each class.

jobs

The main

and the number

classes

are,

of correlation

therefore,

times pij were generated

a total

were 20 test

from a uniform

as follows.

UNCORRELATED: MACHINE

the performance

are its size, as measured

and the degree of correlation of problems

EXPERIENCE

Design

A set of test problems characteristics

COMPUTATIONAL

p,jis an integer from the uniform distribution

CORRELATED: pijis an integer from the uniform

oi is an integer

from the uniform

distribution

[l, loo].

[l,1001.

distribution

[ai + 1, cy + 201, where

Unrelated Parallel Machine Scheduling

pij is an integer

JOB CORRELATED:

an integer

from the uniform

MACHINE

AND

from the uniform

distribution

JOB CORRELATED:

distribution

49

[,$ + 1, ,L$ + 201, where /!?j is

[l, 1001.

pij is an integer

oi@j + 201, where CX~and /Jj are integers

from the uniform distribution from the uniform distribution [l, 201.

[c@

+ 1,

It is in the nature of local search algorithms that they require ‘tuning’ for a specific combinatorial optimization problem. Although the structure of each of our algorithms, which is outlined in Sections 2.1 and 3.1, is determined in advance, some features, such as the fitness function in our genetic algorithm work. Moreover, on the results of a separate set of the other

and the cooling schedule in simulated annealing, require initial computational the setting of numerical values of all parameters for each algorithm is based initial computational tests. All of our initial experiments were performed on 20 test problems with 10 machines and 30 jobs, which were generated as for

problem

sizes.

The parameters

are set independently

of one another

because

of the

prohibitive computational effort in testing combinations. Our computational tests compare the following five algorithms. DA: Descent

algorithm

described

in Section

2.3.

SA: Simulated annealing described in Section 2.4. Initial and final temperatures are Ti = 15 and TK = 0.1. TS: Taboo search described in Section 2.5. The taboo list length is L = 7. GA: Genetic algorithm described in Section 3.2. The population size is s = 100, and crossover and mutation probabilities are P,,,,, = 0.5 and Pmut = 0.005. GD:

Genetic descent described and Pmut = 0.005.

in Section

3.3.

The parameter

values

are s = 20, P,,,,,

= 0.5

Each algorithm was coded in Waterloo C and compiled and run on an IBM 3090 mainframe computer. Two sets of computational tests were undertaken with different limits on computation time. In the first set, a 20 second limit on cpu time per problem was imposed on all algorithms (although this limit is not restrictive in the case of our descent algorithm). By setting the time limit to be independent of problem size, the smaller problems receive a relatively generous computation time allowance, whereas the limit is much more restrictive for larger problems. All algorithms were applied to the full set of 240 test problems. In the second set of tests, algorithms GD, SA and TS were rerun with a limit of 100 seconds of cpu time per problem (algorithms DA and GA were discarded as a result of the runs with a 20 second time limit). It is preferable to evaluate heuristics by comparison with an optimal solution. This approach is not possible here, however, as optimal solutions are unknown for our test problems. Consequently, the best solution value achieved by any algorithm on either run (20 or 100 seconds) is used as a surrogate for the optimal solution value. Our primary performance measure for comparing heuristics is the average (taken over the 20 problems of the same size) relative percentage deviation of the heuristic value from the best known solution value. This is supplemented by a secondary measure; the corresponding maximum relative percentage deviation. The consistency of results across the different correlation classes is also investigated. 4.2.

Computational

Results

We first present an overview of the results of computational tests for algorithms DA, GA, GD, SA and TS when a 20 second time limit is imposed. Average relative percentage deviations of the makespan from the best known solution value (our primary performance measure) are summarized in Table 1. We first observe from Table 1 that the simple descent algorithm DA performs surprisingly well, and it has modest computational requirements (all computation times are less than 1 second). The relative deviation averages no more than 8% for any problem size. However, further analysis

C. A. GLASS et al.

50

Table 1. Average relative percentage deviations Time

for all algorithms

Number of machines, number of jobs:

m, n

limit Algorithm

(sets.)

lo,20

lo,50

50,20

50,50

DA

20

5,20 5,50 5,100 3

3

1

7

4

10,100 20,20 20,50 20,100 3

5

8

3

6

6

6

GA

20

5

6

4

15

12

10

13

21

21

16

19

30

GD

20

0

0

1

0

0

1

0

2

2

0

0

2

SA

20

0

1

1

0

1

2

0

1

3

0

0

0

TS

20

1

1

1

1

1

0

0

0

0

0

0

0

Table 2. Average relative percentage deviations Time

I

for algorithms

50,100

GD, SA and TS.

Number of machines, number of jobs: m, n

limit

Algorithm

(sets.)

5,20 5,50 5,100

lo,20

lo,50

10,100 20,20

20,50 20,100

50,20

50,50

50,100

GD

20

0.0

0.5

0.7

0.4

0.5

1.1

0.4

1.7

2.2

0.0

0.3

1.8

SA

20

0.1

0.7

0.8

0.1

1.4

1.5

0.1

0.7

2.7

0.0

0.1

0.4 0.4

TS

20

0.8

0.6

0.5

0.9

0.5

0.2

0.4

0.4

0.3

0.5

0.3

GD

100

0.0

0.1

0.2

0.1

0.1

0.7

0.2

0.9

1.3

0.0

0.1

1.2

SA

100

0.0

0.2

0.6

0.0

0.6

0.5

0.0

0.4

0.8

0.0

0.0

0.2

TS

100

0.8

0.5

0.4

0.9

0.5

0.1

0.4

0.4

0.2

0.5

0.3

0.3

Table 3. Maximum

relative percentage deviations

for algorithms GD, SA and TS.

Number of machines, number of jobs: 5,20 5,50 5,100

10,100 20,20

m, n

lo,20

lo,50

50,20

50,50

50,100

0.5

1.8

2.0

5.3

1.8

4.7

3.6

20,50 20,100 13.0

9.1

0.0

3.6

4.1

0.8

3.1

3.6

0.8

8.9

6.1

2.3

7.7

10.8

0.0

1.1

2.6

3.3

3.0

2.4

5.0

3.4

1.0

3.8

1.5

2.9

9.1

4.1

2.6

0.0

0.7

0.9

1.5

0.8

3.5

3.6

4.3

9.1

0.0

1.1

3.6

0.4

1.5

3.3

0.4

3.6

3.0

0.0

3.8

3.0

0.0

0.0

1.8

3.3

3.0

2.4

5.0

3.4

1.0

3.8

1.5

2.9

9.1

4.1

2.6

reveals that the maximum relative deviation is a lot higher, ranging from 15% for 100 jobs and 5 machines to over 90% for 20 jobs and 50 machines, and for 50 jobs and 50 machines. The results of Table 1 also indicate that the genetic algorithm GA performs very poorly, even compared to DA which requires much less computation time. The other three algorithms give comparable results, however. The poor performance of GA confirms other results [36], which indicate that, without incorporating problem-specific knowledge or other heuristic methods, genetic algorithms may not be very effective for combinatorial optimization problems. Since the performance of GD, SA and TS is much better than that of DA and GA, these three algorithms are selected for more thorough investigation. Further computational tests for algorithms GD, SA and TS were performed using a time limit of 100 seconds. Table 2 lists average relative percentage deviations (our primary performance measure) for both the 20 second and the 100 second time limit, while corresponding maximum relative percentage deviations (the secondary measure) are given in Table 3. Note that the deviations for GD, SA and TS in Table 1 are obtainable from Table 2 by rounding. The average relative percentage deviations in Table 2 indicate that the three remaining algo rithms are very competitive. The overall performance of TS appears to be a little better than for the other two when there is a 20 second time limit, whereas SA appears to perform marginally better when the time limit is increased to 100 seconds. The maximum relative percentage deviations of Table 3 confirm these conclusions. Also, SA appears to give the most consistent

Unrelated Parallel Machine Scheduling

results:

for the 100 second

runs, the maximum

relative

percentage

51

deviation

for SA is less than

4%, whereas it exceeds 9% for GD and TS. The performance of GD on loo-job problems detracts from its overall effectiveness. When there are 20 or 50 jobs, GD yields superior results to those of TS and is comparable to SA. Increasing the time limit has little effect on TS; very few problems achieve a smaller makespan and those which do are not the ones on which TS yields comparatively poorer results. By contrast, SA and GD benefit across all problem sizes from the extra computation time, which is reflected in improvements to both the average and maximum relative percentage deviations given in Tables 2 and 3. It is also of interest to analyze the performance of the algorithms for each correlation class of problems. However, no significant difference is observed in the relative performance of the algorithms

between

correlation

classes.

5. CONCLUDING

REMARKS

This paper compares the performance of genetic algorithms and neighbourhood search methods for the problem of scheduling unrelated parallel machines. The quality of solutions generated by our standard genetic algorithm (GA) is poor. However, incorporation of descent into the genetic algorithm improves its performance substantially; this genetic descent algorithm (GD) is comparable with our simulated annealing and taboo search algorithms (SA and TS). For these three methods, taboo search generates slightly better solutions when there is a relatively small Genetic descent and simulated annealing improve as the time limit time limit (20 seconds). increases, and there is little to choose between the three algorithms for a larger time limit (100 seconds). Our investigation deals with one type of parallel machine scheduling problem which requires only an assignment of jobs to machines. With different optimality criteria or with additional constraints on the jobs, the processing order of jobs on each machine may have to be considered. For this case, a different neighbourhood structure would be required for neighbourhood search methods, and a different representation of solutions would be needed in a genetic algorithm. Therefore, our conclusions on the relative performance of the different local search methods are not valid for these alternative types of parallel machine problems. It is a disadvantage of many local search methods that much experimentation is necessary to establish the ‘right’ structure and ‘correct’ parameter values. Suitable settings for parameters may depend both on problem size and on characteristics of the instance under consideration. It is unrealistic, however, to adopt a different set of parameter values for every instance. Thus, in our computational work, we use the same parameters over a wide range of sizes and instances. This approach is consistent with many other computational studies which rely on the hope or expectation that the performance of the heuristic is fairly robust against different parameter settings. We conclude that if a standard genetic algorithm is used as a ‘black box’ technique, its performance is likely to be poor. To be worthy of consideration for a combinatorial optimization problem, the genetic algorithm should incorporate some problem-specific features or another type of heuristic. Simulated annealing and taboo search are often claimed to provide good quality solutions to a wide range of combinatorial optimization problems. Our computational results are consistent with this claim.

REFERENCES 1. ‘2. 3.

R.M. Karp, Reducibility among combinatorial problems, In Complexity of Computer Computations, (Edited by R.E. Miller and J.W. Thatcher), pp. 85-103, Plenum Press, New York, (1972). S.L. van de Velde, Duality-based algorithms for scheduling unrelated parallel machines, ORSA Journal on Computing 5, 192-205 (1993). E. Davis and J.M. Jaffe, Algorithms for scheduling tasks on unrelated processors, Journal of the Association for Computing Machinery 28, 721-736 (1981).

52 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19.

20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36.

C. A. GLASS et al. A.M.A. Hariri and C.N. Potts, Heuristics for scheduling unrelated parallel machines, Computers d Operations Research 18, 323-331 (1991). E. Horowitz and S. Sahni, Exact and approximate algorithms for scheduling nonidentical processors, Jounzal of the Association for Computing Machinery 23, 317-327 (1976). O.H. Ibarra and C.E. Kim, Heuristic algorithms for scheduling independent tasks on nonidentical processors, Journal of the Association for Cpmputing Machinery 24, 280-289 (1977). J.K. Lenstra, D.B. Shmoys and E. Tardos, Approximation algorithms for scheduling unrelated parallel mai chines, Mathematical Programming 46, 259-271 (1990). C.N. Potts, Analysis of a linear programming heuristic for scheduling unrelated parallel machines, Discrete Applied Mathematics 10, 155-164 (1985). .I. Spinrad, Worst case analysis of a scheduling algorithm, Operations Research Letters 4, 9-11 (1985). F.A. Ogbu and D.K. Smith, The application of the simulated annealing algorithm to the solution of the nlmlCm,x flowshop problem, Computers & Opemtions Research 17, 243-253 (1990). F.A. Ogbu and D.K. Smith, Simulated annealing for the permutation flowshop problem, Omega 19, 64-67 (1991). I.H. Osman and C.N. Potts, Simulated annealing for permutation flow-shop scheduling, Omega 17, 551-557 (1989). C.R. Reeves, Improving the efficiency of tabu search for machine sequencing problems, Journal of the Operational Research Society 44, 375-382 (1993). E. Taillard, Some efficient heuristic methods for the flow shop sequencing problem, European Jownal of Operational Research 47, 65-74 (1990). M. Widmer and A. Hertz, A new heuristic method for the flow shop sequencing problem, European Journal of Operational Research 41, 186-193 (1989). E.H.L. Aarts, P.J.M. van Laarhoven and N.L.J. Ulder, Local-search-based algorithms for job shop scheduling, CQM-Note 106, Centre for Quantitative Methods, Nederlandse Philips Bedrijven B.V., Eindhoven, (1991). M. Dell’Amico and M. Trubian, Applying tabu-search to the job-shop scheduling problem, Annals of Operations Research 41, 231-252 (1993). U. Dorndorf and E. Pesch, Evolution based learning in a job shop scheduling environment, Computers & Operations Research (1994) (to appear). H. Matsuo, C.J. Suh and R.S. Sullivan, A controlled search simulated annealing method for the general jobshop scheduling problem, Working paper 03-44-88, Department of Management, The University of Texas at Austin, (1988). E. Nowicki and C. Smutnicki, A fast taboo search algorithm for the job shop problem, Preprint 8/93, Instytut Cybernetyki Technicznej, Politechniki Wroclawskiej, Wroclaw, (1993). R.H. Storer, S.D. Wu and R. Vaccari, New search spaces for sequencing problems with application to job shop scheduling, Management Science 38, 1495-1509 (1992). E. Taillard, Parallel taboo search techniques for the job shop scheduling problem, ORSA Journal on Computing (1994) (to appear). P.J.M. van Laarhoven, E.H.L. Aarts and J.K. Lenstra, Job shop scheduling by simulated annealing, Opemtions Research 40, 113-125 (1992). R.W. Eglese, Simulated annealing: A tool for operational research, European Jownal of Operational Research 46, 271-281 (1990). P.J.M. van Laarhoven and E.H.L. Aarts, Simulated Annealing: Theory and Applications, Kluwer, Dordrecht, The Netherlands, (1987). F. Glover, Tabu search, Part I, ORSA Journal on Computing 1, 190-206 (1989). F. Glover, Tabu search, Part II, ORSA Jownal on Computing 2, 4-32 (1990). F. Glover, Tabu search: A tutorial, Interfaces 20, 74-94 (1990). How we used it, Annals of Mathematics and A. Hertz and D. de Werra, The tabu search metaheuristic: Artificial Intelligence 1, 111-121 (1990). D.E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, Reading, MA, (1989). G.E. Liepins and M.R. Hilliard, Genetic algorithms: Foundations and applications, Annals of Operations Research 21, 31-58 (1989). M. Lundy and A. Mees, Convergence of an annealing algorithm, Mathematical Programming 34, 111-124 (1986). D.T. Connolly, An improved annealing scheme for the QAP, European Journal of Operational Research 46, 93-100 (1990). F. Glover, Future path for integer programming and links to artificial intelligence, Computers & Operations Research 13, 533-549 (1986). A. Kolen and E. Pesch, Genetic local search in combinatorial optimization, Discrete Applied Mathematics 48, 273-284 (1994). N.L.J. Ulder, E.H.L. Aarts, H.-J. Bandelt, P.J.M. van Laarhoven and E. Pesch, Genetic local search algorithms for the traveling salesman problem, In Parallel Problem Solving from Nature, Lecture Notes in Computer Science, Volume 496, pp. 109-116, (Edited by H.P. Schwefel and R. Manner), Springer-Verlag, Berlin, (1991).