Enhancing Genetic Programming based Hyper ...

Enhancing Genetic Programming based Hyper-Heuristics for Dynamic Multi-objective Job Shop Scheduling Problems Su Nguyen

Mengjie Zhang

Kay Chen Tan

Department of Industrial and Systems Engineering International University - VNUHCM Ho Chi Minh City Email: [email protected]

Evolutionary Computation Research Group Victoria University of Wellington PO Box 600, Wellington, New Zealand Email: [email protected]

Department of Electrical and Computer Engineering National University of Singapore Singapore, 117576 Email: [email protected]

Abstract—Genetic programming based hyper-heuristics have been an suitable approach to designing powerful dispatching rules for dynamic job shop scheduling. However, most current methods only focus on a single objective while practical problems almost always involve multiple conflicting objectives. Some efforts have been made to design non-dominated dispatching rules but using genetic programming to deal with multiple objectives is still very challenging because of the large search space and the stochastic characteristics of job shops. This paper investigates different strategies to utilise computational budgets when evolving dispatching rules with genetic programming. The results suggest that using local search heuristics can enhance the quality of evolved dispatching rules. Moreover, the results show that there are some differences in evolving rules for single objectives and for multiple objectives and that it is difficult to efficiently estimate the Pareto dominance of rules.

I. I NTRODUCTION Scheduling in job shop environments has been a wellknown research problem in the literature. Job shop scheduling (JSS) aims to determine the sequence in which jobs will be process at each machine or work centre such that objectives of interest can be optimized (e.g., minimise the mean tardiness or maximum flowtime). JSS problems are usually classified as static and dynamic. This paper mainly focuses on dynamic JSS (DJSS) in which jobs can arrive at random over time and the attributes (processing time, routes through the machines, due date) of jobs are not known in advance. This problem is particularly interesting as it reflects the dynamic changes and uncertainties in the real job shops. Dispatching rules are a particularly simple scheduling heuristic, which can be used to prioritize jobs in the queues. Whenever a machine is available and there are jobs waiting to be processed on that machine, dispatching rules compute a priority for each available job in the queue. Basically the priorities of jobs are a function of some job attributes (e.g. its processing time), and system attributes (e.g. number of jobs in the system). Jobs in the queue with the highest priority will be processed next. Dispatching rules are normally quite easy c 978-1-4799-7492-4/15/$31.00 2015 IEEE

to understand and implement. Compared to other optimisation methods for scheduling, dispatching rules are very efficient. Moreover, because each scheduling decision is made at the latest possible moment (i.e., when machine become idle), dispatching rules adapt quickly to unexpected changes, which makes them particularly suited for stochastic and dynamic scheduling problems. Their flexibility to incorporate domain knowledge and expertise also makes dispatching rules a useful tools in practical situations. Designing a good dispatching rule, especially for DJSS, is not easy as many aspects need to be taken into account such as job routings, process times, due dates, etc. Although a large number of studies have been conducted and many dispatching rules are proposed [1], [2], finding a suitable rule for a particular environment is still difficult. With the advances in computing power, several machine learning methods [3], [4], [5], [6], [7], [8], [9] have been proposed to discover new effective dispatching rules for DJSS problems. Within these methods, Genetic Programming (GP) [10], [11] is the most popular because it is a straightforward method to evolve dispatching rules in the form of a priority function. GP has recently emerged as a promising approach to dealing with scheduling problem [12], [13], [14], [5], especially for designing dispatching rules in dynamic manufacturing environments [15], [9]. GP methods in this case are usually referred to as GP based hyper-heuristics (GPHH). One reason that makes GP a suitable approach to designing dispatching rules is that priority functions can easily be represented as GP individuals (e.g. tree-based representation). Moreover, the evolved rules can be (partially) interpreted, which is very helpful for production operators and managers. Previous works have focused on representations of dispatching rules [5], [13], [16] and evolutionary search mechanisms to help GP evolve more effective rules and deal with practical requirements [15]. In most of these works, GP is used to evolve dispatching rules by training them on a set of training instances/scenarios and the best rule evolved by GP is tested. Multiple objectives are commonly found in real systems

but studies on this issue are still limited. Most of studies on GPHH for DJSS are restricted to one particular objective. Tay and Ho [9] is the first work that focuses on multi-objective DJSS (MO-DJSS) problems. In their study, they converted the multi-objective problem to a single objective problem by optimising a linearly weighted sum of all the objectives. However, the rule evolved by this method is not competitive as pointed out by [3]. In their study, Hildebrandt et al. [3] showed that rules evolved by Tay and Ho [9] are only slightly better than the earliest release date (ERD) rule and quite far away from the performance of the shortest processing time (SPT) rule with mean flowtime as the objective. This suggested that a linear combination of objectives may not be a suitable approach to deal with MO-DJSS. Nguyen et al. [17], [15] examined some multi-objective GPHH (MOGPHHs) and showed that it is possible to simultaneously deal with multiple conflicting objectives and generate competitive as well as robust dispatching rules. Although the initial results with MOGPHH are promising, evolving rules when taking into account multiple objectives is challenging. The goal of this paper is to enhance the performance of MOGPHHs by (1) effectively utilising the computational budget, i.e. the number of simulation replications used to evolve rules (more details will be provided in Section III), and (2) apply local search to refine the rules evolved with GP. The first point has been preliminarily investigated by Hildebrandt et al. [3] for single objective DJSS; however, it has never been considered for MOGPHHs. Meanwhile, the second point has not been investigated in the literature of GPHH for scheduling. Follows are some particular research objectives for this study: 1) Evaluate the effectiveness of different strategies to utilise computational budgets and their influence on the performance of MOGPHHs, 2) Develop and evaluate the effectiveness of local search heuristics in MOGPHHs, and 3) Compare the performance and analyse the behaviour of proposed MOGPHH methods. In the next section, we provide the background of JSS problems and GPHH methods. Section III describes the proposed multi-objective GPHH methods and the job shop simulation models used for training and testing. The experimental results, the comparison and analysis of evolved dispatching rules are presented in Section V. Section VI gives some conclusions from this research and directions for future studies. II. BACKGROUND This section provides an overview of dispatching rules in DJSS and GPHH for heuristic generation with a special focus on scheduling problems. A. Scheduling in DJSS Scheduling has been a popular research topic in operations research and computer science for the last few decades because of its computational complexity and practical applications. For static JSS where jobs and their information are fixed

and known in advance, optimisation is the major research direction. Because JSS is a NP-hard problem [18], exact methods are too expensive to apply in practice. Heuristics and meta-heuristics have been developed to help find good solutions in reasonable computational times. A comprehensive review of these methods is given in Potts and Strusevich [19] and Ouelhadj and Petrovic [20]. Although there have been many meta-heuristics (e.g., evolutionary computation) proposed for scheduling problems, not much has been used for dynamic environments. Dispatching rules are the most popular approach for JSS problems in dynamic environments. There have been hundreds of dispatching rules proposed in the literature to deal with different types of manufacturing environments. Dispatching rules are usually classified based on the information used to make scheduling decisions (e.g. static, dynamic, local, global) [2] and how these pieces of information are combined [21]. The effectiveness of a dispatching rule depends on how it copes with the dynamic changes of the shops and the ability to take into account different factors that can affect the considered objective to be optimised. The comparisons of different dispatching rules have been continuously done in many studies [22], [23], [24], [25], [3]. The comparison was usually performed under different characteristics of the shop because it is well-known that the characteristics of the shop can significantly influence the performance of the dispatching rules. Different objectives were also considered in these studies as they are the natural requirements in real world applications. B. GPHH for scheduling Recently, GP has become popular in the field of hyperheuristics and it is known as genetic programming based hyper-heuristics (GPHH) [26]. Because GP is able to represent and evolve complex programs or rules, it naturally becomes an excellent candidate for heuristic generation. GPHH has also been applied to evolve dispatching rules for scheduling problems [13], [14], [27]. Several GPHH methods have also been proposed for JSS problems. Atlan et al. [28] applied GP for JSS problems. However, the focus of their paper is on finding the solution for a particular problem instance. Miyashita [7] examined three potential multi-agent models to evolve dispatching rules in multiple machine environments: (1) a homogeneous model where all machines share the same dispatching rule, (2) a distinct agent model where each machine employs its own evolved rule, and (3) a mixed agent model where two rules can be selected to prioritise jobs depending on whether the machine is a bottleneck. The experiments showed that the distinct agent model provided better results in the training stage compared to the homogeneous model but had some over-fitting problems. The mixed agent model was the most robust in all the experiments. However, the use of the mixed agent model depends on the prior-knowledge about the bottleneck machine, which can change in dynamic situations. To handle this issue, Jakobovic and Budin [5] proposed a new GP method called GP-3 to provide some adaptive behaviour for the evolved rules. In their method, GP

is used to evolve three components of the rules including a decision tree and two dispatching rules for bottleneck and non-bottleneck machines. The purpose of the decision tree is to identify whether a considered machine is a bottleneck and decide which one of the two evolved rules should be applied. The experiments showed that this method can provide better rules than a simple GP method. However, it is noted that the superior performance of GP-3 will depend on the bottleneck machines. If the load levels between machines in the shops are rather similar (existence of multiple bottleneck machines), the information/output from the decision tree in GP-3 may not be very helpful. Nguyen et al. [16] investigated different representations of dispatching rules with GP. The experiments showed that a mixed representation based on decision-tree like representation and arithmetic representations provided the best the results. However, they mainly examined their rules on static scheduling instances. Tay and Ho [9] performed a study on using GP for multiobjective JSS problems. In their method, three objectives are linearly combined (with the same weights) into an aggregate objective, which is used as the fitness function in the GP method. The experiments showed that the evolved rules are quite competitive as compared to simple rules but still have trouble dominating the best rule for each single objective. In another study, Hildebrandt et al. [3] explained that the poor performance of the rules evolved by Tay and Ho [9] is caused by the use of a linear combination of different objectives and the fact that the randomly generated instances cannot effectively represent the situations that happen in a long term simulation. For that reason, Hildebrandt et al. [3] evolved dispatching rules by training them on different simulation scenarios and only minimised the mean flow time. Some aspects of the simulation models were also discussed in their study. The experimental results showed that the evolved rules were quite complicated but effective as compared to other existing rules. Moreover, these evolved rules are also robust when tested with another environment. However, their work did not consider how to handle multiple conflicting objectives. Nguyen et al. [15] proposed a cooperative coevolution GPHH for multi-objective dynamic JSS problems. In that work, the due dates of new jobs are assumed to be assigned internally and two scheduling rules (dispatching rule and due date assignment rule) are simultaneously considered in order to develop effective scheduling policies. While the representation of the dispatching rules is similar to those in other GP methods, the operation-based representation [29] is used to represent the due date assignment rules. The results showed that the evolved scheduling policies can outperform scheduling policies from different combinations of existing dispatching rules and duedate assignment rules in different simulation scenarios. In another study, Beham et al. [30] utilise parallel technologies to evolve dispatching rules for a flexible job shop with a large terminal and function sets. They develop three new GP methods based on island models and SASEGASA [31] in which rules are evolved in multiple subpopulations. The results show that the SASEGASA method can cope

better with the states of exception in the simulation than island based methods. Pickardt and Juergen [32] proposed a two-stage approach to evolving dispatching rule sets for semiconductor manufacturing. In the first stage, GP is used to evolve general dispatching rules. The best obtained dispatching rule is combined with a list of benchmark dispatching rules to generate a set of candidate rules. In the second stage, a µ + λ evolutionary algorithm (EA) [32] is used to select the most suitable dispatching rule in the set of candidate rules for each work centre in the shop. The experiments in this paper compare the performance of the two-stage hyper-heuristics with the pure GP and EA hyper-heuristics. The results show that the three hyper-heuristics outperformed benchmark dispatching rules and the two-stage hyper-heuristics produced significantly better performance than the other two hyperheuristics. A more detailed review of GP for other scheduling problems can be found in Nguyen et al. [16]. III. MOGPHH S FOR MO-DJSS In this section, we first introduce the simulation model used for training and testing with MOGPHHs. Then, we describe the MOGPHHs studied in this paper and key aspects to be investigated. 1) Simulation models: For dynamic scheduling problems where arrivals of jobs are unknown in advance, discrete event simulation is usually applied to assess the quality of evolved rules. Although it is computationally more expensive than using training instances, it is considered a more appropriate approach to training heuristics for dynamic scheduling problems. As pointed out by Hildebrandt et al. [3], training instances normally have overload issues at the beginning and cannot really characterise the typical situations occurring in long-term simulations of manufacturing systems with dynamic/continuous job arrivals. All experiments in this paper are based on the simulation model of a symmetrical job shop which has been used in previous studies on dispatching rules [24], [15]. Here are the simulation configurations: • 10 machines • Each job has 2 to 14 operations (re-entry is allowed) • Processing times follow discrete uniform distribution U [1, 99] • Job arrivals follow Poisson process • Due date = current time + allowance factor × total processing time (allowance factor of 4 is used in our experiments) • Utilisation of the shop is 95% • No machines break-down; preemption is not allowed In each simulation replication, we start with an empty shop and the interval from the beginning of the simulation until the arrival of the 500th job is considered as the warm-up time and the statistics from the next completed 2000 jobs [25] will be used to calculate performance measures. Five scheduling performance measures examined in our experiments are (1) mean flowtime, (2) maximum flowtime, (3) percentage of tardy jobs, (4) mean tardiness, (5) Maximum Tardiness. The

TABLE I P ERFORMANCE MEASURES FOR DJSS P

j∈C

Prepare training data

Start

fj

Mean Flowtime

F=

Maximum Flowtime

Fmax = maxj∈C {fj }

Percentage of Tardy Jobs

%T = 100 ×

Generate the initial population

|C|

Training Datasets/ Simulation Models

|T| |C| j∈T (Cj −dj ) |T|

P

Mean Tardiness

T=

Maximum Tardiness

Tmax = maxj∈T {Cj − dj }

Start new generation

Evaluate Individual/ Heuristic

calculation of these five performance measures (objectives) are shown in Table I. Although this simulation model is relatively simple, it still reflects key issues of real manufacturing systems such as dynamic changes and complex job flows. This section only considers a shop with high utilisation (95%) and tight due date (allowance factor of 4) because scheduling in this scenario is more challenging, and therefore easier to demonstrate the usefulness of GP. In order to reliably measure the effectiveness of evolved rules, a large number of simulation replications are usually needed (e.g. 30 to 50 simulation replications are usually needed to accurately estimate the performance of rules in the scenario described here). However, using simulation to evaluate the fitness of the evolved rules is also the most timeconsuming part in GP for JSS. Therefore, only a small number of replications are usually used for fitness evaluations during the training process. A. Genetic programming based hyper-heuristic Figure 1 presents the basic framework for evolving scheduling heuristics with GP. The procedure in this figure starts by preparing training data (datasets or simulation models) for the considered scheduling problems. The initial population of heuristics is then randomly generated. In each generation of the evolutionary process, each heuristic is used to provide scheduling decisions for different instances/scenarios in the training datasets or simulation models. The obtained scheduling performance is used to calculate the fitness of each heuristic. The fitness values obtained by heuristics in the population decide the chance of each heuristic to survive and reproduce (with genetic operations) in the next generation. More details about the scheme to select solutions for the next generation are provided in Section III-C. The same routine is applied until the termination condition is met. B. Representation of dispatching rules This paper uses the traditional GP tree to represent dispatching rules, the same as previous studies [14], [3], [15]. Table II shows the terminal set and function set used by GP to construct priority functions. The attributes in the tables have been extensively used in the existing dispatching rules as well as GP for JSS. For the function set, four basic arithmetic operators and min/max are used to construct composite dispatching rules (the protected division is similar to normal

Calculate Fitness Function Go to the next individual

Generate new population via genetic operations

All individuals are evaluated ?

No

Yes No End

Fig. 1.

Yes

Stopping condition met?

Genetic programming based hyper-heuristic (GPHH). TABLE II T ERMINAL AND FUNCTION SETS OF GP

Symbol

Description

rJ RJ RO RT PT DD RM SL WT #

job release time (arrival time) operation ready time number of remaining operation within the job. work remaining of the job operation processing time due date of the job machine ready time slack of the job = DD − (t + RT) is the current waiting time of the job = max(0, t − RJ) Random number from 0 to 1

NPT WINQ APR

processing time of the next operation work in the next queue average operation processing time of jobs in the queue

Function set

+,−,×, %, min, max

∗

t is the time when the sequencing decision is made.

division but returns a value of 1 when division by 0 is attempted). For genetic operations, we employ the standard subtree crossover and subtree mutation. The subtree crossover creates new individuals for the next generation by randomly recombining subtrees from two selected parents. Meanwhile, the subtree mutation is performed by selecting a node of a chosen individual and replacing the subtree rooted by that node with a newly randomly-generated subtree. C. Search algorithm The HaD-MOEA algorithm [33] is applied here to explore the Pareto front of non-dominated dispatching rules regarding the five performance measures discussed in Section III-1. The

Algorithm 1: MOGPHH algorithm load training replications S ← {S1 , S2 , . . . , ST } randomly initialise the population P ← {X1 , X2 , . . . , Xpopsize } P e ← {} and generation ← 0 while gen ≤ maxGeneration do foreach Xi ∈ P do Xi .objectives ← apply Xi to a set of replications Sgen ⊂ S end calculate the Harmonic [33] and the ranks S distance e for individuals in P P S P e ← select(P P e ) P ← apply crossover, mutation to P e gen ← gen + 1 end return P e

Xi .objectives is the average values of the five performance measures obtained by applying Xi to a set of training replications Sgen ⊂ S. HaD-MOEA can be considered as an extension of NSGA-II [34] and it was shown to work well on the problems with many objectives.

D. Strategies to utilize computational budgets One problem with GPHH is the high computational costs because a lot of simulation replications need to be performed and GP trees (individuals) have to be evaluated many times (when a rule is tested). As suggested by Hildebrandt et al. [3], only one replication (corresponding to one random seed) for each fitness evaluation is needed; however, it is more effecitive to change the replication (use a different random seed for simulation) when moving to a new generation. This strategy has been shown to be beneficial to prevent GP from overfitting to certain situations (replications). To see whether changing replications every generation will also benefit MOGPHHs, we will investigate two strategies: •

•

Fixed set of training replications across all generations: the same set of replications will be used in each evaluation of rules to calculate Xi .objectives. We refer this method as MOGPHHf. Changing the replication used in each generation: each generation will employ a specific replication to evaluate Xi .objectives of evolved rules. As one replication is not accurate enough to assess the effectiveness of rules, a full evaluation (with a large number of replications) will be applied at the end of a generation to see what rules are truly non-dominated and should be included in an external archive. The archive and the population will be used to generate rules for the next generation. In this case, the final rules obtained from the algorithm are stored in the external archive. We refer this method as MOGPHHc.

E. Local search heuristic As the search space of GP is large, MOGPHHs may have trouble searching for competitive rules. To enhance the exploitation ability of MOGPHHs, we propose a simple local search heuristic to refine the rules evolved by the MOGPHH in each generation. The local search can be described as follows: 1) Select a rule X in the population using binary selection in HaD-MOEA [33] 2) Generate a neighbor rule X 0 by mutating X 3) If X 0 dominates X , X ← X 0 4) Return to step 2 if the number of generated X 0 is fewer than maxstep The method that employs this local search heuristic and fixed set of training replications are referred to as MOGPHHfls. IV. PARAMETER SETTINGS The crossover rate and mutation rate used in the three methods are 85% and 15%, respectively. The maximum depth of GP trees is 8. A population size of 200 is used in this study. The termination conditions for the proposed MOGPHH methods are 100,000 simulation replications. The initial GP population is created using the ramped-half-and-half method [10]. For MOGPHHf, the number of fixed replications is 5. In MOGPHHc, the number of replications for full evaluations is 50 and 16 rules will be selected from the population for full evaluations in a generation. These parameters are selected such that MOGPHHf and MOGPHHc will stop after 100 generations (or 100,000 replications). The size of external archive in MOGPHHc is also 200. For MOGPHHfls, the number of rules selected for refinement are 20 and maxstep in the local search heuristic is 10. For testing, each rule obtained by the three methods is evaluated by using 50 independent replications. V. R ESULTS The results shown in this section are from 30 independent runs of each MOGPHH method. A. Performance metrics In order to measure the effectiveness of the three MOGPHH methods, we will consider three aspects. The first one is the actual number of non-dominated (ANND) rules. This metric is important in MOGPHHs because the non-dominated rules based on training objectives are not necessarily non-dominated when they are tested with a large number of simulation replications (after evolution). Good MOGPHHs should prevent the overfitting issue and results in high ANND. The second metric is the non-dominated ratio (NR). NR is an n-ary Pareto dominance metric [35] to compare the quality of solution sets from various algorithms. Representing the Pareto fronts evolved by n algorithms by P1e , P1e ,..., P1e , this metric measures the non-dominated ratio of solutions in the Pareto front obtained by one algorithm compared to those

140 120

170

100 60

80

Non−dominated Ratio

160 150

20

40

140

Number of non−dominated programs (testing)

130

MOGPHHf

MOGPHHfls

(a) Actual number dominated rules Fig. 2.

MOGPHHc

of

MOGPHHf

non-

MOGPHHfls

MOGPHHc

(b) Non-dominated ratio

Comparison of the three proposed MOGPHHs.

obtained by the other algorithms. Mathematically, the N Ri for the method i is formulated as: N Ri =

|P ∗ ∩ Pie | P∗

(1)

where P ∗ is the approximated Pareto front obtained by extracting the non-dominated rules from the set of all rules evolved by all methods. The method with higher N Ri is better. Finally, we are also interested in how MOGPHH optimises each particular objective by comparing the best rules for the corresponding performance measure (e.g., mean flowtime). B. Actual number of non-dominated rules The ANND for each run of MOGPHHs is shown in Fig. 2 (a). The results show that MOGPHHfls and MOGPHHc have higher ANND as compared to MOGPHHf. It is easy to understand as MOGPHHf evolved rules based on a small set of training replications which cannot reflect accurately the non-dominance relationship between rules. MOGPHHfls is more competitive as it evolves better rules as compared to MOGPHHf. MOGPHHc also has higher ANND because the rules in the external archive are more accurately evaluated. From the detailed results, we see that most rules evolved by MOGPHHc are truly non-dominated. The reason for the small gap between MOGPHHc and MOGPHHfls is because the rules in the external archive at the end of the run can be fewer than 200 while MOGPHHf and MOGPHHfls have 200 non-dominated rules in most of their runs. These observations indicate that both strategies to management computational budget and the local search heuristic significantly influence the number of non-dominated rules found by MOGPHHs. C. Non-dominated ratio The results in Fig. 2 (b) clearly show that MOGPHHf and MOGPHHfls are significantly better than MOGPHHc. This indicates that using fixed subset of training replications are more competitive than using a different replication in each

generation regardless of the inaccuracy in estimating the nondominance relationship between rules. The results here are different from those with a single objective [3]. While evolving rules with different replications through the evolution tends to improve the effectiveness and robustness of rules, it is not the case for multiple objectives. One possible reason is that the non-dominance relationship is harder to estimate with just a single replication and it is also hard to select rules for genetic operators if non-dominance between rules changes in every generation. A more effective approach to select non-dominated rules is needed in this case if we want to take advantages of changing replication strategy. Meanwhile, the local search heuristic can effectively enhance the quality of evolved rules. As MOGPHH has to explore different potential rules, its exploitation is relatively weak; therefore, using local search can improve the exploitation ability of MOGPHH. Utilising the local search heuristic in this case can help MOGPHH quickly identify dominated rules in the population and focus better on more potential rules. However, it should be noted that selecting suitable rules to apply local search is not a trivial task and MOGPHHfls will have difficulty with the traditional binary selection approach because the search space of MOGPHHs is very large. A smarter approach for selection will be needed to further improve the performance of MOGPHHfls. D. Single objective The best rules for a particular performance measure is recorded and their results are shown in Fig. 3. Again, MOGPHHf and MOGPHHfls are more competitive than MOGPHHc for most performance measures. MOGPHHf seems to be less robust as compared to other methods (as it has larger variances). Because we deal with five objectives simultaneously, this problem is expected. MOGPHHfls with the support of the local search heuristics can find the extreme rules better than other methods. This again shows the importance of local search. E. Further discussion Dealing with multiple objectives is hard. Although some methods [9], [15] have been proposed to deal with this issue, there are still many limitations. For multiple objectives, finding the Pareto front is very useful. For example, Fig. 4 shows the Pareto front obtained by the three MOGPHH methods. From this figure, the decision makers can identify some good tradeoff rules, i.e. the rules that result in low percentage of tardy jobs without deteriorating maximum flowtime or maximum tardiness. Multiple objectives also influence the complexity of MOGPHHs. Evolving a set of non-dominated rules is surely more tricky than a single objective and different issues needed to be handled. Although there are many evolutionary multiobjective optimisation methods proposed in the literature, they mainly focus on find instance solutions rather than heuristics. For that reason, there is no guarantee that these methods will work with GPHHs and new specialised MOGPHHs are still needed.

4000 5900

14

2100

5800

3800

800

2080

3600

Max Tardiness

5500

3400

700

10

750

Mean Tardiness

% Tardy

5700 5600

Max Flowtime

12

2060 2040

650

5400

8

3200

Mean Flowtime

2020 2000 1980 1960

MOGPHHf

MOGPHHfls

MOGPHHc

MOGPHHf

(a) Mean Flowtime

MOGPHHfls

MOGPHHc

MOGPHHf

(b) Max Flowtime Fig. 3.

MOGPHHfls

MOGPHHc

MOGPHHf

(c) % Tardy

MOGPHHfls

MOGPHHc

MOGPHHf

(d) Mean Tardiness

MOGPHHfls

MOGPHHc

(e) Max Tardiness

Comparison of the three proposed MOGPHH.

60000

1000

2000 3000

10000

60000

2000

MF

60

10000

MaxF

2000

20

PT

2000

3000

20

Fig. 4.

60

10000

60000

MaxT

10000

1000

MT

60000

Non-dominated rules evolved for DJSS.

Another problem with evolving heuristics for multiobjective DJSS is overfitting. The chance of creating overfitted heuristics through crossover and mutation is actually quite high when Pareto dominance is used as the criteria for individual selection in MOGPHHs. A rule in the Pareto front accidentally generated may not be dominated by other good non-dominated rules, especially when many objectives are considered, even though the rule can contain many useless components. This issue also occurs in single objective GPHH; however, it is

less severe because only one single objective is considered and later generation can find more compact rules [36] to replace the unnecessarily lengthy rules. If these issues can be handled, the proposed MOGPHH can be applied effectively to DJSS but also for other hard computational problems [37], [38]. VI. C ONCLUSIONS This paper investigates two aspects that can enhance the effectiveness of MOGPHHs: (1) the selection of training replications through the evolution process of MOGPHHs and

(2) local search. The experimental results show that changing replications is not as useful as it is in the case with a single objective. The reason is that the non-dominance relationship cannot be captured easily with a single simulation replication. Meanwhile, the proposed local search heuristic can effectively enhance the quality of evolved rules. Rules evolved with MOGPHH and the local search heuristic are more competitive than other MOGPHH methods in terms of Pareto dominance and each particular objective. In the future studies, we will focus on designing more effective and efficient local search heuristics. Other advanced approaches to utilize computational budgets will be explored to further enhance the quality of MOGPHHs. R EFERENCES [1] S. S. Panwalkar and W. Iskander, “A survey of scheduling rules,” Operations Research, vol. 25, pp. 45–61, 1977. [2] M. L. Pinedo, Scheduling: Theory, Algorithms, and Systems, 3rd ed. Springer, 2008. [3] T. Hildebrandt, J. Heger, and B. Scholz-Reiter, “Towards improved dispatching rules for complex shop floor scenarios: a genetic programming approach,” in GECCO ’10: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, New York, USA, 2010, pp. 257–264. [4] H. Ingimundardottir and T. Runarsson, “Supervised learning linear priority dispatch rules for job-shop scheduling,” in Learning and Intelligent OptimizatioN (LION 5), 2011, pp. 263–277. [5] D. Jakobovic and L. Budin, “Dynamic scheduling with genetic programming,” in EuroGP’06: Proceedings of the 9th European Conference on Genetic Programming, 2006, pp. 73–84. [6] X. Li and S. Olafsson, “Discovering dispatching rules using data mining,” Journal of Scheduling, vol. 8, pp. 515–527, 2005. [7] K. Miyashita, “Job-shop scheduling with GP,” in GECCO’00: Proceedings of the Genetic and Evolutionary Computation Conference, 2000, pp. 505–512. [8] L. Nie, X. Shao, L. Gao, and W. Li, “Evolving scheduling rules with gene expression programming for dynamic single-machine scheduling problems,” The International Journal of Advanced Manufacturing Technology, vol. 50, pp. 729–747, 2010. [9] J. C. Tay and N. B. Ho, “Evolving dispatching rules using genetic programming for solving multi-objective flexible job-shop problems,” Computer & Industrial Engineering, vol. 54, pp. 453–473, 2008. [10] J. R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, 1992. [11] W. Banzhaf, P. Nordin, R. Keller, and F. Francone, Genetic Programming: An Introduction. Morgan Kaufmann, San Francisco, 1998. [12] C. Dimopoulos and A. M. S. Zalzala, “Investigating the use of genetic programming for a classic one-machine scheduling problem,” Advances in Engineering Software, vol. 32, no. 6, pp. 489–498, 2001. [13] C. D. Geiger, R. Uzsoy, and H. Aytug, “Rapid modeling and discovery of priority dispatching rules: An autonomous learning approach,” Journal of Heuristics, vol. 9, no. 1, pp. 7–34, 2006. [14] C. D. Geiger and R. Uzsoy, “Learning effective dispatching rules for batch processor scheduling,” International Journal of Production Research, vol. 46, pp. 1431–1454, 2008. [15] S. Nguyen, M. Zhang, M. Johnston, and K. C. Tan, “Automatic design of scheduling policies for dynamic multi-objective job shop scheduling via cooperative coevolution genetic programming,” IEEE Transactions on Evolutionary Computation, vol. 18, no. 2, pp. 193–208, 2014. [16] ——, “A computational study of representations in genetic programming to evolve dispatching rules for the job shop scheduling problem,” IEEE Transactions on Evolutionary Computation, vol. 17, no. 5, pp. 621–639, 2013. [17] ——, Dynamic multi-objective job shop scheduling: a genetic programming approach, ser. Studies in Computational Intelligence. Berlin and Heidelberg: Springer, 2013, vol. 505, pp. 251–282. [18] M. R. Garey, D. S. Johnson, and R. Sethi, “The complexity of flowshop and jobshop scheduling,” Mathematics of Operations Research, vol. 1, no. 2, pp. pp. 117–129, 1976.

[19] C. N. Potts and V. A. Strusevich, “Fifty years of scheduling: a survey of milestones,” Journal of the Operational Research Society, vol. 60, no. Supplement 1, pp. 41–68, 2009. [Online]. Available: http://www.palgravejournals.com/jors/journal/v60/ns1/abs/jors20092a.html [20] D. Ouelhadj and S. Petrovic, “A survey of dynamic scheduling in manufacturing systems,” Journal of Scheduling, vol. 12, no. 4, pp. 417– 431, 2009. [21] A. Jones and L. C. Rabelo, “Survey of job shop scheduling techniques,” NISTIR, National Institute of Standards and Technology, Gaithersburg, USA, Tech. Rep., 1998. [22] V. Sels, N. Gheysen, and M. Vanhoucke, “A comparison of priority rules for the job shop scheduling problem under different flow timeand tardiness-related objective functions,” International Journal of Production Research, 2011. [23] C. Rajendran and O. Holthaus, “A comparative study of dispatching rules in dynamic flowshops and jobshops,” European Journal of Operational Research, vol. 116, no. 1, pp. 156–170, 1999. [24] M. S. Jayamohan and C. Rajendran, “New dispatching rules for shop scheduling: a step forward,” International Journal of Production Research, vol. 38, pp. 563–586, 2000. [25] O. Holthaus and C. Rajendran, “Efficient jobshop dispatching rules: Further developments,” Production Planning & Control, vol. 11, no. 2, pp. 171–178, 2000. [26] E. K. Burke, M. R. Hyde, G. Kendall, G. Ochoa, E. Ozcan, and J. R. Woodward, “Exploring hyper-heuristic methodologies with genetic programming,” in Computational Intelligence, ser. Intelligent Systems Reference Library, C. Mumford and L. Jain, Eds. Springer Berlin Heidelberg, 2009, vol. 1, pp. 177–201. [27] W. J. Yin, M. Liu, and C. Wu, “Learning single-machine scheduling heuristics subject to machine breakdowns with genetic programming,” in CEC’03: IEEE Congress on Evolutionary Computation, 2003, pp. 1050–1055. [28] L. Atlan, J. Bonnet, and M. Naillon, “Learning distributed reactive strategies by genetic programming for the general job shop problem,” in Proceedings of the 7th Annual Florida Artificial Intelligence Research Symposium, 1994. [29] S. Nguyen, M. Zhang, M. Johnston, and K. C. Tan, “Genetic programming for evolving reusable due-date assignment models in job shop environments,” Evolutionary Computation, (accepted). [30] A. Beham, S. Winkler, S. Wagner, and M. Affenzeller, “A genetic programming approach to solve scheduling problems with parallel simulation,” in IEEE International Symposium on Parallel and Distributed Processing, 2008. IPDPS 2008, 2008, pp. 1–5. [31] M. Affenzeller and S. Wagner, “Sasegasa: A new generic parallel evolutionary algorithm for achieving highest quality results,” Journal of Heuristics, vol. 10, no. 3, pp. 243–267, May 2004. [32] C. W. Pickardt, T. Hildebrandt, J. Branke, J. Heger, and B. Scholz-Reiter, “Evolutionary generation of dispatching rule sets for complex dynamic scheduling problems,” International Journal of Production Economics, vol. 145, no. 1, pp. 67–77, 2013. [33] Z. Wang, K. Tang, and X. Yao, “Multi-objective approaches to optimal testing resource allocation in modular software systems,” IEEE Transactions on Reliability, vol. 59, no. 3, pp. 563–575, 2010. [34] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist multiobjective genetic algorithm: NSGA-II,” IEEE Transactions on Evolutionary Computation, vol. 6, no. 2, pp. 182 –197, 2002. [35] C.-K. Goh and K. Chen Tan, “A competitive-cooperative coevolutionary paradigm for dynamic multiobjective optimization,” IEEE Transactions on Evolutionary Computation, vol. 13, no. 1, pp. 103–127, Feb 2009. [36] R. Keller and R. Poli, “Cost-benefit investigation of a geneticprogramming hyperheuristic,” in Proceedings of the 8th International Conference on Artifcial Evolution, 2007, pp. 13–24. [37] C. Cheong, K. Tan, D. Liu, and C. Lin, “Multi-objective and prioritized berth allocation incontainer ports,” Annals of Operations Research, vol. 180, no. 1, pp. 63–103, 2010. [38] H. Tang, K. Tan, and Z. Yi, “A columnar competitive model for solving combinatorial optimization problems,” Neural Networks, IEEE Transactions on, vol. 15, no. 6, pp. 1568–1574, Nov 2004.