Page 1. A proposal four-based-view framework to evaluate optimization tools*. Lamia Trabelsi1 ... to develop our new tool for combinatorial optimization problem.
A proposal four-based-view framework to evaluate optimization tools* Lamia Trabelsi1 and Talel Ladhari2
Abstract— This paper presents a new four-view-based framework to survey optimization approaches and the pioneering studies of optimization. The new four-view-based framework allows us to mainly understand optimization. Moreover, every proposed view explores a specific fundamental aspect of optimization. Therefore, classifying, organizing and comparing proposed works is much easier. The proposed four-view-based framework allow us to depict the lacks of the proposed frameworks and tools. Moreover, it allows us to raise many challenges to develop our new tool for combinatorial optimization problem with optimization property.
I. INTRODUCTION Approximate approaches have increasingly caught researchers’ interests in view of solving difficult problems (Continuous and/or Combinatorial). Approximate approaches are easy to implement and they are able to provide acceptable solutions in a reasonable time. Because of the growing interest in approximate methods in the last decade, a particular intention has been given to the development of ready-to-use tools to deal with approximate methods. It is in this context that our research is situated. We would like to propose a new web-based tool specific to Combinatorial Optimization Problem under Permutation Property (COP-PP). To develop the new tool, a an important preliminary review should be done to assess how previous frameworks managed to deal with solving optimization problems in general. In this paper, we will propose a four-view-based framework as a preamble to understanding optimization approaches. This proposed framework will serve to classify and categorize several tools of optimization. This framework is inspired from the frameworks conducted to classify the proposed methods for the development of the service-based system [1], to classify the scenario approaches [2] or to evaluate the goal-based approaches [3]. In section II, the proposed four-view-based framework will be described. Section III reviews several tools and evaluates them according to the proposed view. In section IV, tools’ limitations and difficulties will be described and future research directions will be presented. *This work was not supported by any organization 1 Lamia Trabelsi is with Ecole Suprieure des Sciences Economiques et Commercial de Tunis (ESSECT), 4, Rue Abou Zakaria El Hafsi - 1089 Montfleury - Tunisie. lamia tr2001@yahoo.fr 2 Talal Ladhari is with College of Business, Umm Al-Qura University, Umm Al-Qura, Saudi Arabia and with Ecole Suprieure des Sciences Economiques et Commercial de Tunis (ESSECT), 4, Rue Abou Zakaria El Hafsi - 1089 Montfleury - Tunisie.
talel ladhari2004@yahoo.fr
II. T HE PROPOSED
FOUR - VIEW- BASED FRAMEWORK : AN OVERVIEW
The proposed framework consists of four views (see figure 1). Each view will try to answer one facet of the resolution approach. The proposed four views of the framework are: • The Subject view: relates to ”What are” the resolution approaches in general. This view defines resolution approaches and classifies them into exact and heuristic approaches. • The Usage view: relates to the goals of using the resolution approach. It raises the question ”Why using the resolution approach?”. This view will present the related problems, classify their relative objective functions and categorize problem instances. • The Method view: In particular, this section focuses on the solution, the hybridization of the resolution approach’s techniques and the calibration of the relevant approaches to solve a given problem. • The Tool view: focuses on the tools used to develop the resolution approach. There is a wide range of tools in literature. The main used tools are evaluation tools, approach representation, documentation, and visualization tools.
Fig. 1.
The proposed framework
A set of facets are associated to each view. These facets will serve as a classification tool helping to categorize the optimization approaches according to this view. Every facet will be measured by attributes (metrics). Attributes will be affected by values. Attribute’s values are defined within a domain. A domain may be a predefined type (INTEGER, BOOLEAN, ....), an enumerated type (ENUM x, y, z), or a structured type (SET or TUPLE). A. The Subject view: What is the resolution approach? Resolution approaches are used to solve an optimization problem. The family facet relates to categorizing optimiza-
TABLE I T HE SUBJECT VIEW Subject view: What is the resolution approach? Exact Resolution approach family Approximate
{Branch and bound, mathematical programming} {Constructive heuristic, metaheuristic}
TABLE II T HE USAGE VIEW Usage view: Why using the resolution approach? Classification {Combinatorial; Continuous; Linear} Nature {Deterministic problem; Stochastic problem} Type {mono; multi-objectives} Type {Real-life; Standard; Random}
Problem feature Solve OP
Objective function Benchmark
TABLE III T HE METHOD VIEW Method view: How to achieve the resolution of an optimization problem Solution Approach Creation Approach Evaluation
Solution Representation
Type {Linear; non-linear} Encoding schemes{True; False} {Reject, Penalize objective function, Repair, Preserve}
Solution infeasibility Technics {Create, Hybridize, Assembly} Technics {Calibre, Fitness landscape analysis, Result Comparison, Sensitivity Analysis }
tion methods into two families: exact and approximate approaches (see table I).The exact methods provide an optimal solution. The approximate methods give us, in a reasonable time, a feasible, yet, generally not an optimal solution. Among the basic heuristic methods we usually distinguish between constructive methods and metaheuristic methods. B. The Usage view: Why using a resolution approach This view focuses on the usage of approximation approaches (see table II). Principally, they are used to solve optimization problems. The Solve Problem facet deals with the problem features, the objective function and benchmark attributes. The Problem can be classified depending on decisions variables and constraints type. When decision variables are discrete, the problem is a combinatorial optimization problem. Otherwise, if both the objective functions and constraints are linear, the problem is a linear problem. An optimization problem can be deterministic or a stochastic. The objective function could be mono-objective or multiobjective function. Problem instances could be real-life instances, libraries of standard instances or randomly generated instances. C. The Method view: How to achieve the resolution of an O.P. The Method view deals with the common techniques used by approaches to solve an optimization problem(see table III). The solution facet deals with the solution representation, the encoding schemes and the solution infeasibility attributes. The solution representation attribute deals with the representation of a solution such as a linear or a nonlinear presentation based generally on graph structures such as trees. Solution representation raised the encoding schemes problem between a solution representation used by the optimization approach and a representation solution of an optimization problem. For example, the Random-Key (RK) encoding
proposed by [4] can be used to transform a real permutation to a new integer permutation for flowshop problem. The infeasibility solution attribute deals with techniques used to handle infeasible solutions in optimization methods. The simplest one in the rejection of infeasible solutions. Other approaches are proposed such as the reparation of new generated solution, the development of optimization operators that generate only feasible solutions and the objective function extended by a penalized class to explore a new zone in a discontinuous feasible regions’ search space. The approach creation facet deals with the creation of new optimization approaches. It could be done creating a new optimization approach from scratch, by existing components assembly or by hybridization. The creation of a new optimization approach deals with the creation from scratch of new approaches like constructive heuristics. Components composition deals with the creation of a new approach based on the assembly of existing components (heuristics, operators, metaheuristics). Hybridization of several resolution approaches could enhance the quality of obtained results. Hybridization techniques could be proposed in a low level to enhance a given approach by replacing and/or extending its functionality by other approaches. Hybridization could be proposed in a high level combinations by cooperating over a relatively well defined interface. Readers will be referred to the pioneering frameworks of [5] and [6] which provided good taxonomies for hybridization classification. The approach evaluation facet can be done by : • Result Comparison by an Average Relative Percentage Deviation (ARPD) or by an LB (Lower Bound) approach. • Calibration to obtain the best parameter combination generally by design of experiment (DOE) [7] such as the simple comparative test or analysis of variance ANOVA. Other studies use a heuristic to find the best values of the heuristic parameters like those of [8] and [9] referenced
TABLE IV T HE TOOLS VIEW Tools view: which tools used for the manipulation of resolution approach Black box Approach representation form {Template class, Unified view, Algorithm model , Engine, White box Flow Chart } Design pattern use IsUsed{True; false } Extending supports Extension type {specialization; binary files; plug-ins} Execution order tools facet {sequential (batch); a parallel}
•
•
by [6]. Fitness landscape Analysis deals with analyzing and predicting the efficiency and performance of metaheuristics by studying the properties of their relative landscapes like landscape ruggedness, structure of the attraction basin and the number and frequency of local optima in a landscape. For more details about fitness landscapes and the different metrics, readers are invited to consult the thesis of [10] and the recent paper of [11]. Analysis of optimization approach’s sensitivity which is defined in [12] as ”a systematic procedure used to explore how an optimal solution responds to changes in inputs which are typically either known values which might vary in the future, or parameters whose values are open to question. Thus, the analysis is based on a prior assumption that optimization is center stage, with uncertainty viewed as a potentially disruptive factor. The analysis aims at discovering how sensitive the ’optimal’ solution is to changes in crucial factors.”
D. The Tools view: Which tools to implement the approach? This view deals with tools used to implement optimization approaches, particularly, approximate approaches (see table IV). The approach representation form facet deals with how the approach presents its optimization approach. An optimization approach can be viewed as a Black Box or a White Box which consists of other components. When viewed as a white box, several studies try to provide a unified view to describe the resolution approach particularly in metaheuristics. The propositions are generally formulated as template algorithms and specialized for a specific class of metaheuristics like those proposed by ([13], [14]). [15] proposed a labeled flowchart. [16] presented a unified framework of a metaheuristic based on a trade-off between Memory and new solutions. [17] proposed a generic algorithm model. It consists of a data model containing data, an operator model containing operators and an engine model executing stepwise the operator graph. The design pattern facet indicates if the proposed used the design patterns to define collaboration between between optimization approach’s components. Several Frameworks such as Searcher [18] and MAFRA [19] introduced design patterns to design collaboration between their provided classes. Among the design patterns used, we quote Strategy Pattern, Template Pattern, and State pattern. The extending supports facetindicates how proposed tools extends their work with a new optimization approach. Gen-
erally, extending classes’ frameworks is done by the specialization of the abstract super-class. Others proposed the plugins-based framework like in the work of [20]. The proposals of [21], [22] are based on manipulating files. In [21], a pHubWSP (p-Hub Web services Portal), the integration of new services is made by providing pre-compiled binary codes and their relative XML description. The proposed framework [22] is based on bundles (components)1. The execution order tools facet order deals with the execution order of the algorithm’s components. Execution can be sequential (batch); in a parallel model. A good taxonomy presented for parallel execution and its relative architecture is presented in [5], [6], [13]. E. Frameworks and support tools Evaluation III. P ROPOSED
WORKS IN OPTIMIZATION FIELD :
E VALUATION We propose to classify the several tools dealing with resolution approaches according to the proposed views. Our great interest has been to focus on the study of frameworks. We think that frameworks are the basic backbone for building suitable structures for optimization approaches. As several ready-to-use tools have been proposed in the literature to deal with heuristic methods, we mainly focus on references dealing with frameworks such as HeuristiticLAB [17], HotFrame [23], Templar [24], Paradis-EO [13], Searcher [18] and OSGILIATH [25] and applications like E-OCEA [26], LEKIN [27] or LISA [28] dealing with scheduling problems 2 . For more details about frameworks, interested readers are invited to read the survey of [29] which compares features of several frameworks. A. Capabilities evaluation Frameworks are in general white boxes providing readyto-use classes to implement optimization approaches. Frameworks like Paradis-EO [13]; HeuristiticLAB [17] have been proposed to deal with approximate approaches in general. Other frameworks have addressed specific approximate approaches like metaheuristics-based local search like HotFrame [23]; [24] and Searcher [18] or OSGILIATH [25] for evolutionary algorithm. Applications such as E-OCEA [26], LEIKIN [27] or LISA [28] are generally black box tools and they are proposed to solve scheduling problems by exact algorithms like Branch & Bound or simple rules. Moreover, 1 Bundle is a jar format file contains compiled and packaged java classes and a configuration file. [22] 2 particulary permutation flow shop problem
TABLE V T OOLS EVALUATION IN THE SUBJECT VIEW Frameworks& supports tools HotFrame , Searcher, Templar Paradis-EO, HeuristicLAB, BCOO LISA, LEKIN OSGILIATH e-Ocea p-HubWSP
Subject view: What is the resolution approach? Exact —— —— Basic rules —— Branch and bound, ILoG sub module ——
Approximate Local search based metaheuristics Heuristic; metaheuristic Basic heuristic Evolutionary algorithm Basic heuristic metaheuristic
TABLE VI T OOLS EVALUATION IN THE USAGE VIEW Frameworks & supports tools Paradis-EO, Heuristic LAB e-Ocea, LISA, LEKIN Hotframe, OSGILIATH BCOO, Templar p-hubWS
Usage view: Why using the resolution approach? Problem Objective function Combinatorial, Continuous Mono; multi COP (scheduling problem) Mono Combinatorial, Continuous Mono COP Mono P-Hub Mono
Benchmark Supported by HeurisitcLab Supported ——– ——Supported
TABLE VII T OOLS EVALUATION IN THE METHOD VIEW Frameworks & tools Paradis-EO, Hotframe, OSGILIATH, HeuristicLAB, BCOO, Templar e-Ocea, LISA, LEKIN, p-hubWS
Method view: How to achieve the resolution of an optimization problem? Solution Create Function penalty is supported by Paradis-EO Create, Hybridize, Assembly
a. Solution representation not mentioned b. Solution infeasibility not treated
they proposed a basic heuristics to solve their problems. For p-HUBWebServices, they focus on metaheuristic approaches to solve their relative problems. The table Vsummarizes the tools capabilities on the subject view: In the usage view(see table VI), frameworks in general are dedicated to solve several classes of problems specifically continuous and combinatorial problems like Hotframe; HeuristicLAB and Paradis-EO. More interest has been given to COP by TEMPLAR and BCOO. Support tools are more specific to a given class of problems such as scheduling problems for E-OCEA; Lekin and LISA and respectively pHub for p-HubWebServices. In general, frameworks focused more on the mono-objective function except HeuristicLab and Paradis-EO which have proposed classes to deal with mono and multi-objective functions. The studied support tools and the HeuristicLab framework provide benchmark utilities to generate problem instances to solve problems. In general, Frameworks allow for the development of several representations of solutions (see table VII). In particular, HotFrame; Paradis-EO and HeuristicLab are much richer frameworks for this purpose. Support tools are more specific to deal with their problems at hand (not motioned). However, no frameworks or support tools have tried to provide tools for handling solution-encoding schemes as an adaptation between different solutions encoding or for handling constraints for solution infeasibility, except Paradis-EO which provides classes to handle penalized objective functions. Evaluation of the performance of an optimization approach is omitted by the studied frameworks except E-OCEA which provides tools to find the best algorithm to solve a scheduling problem and
semi-automatic generation tools provided by E-OCEA
Evaluate Fitnesslandscape supported by HeuristicLAB Approaches Comparison supported by E-OCEA
HeuristicLab which provides some fitness landscape metrics. Moreover, calibration and intensification tools have not been supported by the most studied cases. Frameworks proposed classes in order to create new approaches by hybridizing or by assembling components. Hybridization has not been considered by the other studied tools. However, E-Ocea proposed a semi-automatic tool for the generation of new algorithms. Most applications and supports tools view approaches as a black box form (see table VIII). Therefore, extending and adding new methods will be done by file (binary or with an XML file). For OSGiLiath, extension will be done by bundle JAVA. Generally, frameworks present a template Class as a representation form for their approach, except HeuriticLAb which views the optimization approach as an algorithm model and Templar as engine-executing capabilities. Using patterns to define collaboration between components is omitted in several frameworks except searcher which has used template, strategy and state patterns. Parallelization is supported by Paradis-EO, Templar and E-OCEA frameworks. IV. F UTURE RESEARCH DIRECTIONS By studying the capability of tools, two challenges have been raised: the user requirements issues and the creation and/or hybridization of an heuristic approach problem. The following sections will detail the two issues and our future directions. 1) The users’ Requirements issues: When building a tool or a framework for an optimization approach, we believe that the most important question is what the user wants from the proposed tool and how to achieve it. As surveyed
TABLE VIII T OOLS EVALUATION IN THE TOOLS VIEW Frameworks & supports tools Paradis-EO Templar HeuristicLab Hotframe, searcher, BCOO OSGILIATH e-Ocea LISA; LEKIN p-hubWS
Tools view: which tools used for the manipulation of resolution approach? Approach form Pattern Use Extending components TemplateClass False Specialization Engine execution False Specialization Algorithm model False Plug-ins TemplateClass Searcher Specialization &Hotframe Template False Oriented service, Bundle Black box False XML and binary files Black box False Binary file Black box False web service, XML and binary files
Fig. 2.
Execution order Parallel, sequential Parallel, sequential Sequential Sequential sequential, Distributed Parallel, sequential Sequential Sequential
Solution overview
in the literature, the most answered question is ” how to solve an optimization problem and by what means ”. That is the essential module which has been developed in the cited frameworks. Most of the studied tools lack evaluation, calibration and analysis of robustness tools in order to improve optimization approaches. We believe that the proposed tools focus more on how technologies are used and how classes will be developed and omitted at the expense of what a user really wants. Our future research direction consists in developing a new system based on a requirement model. The requirement model will enact users’ requirement in optimization. The requirement model should be generic and evolving. It should be generic to model most users’ requirements. It should be evolving to model new users’ requirements into an easy model. 2) The creation and/or hybridization of an heuristic approach problem: As mentioned before, frameworks such as ParadisEO [13] and HeuristicLab [30] are complex frameworks and often hard to use. They require a deep comprehension to manipulate their available operators. All the previous mentioned frameworks aim at proposing classes and they do not provide a guideline for young researchers to effectively exploit components in the existing frameworks. Then, the construction of new algorithms or the hybridization
of existing objects are generally difficult. On the other hand, few researchers tried to provide a general view of metaheuristics. Most of them, except [16], describe a specific group of metaheuristics. Our intention as further research will focus on proposing a new unified model for approximate methods. In fact, describing metaheuristic capabilities as a unified view allows for an easy use of the framework. All components will be categorized and well defined. The selection of components from available heuristics to create metaheuristics will be easy. Viewing the optimization approach as white box is crucial to deal with composition and hybridization. In this way, to construct a composition, attention will be given to collaboration between operators. A new collaboration structure will be defined for operators. 3) An Overview of our tools for COP-PP: As shown in figure 2, a requirement model will be identified to express researchers’ requirements in optimization field. Every enacted requirement will be implemented as a set of functionalities in our tool. In order to guide the researcher in effectively exploring framework classes and creating new approaches, a unified model will be proposed. The new unified model will describe metaheuristic concepts in a way every metaheuristic will be viewed as an instance of the proposed model. In addition, our new tool will be guided by this model to
classify, describe and extract operator components. V. C ONCLUSION Several resolution methods have been proposed in the literature to deal with difficult and complex problems. In this paper, we have reviewed several frameworks and proposed tools to implement optimization approaches and to solve optimization problems. In particular, we focused on frameworks. To evaluate capability of tools, we proposed a four-view framework consisting of the following views: Subject; Usage; Methods and Tools views. Our study shows that frameworks are often hard to use or are specific to one unique optimization approach family. On the other hand, both tools and frameworks generally focus on how to solve problems and omit other facets like evaluation of the obtained results or a sensitivity analysis. For this raison, in order to develop a decision support tools for combinatorial problems with permutation problems, our future research direction is to propose a new requirement model to enact user requirements and a new unified view for heuristic approaches. R EFERENCES [1] R. Kaabi, “Une approche mthodologique pour la modlisation intentionnelle des services et leur oprationnalisation,” Ph.D. dissertation, Universit Paris I, Panthon, Sorbonne, 2007. [2] C. Rolland, C. B. Achour, C. Cauvet, M. Jarke, P. Haumer, K. Pohl, and et al., “A proposal for a scenario classification framework,” 1996. [3] E. Kavakli and P. Loucopoulos, “Goal modeling in requirements engineering: Analysis and critique of current methods,” in Information Modeling Methods and Methodologies, J. Krogstie, T. A. Halpin, and K. Siau, Eds. Idea Group, 2005, pp. 102–124. [4] I.-H. Kuo, S.-J. Horng, T.-W. Kao, T.-L. Lin, C.-L. Lee, T. Terano, and Y. Pan, “An efficient flow-shop scheduling algorithm based on a hybrid particle swarm optimization model,” Expert Systems with Applications, vol. 36, no. 3, Part 2, pp. 7027 – 7032, 2009. [5] G. R. Raidl, “A unified view on hybrid metaheuristics.” in Hybrid Metaheuristics, ser. Lecture Notes in Computer Science, F. Almeida, M. J. B. Aguilera, C. Blum, J. M. Moreno-Vega, M. P. Prez, A. Roli, and M. Sampels, Eds., vol. 4030. Springer, 2006, pp. 1–12. [6] E.-G. Talbi, “A taxonomy of hybrid metaheuristics,” Journal of Heuristics, vol. 8, no. 5, pp. 541–564, Sept. 2002. [7] D. Montegromy, Design and Analysis of Experiments, 5th ed. John Wiley and Sons Inc, 2001. [8] F. Abbatista, N. Abbatista, and L. Caponetti, “Evolutionary an Cooperative Agents Model for Optimization,” in Proceedings of the second IEEE International Conference on Evolutionary Computation (ICEC), IEEE, Ed., vol. 2. IEEE Press, 1995, pp. 668–671. [9] K. Shahookar and P. Mazumder, “A genetic approach to standard cell placement using meta-genetic parameter optimization.” IEEE Trans. on CAD of Integrated Circuits and Systems, vol. 9, no. 5, pp. 500– 511, 1990. [10] T. Jones, “Evolutionary Algorithms, Fitness Landscapes and Search,” Ph.D. dissertation, University of New Mexico, 1995. [Online]. Available: http://jones.tc/research/phd.html [11] E. Pitzer and M. Affenzeller, “A comprehensive survey on fitness landscape analysis,” in Recent Advances in Intelligent Engineering Systems, ser. Studies in Computational Intelligence. Springer, 2012, pp. 161–191. [12] J. Rosenhead, “Robustness analysis,” 2010. [13] E.-G. Talbi, Metaheuristics : from design to implementation. Wiley, 2009. [14] E. D. Taillard, L. M. Gambardella, M. Gendreau, and J.-Y. Potvin, “Adaptive memory programming: A unified view of metaheuristics,” European Journal of Operational Research, vol. 135, no. 1, pp. 1–16, 2001. [15] M. Gendreau and J.-Y. Potvin, “Metaheuristics in combinatorial optimization,” Annals OR, vol. 140, no. 1, pp. 189–213, 2005. [16] J. Branke, M. Stein, and H. Schmeck, “A unified framework for metaheuristics,” in GECCO, 2003, pp. 1568–1569.
[17] S. Wagner, A. Beham, G. Kronberger, M. Kommenda, E. Pitzer, M. Kofler, S. Vonolfen, S. Winkler, V. Dorfer, and M. Affenzeller, “Heuristiclab 3.3: A unified approach to metaheuristic optimization,” Proceedings of the VII Congreso Espanol sobre Metaheursticas, Algoritmos Evolutivos y Bioinspirados (MAEB 2010), Valencia, Spain., 2010. [18] S. Andreatta A., Carvalho and R. C., “A framework for local search heuristics for combinatorial optimization problems,” in Optimization Software Class Libraries, ser. Operations Research/Computer Science Interfaces Series, S. Vo and D. L. Woodruff, Eds. Springer US, 2002, vol. 18, pp. 59–79. [19] N. Krasnogor and J. Smith, “Mafra: A java memetic algorithms framework,” in Workshops Proceedings of the 2000 International Genetic and Evolutionary Computation Conference (GECCO2000), A. Wu, Ed., 2000. [Online]. Available: http://www.cs.nott.ac.uk/ nxk/PAPERS/womaMafra.pdf [20] S. Wagner, S. Winkler, E. Pitzer, G. Kronberger, A. Beham, R. Braune, and M. Affenzeller, “Benefits of plugin-based heuristic optimization software systems,” in Proceedings of the 11th international conference on Computer aided systems theory, ser. EUROCAST’07. Berlin, Heidelberg: Springer-Verlag, 2007, pp. 747–754. [21] M. P. P´erez, F. Almeida, J. M. M. Vega, V. B. P´erez, and A. Santos, “Exposing metaheuristics as web services in distributed systems using opencf,” Electronic Notes in Discrete Mathematics, vol. 36, pp. 151– 158, 2010. [22] P. Garc´ıa-S´anchez, J. Gonz´alez, P. A. Castillo, J. J. M. Guerv´os, A. M. Mora, J. L. J. Laredo, and M. G. Arenas, “A distributed service oriented framework for metaheuristics using a public standard,” in NICSO, 2010, pp. 211–222. [23] A. Fink and S. Vo, “Hotframe: A heuristic optimization framework,” in Optimization Software Class Libraries, ser. Operations Research/Computer Science Interfaces Series, S. Vo and D. L. Woodruff, Eds. Springer US, 2002, vol. 18, pp. 81–154. [24] M. S. Jones, G. P. McKeown, and V. J. Rayward-Smith, “Distribution, Cooperation, And Hybridization For Combinatorial Optimization,” in Optimization Software Class Libraries. Kluwer, 2002, pp. 26–57. [25] P. Garc´ıa-S´anchez, J. Gonz´alez, P. A. Castillo, M. G. Arenas, and J. J. M. Guerv´os, “Service oriented evolutionary algorithms,” Soft Comput., vol. 17, no. 6, pp. 1059–1075, 2013. ´ [26] V. Tkindt, J.-C. Billaut, J.-L. Bouquard, C. Lente, P. Martineau, E. Nron, C. Proust, and C. Tacquard, “The e-ocea project: towards an internet decision system for scheduling problems,” Decision Support Systems, vol. 40, pp. 329–337, 2005. [27] N. Asadathorn, “Scheduling of assembly type of manufacturing systems: Algorithms and systems developments,” Ph.D. dissertation, Department of Industrial and Manufacturing Engineering, New Jersey Institute of Technology, Newark (USA),, 1997. [28] H. Braesel and S. N., “Lisa–fit for cooperative development,,” Sixth Workshop on Models and Algorithms for Planning and Scheduling Problems (MAPSP’03), pp. 107 – 108, 2003. [29] J. A. Parejo, A. R. Corts, S. Lozano, and P. Fernandez, “Metaheuristic optimization frameworks: a survey and benchmarking.” Soft Comput., vol. 16, no. 3, pp. 527–561, 2012. [30] S. Wagner, G. Kronberger, A. Beham, S. M. Winkler, and M. Affenzeller, “Model driven rapid prototyping of heuristic optimization algorithms.” in EUROCAST’09, 2009, pp. 729–736.
