Bio-inspired Combinatorial Optimization: Notes on Reactive and

Bio-inspired Combinatorial Optimization: Notes on Reactive and Proactive Interaction Carlos Cotta and Antonio J. Fern´ andez-Leiva Dept. Lenguajes y Ciencias de la Computaci´ on, ETSI Inform´ atica, Campus de Teatinos, Universidad de M´ alaga, 29071 M´ alaga – Spain {ccottap,afdez}@lcc.uma.es

Abstract. Evolutionary combinatorial optimization (ECO) is a branch of evolutionary computing (EC) focused on finding optimal values for combinatorial problems. Algorithms ranging in this category require that the user defines, before the process of evolution, the fitness measure (i.e., the evaluation function) that will be used to guide the evolution of candidate solutions. However, there are many problems that possess aesthetical or psychological features and as a consequence fitness evaluation functions are difficult, or even impossible, to formulate mathematically. Interactive evolutionary computation (IEC) has recently been proposed as a part of EC to cope with this problem and its classical version basically consists of incorporating human user evaluation during the evolutionary procedure. This is however not the only way that the user can influence the evolution in IEC and currently one can find that IEC has been been successfully deployed on a number of hard combinatorial optimization problems. This work examines the application of IEC to these problems. We describe the basic fundament of IEC, present some guidelines to the design of interactive evolutionary algorithms (IEAs) to handle combinatorial optimization problems, and discuss the two main models over which IEC is constructed, namely reactive and proactive searchbased schemas. An overview of the existing literature on the topic is also provided. We conclude with some reflections on the lessons learned, and the future directions that research might take in this area.

1

Introduction

Combinatorial optimization is ubiquitous and comprises an enormous range of practical applications. Problems arising in this area are typically hard to solve –due to both the size of the associated search spaces and the intrinsic complexity of efficiently traversing them in order to find the optimal solution– and thus the use of powerful solving methodologies is required. Among these, bio-inspired algorithms emerge as cutting-edge tools, due to their search power. Bio-inspired algorithms (including, evolutionary computation methods, swarm intelligence, and metaheuristics) have been shown to be adequate tools for combinatorial optimization in many different areas, and one of their most important characteristics, particularly inherent to evolutionary computation (EC), is their flexibility J. Cabestany, I. Rojas, and G. Joya (Eds.): IWANN 2011, Part II, LNCS 6692, pp. 348–355, 2011. c Springer-Verlag Berlin Heidelberg 2011 

Bio-inspired Combinatorial Optimization

349

to be adjusted to different problem domains, that is to say, in certain form these methods represent generic procedures that can be used with certain adjustments guided by the user to handle a plethora of combinatorial optimization problems. However, in spite of their proved efficacy as optimization methods, recently the need to exploit problem knowledge in order to obtain solutions of better quality as well as accelerate the optimization process has become evident [1–3]. In this sense, the programmer (i.e., the user) has usually incorporated specific information about the problem to guide the search; this has been done for instance via the hybridization with other techniques [4, 5], by designing specific genetic operators or by defining intelligent representations with inherent information in them. Other forms of adding problem-knowledge to an EC algorithm are possible though. However, there still exists one main complication that lies precisely in the difficulty to characterize the subjective interest through a certain mathematical expression or algorithm that can be optimized. This difficulty is generally common to those problems in which the search has to be conducted (directly or indirectly, completely or partially) in a psychological space. Within the framework of the metaheuristics –and more specifically of evolutionary computing– the solution that has been proposed is the so-called interactive evolutionary computing (IEC). In a broad sense, IEC is an approach based on the optimization of a certain target system, using evolutionary computing and interacting with a human user. Traditionally, this interaction was based on the subjective assessment of the solutions generated by the algorithm; in this line see for instance the seminal work of Dawkins [6] as well as different applications in artistic fields (see, e.g., the proceedings of EvoMUSART), industrial design, processing of audiovisual information, data mining or robotics, among other fields [7]. The common nexus of classical IEC is the existence of a reactive search-based mechanism in which the user provides some feedback to the demands of the running evolutionary algorithm. Even though they represent a powerful advance for the optimization of problems requiring some kind of subjective evaluation, classical IEC methods also have an important limitation: the fatigue of the human user that is produced by the continuous feedback that the subjacent EC technique demands to the user. Advanced IEC techniques smooth this drawback by employing proactive algorithms that are able to guess the further user interactions and thus reduce the requirement of user interventions. The aim of this paper is to provide a general overview on user-centric evolutionary computation, and more specifically on both interactive EC and proactive search when they are applied to combinatorial optimization problems.

2

Interactive Evolutionary Computation

Generally speaking, interactive evolutionary computation (also termed indistinctly as user-centric evolutionary computation) is an optimization paradigm that incorporates the user intervention during the search process in an evolutionary algorithm; more specifically, IEC basically consists of a computational

350

C. Cotta and A.J. Fern´ andez

model that promotes the communication between a human user and an automated evolutionary algorithm (EA). In a classic view of IEC, the user usually acts as the fitness evaluation function in standard evolutionary algorithms (EA) and is continuously required by the EA to provide the assessment of candidate solutions; in other words, the EA is responsible for evolving the population of individuals in the evolutionary optimization process where as the user evaluates the outputs generated by the EA. As shown further on in this paper, more modern models of IEC propose different ways to attain the collaboration between the human user and the EA. IEAs have already been implemented in all the standard types of EC as for instance in genetic programming [8, 9], genetic algorithms [10], evolution strategies [11], and evolutionary programming [12] just to name a few. Interactivity has also been added to a number of cooperatives models (e.g., [13–15]).

3

Design Principles for Effective IEAs

It is important to underline a fundamental fact in relation to the use of IEC and the context over which it is applied: the applications of IEC are often conducted on domains in which there is no exact (or reasonably good) way to assess the quality of the solutions, and this is precisely the reason why it is necessary to involve the user in the loop (the applications of aesthetic nature are appropriate examples, although they are not unique). In any case, no general approach for the design of effective interactive evolutionary algorithms exists in a well-defined sense, and hence this design phase must be addressed from an intuitive point of view as well. However, in order to help the reader to understand the mechanisms of IEAs, let us consider in what follows the main forms that exist in the literature to design IEAs. In general the user can influence the optimization process in several ways that basically can be summarized as follows: – Allowing the user to evaluate –even if just sporadically– the candidate solutions that are generated by the evolutionary algorithm during the search process. – Allowing the user to order (or classify) the set of candidates according to some (possibly psychological) criteria provided specifically by them. For instance the user might select solutions to be maintained in further generations (in case of elitist evolutionary algorithms). – Allowing the user to reformulate the objective function. For instance the user interaction might consist of refining this function by adjusting the weighing of specific terms. – Allowing the user to modify the set of constraints attached originally to the definition of the problem. More specifically, the user might provide new (or remove existing) constraints dynamically; this means that the user can impose additional constraints in the form of hard constraints (i.e., their satisfaction is mandatory) or soft constraints (i.e., they can be violated but in this case the violation generates an extra penalty value to be added to the fitness value of the solution).

Bio-inspired Combinatorial Optimization

351

– Allowing the user to change dynamically some parameters of the algorithm; for instance, a soft change might be to assign new values for genetic operators application probability; also, the user might determine the choice of the genetic operators (playing thus the role of a hyper-heuristics selector that works inside the subjacent EA mechanism). – Allowing the user to incorporate additional mechanisms for improving the optimization process. This is for instance the case when the user decides dynamically to add a restarting mechanism during the optimization process in case of stagnation (or premature convergence) or add some local-search methods with the aim of obtaining a memetic version of running evolutionary algorithm. Moreover, the user might decide wether the phases of restarting/local-improvement should be performed only on a reduced subset of the population (e.g., the user can play the role of a selector that decides which solutions should undergo local improvement what in other words means the decision of using partial Lamarckism i.e., not using local search on every new solution computed, but only on some of them). From a global perspective, the basic idea is to let the user affect the search dynamics with the objective of driving (resp. deviating) the search towards (resp. from) specific regions of the solution space. There are alternative ways to reach this objective. For instance, [16] proposes using techniques of dimensionality reduction to project the population of the EA to a bidimensional plane that is displayed to the user and over which the user selects the most promising candidates. It is also worthwhile to mention the work conducted in the area of multi-objective IEC [17, 18] in which the aim is to direct the exploration toward particular regions of the Pareto front. Again this kind of participation only represents one of the manifold forms that exist to fix search priorities.

4

IEC Limitations: The Fatigue of the Human User

The classical IEC as described previously can be catalogued as a reactive interaction procedure in which the user operates under the demand of feedback from the subjacent evolutionary computation technique. In this context, as already mentioned, one of the major concerns of classic IEC approaches is the fatigue that they cause in the human user; this fatigue is the result of demanding continuously feedback from the subjacent evolutionary algorithm. This section is devoted to discussing different mechanisms already described in the literature to mitigate this fatigue. For instance, this can be done by reducing the accuracy of the judgements required of them [19], via the use of micropopopulations [20], or via the use of prediction methods of quality. These methods can be used for example, to make a pre-selection of promising solutions, presenting to the user (or to a set of users if a cooperative model is considered) just a reduced number of solutions to evaluate. Typically, this pre-selection can be attained from metrics that measure the distance between the tentative solutions and those for which a subjective assessment exists.

352

C. Cotta and A.J. Fern´ andez

Another more sophisticated approach to reduce the human user fatigue in IEAs is by replacing the reactive collaboration by a proactive reaction in which the intervention of the user is optional and the algorithm runs autonomously [11]. In this context IEC is usually identified as proactive user-centric evolutionary search/optimization. This is the case when the IEC model employs computational learning techniques to predict the adequacy of the solutions still to be evaluated. If the prediction model of this adequacy is sufficiently adjusted, then alternating phases between optimization via IEC and optimization via the predictive model can be conducted [21]. In general, an approach of this type has the difficulty of finding a measure of the adequate distance that captures the subjective preferences of the human user. Another potential difficulty is the inherent noise that often exists in the human response (due for instance to the fatigue of user, to the evolution of their subjective perception, or to an adjustment of its response to the characteristics of the solutions in the current generation). Additionally, [22] approaches the problem of the fatigue in IEC and proposes an interactive genetic algorithm with an individual fitness not assigned by a human; the basic idea is to automatically compute the fitness value of randomly selected individuals from the population by recording the time employed to mark them as valid or invalid candidates and then performing a transformation from the time space to the fitness space. The proposal was applied to a problem of fashion design. [13] describes a mixed-initiative interaction technique for an interactive GA in which a simulated expert created by using a machine learning model (i.e., in particular via fuzzy logic modeling) can share the workload of interaction with the human expert; in addition the human user preferences are constantly been learnt. This collaborative framework also allows the system to observe the learning behaviors of both the human and simulated expert, while utilizing their knowledge for search purposes. In [23] an interactive genetic algorithm applied over a nurse scheduling problem does not only generate an offspring that is further evaluated by the user but also includes a mechanism to learn the criteria of the user evaluation; this knowledge is then applied to construct schedules under the learn criteria. The best schedules are displayed for the user with the aim of providing them with a decision mechanism to choose which parts of the schedules to adopt, or to improve.

5

Conclusions

This paper tries to provide a general overview on interactive evolutionary computation (IEC) identifying its main components, advantages and disadvantages. One of the main conclusions that can be drawn from the literature on IEC is that it constitutes a versatile and effective optimization paradigm to tackle combinatorial optimization problems whose candidate solutions have to be evaluated in a psychological space (for instance because it is difficult to translate the fitness evaluation function into a mathematical formulation). Indeed, IEC represents

Bio-inspired Combinatorial Optimization

353

one of the main paradigms to cope with this kind of problem, and provides an appropriate framework to seamlessly integrate human knowledge into evolutionary computation techniques. Traditionally IEC was based on a reactive optimization model in which the subjacent evolutionary algorithm demands the intervention of the user by requiring from them some kind of feedback; in the most classic models, the user just acts as a mere fitness evaluator. We have though discussed other models in which the user interacts with the running EA in a number of ways, as for instance adding constraints to bias the search, reformulating the objective function by changing the weights of its parameters, identifying the best solution candidates, or redesigning (parts of) the chromosome representation, among others. We have also discussed the main drawback of IEC, that is to say, the fatigue that affects the human user as a consequence of the continuous requirement of feedback. A model that mitigates this problem consists of replacing the reactive answer of the user by a proactive approach in which the subjacent running algorithm usually infers the user’s answer before the feedback demand. A number of proactive schemas, that have in common the incorporation of a specific learning mechanism, have also been discussed throughout the paper. The flexibility of the proactive approach makes it helpful in cases in which the user wants to obtain an added value, but makes it also useful in complex optimization problems with perfectly well defined evaluation functions; in these cases the inherent skills of perception and information processing of the human user can help to both lead the search towards suboptimal regions of the search space and avoid the stagnation (or even premature convergence) of the algorithm in specific parts of this space. User-centric optimization should be considered as a natural mechanism to cope with combinatorial optimization problems in which subjective evaluation of candidates is required. The interested reader is also referred to [7] and [24] that present surveys for interactive evolutionary computation and human-guided search respectively. Acknowledgements. This work is supported by project NEMESIS (TIN-200805941) of the Spanish Ministerio de Ciencia e Innovaci´on, and project TIC-6083 of Junta de Andaluc´ıa.

References 1. Hart, W.E., Belew, R.K.: Optimizing an arbitrary function is hard for the genetic algorithm. In: Belew, R.K., Booker, L.B. (eds.) Proceedings of the 4th International Conference on Genetic Algorithms, pp. 190–195. Morgan Kaufmann, San Mateo (1991) 2. Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation 1(1), 67–82 (1997) 3. Bonissone, P., Subbu, R., Eklund, N., Kiehl, T.: Evolutionary Algorithms + Domain Knowledge = Real-World Evolutionary Computation. IEEE Transactions on Evolutionary Computation 10(3), 256–280 (2006)

354

C. Cotta and A.J. Fern´ andez

4. Puchinger, J., Raidl, G.R.: Combining metaheuristics and exact algorithms in com´ binatorial optimization: A survey and classification. In: Mira, J., Alvarez, J.R. (eds.) IWINAC 2005. LNCS, vol. 3562, pp. 41–53. Springer, Heidelberg (2005) 5. Moscato, P., Cotta, C.: A modern introduction to memetic algorithms. In: Gendreau, M., Potvin, J.-Y. (eds.) Handbook of Metaheuristics, 2nd edn. International Series in Operations Research and Management Science, vol. 146, pp. 141–183. Springer, Heidelberg (2010) 6. Dawkins, R.: The BlindWatchmaker. Longman, Essex (1986) 7. Takagi, H.: Interactive evolutionary computation: Fusion of the capabilities of EC optimization and human evaluation. Proceedings of the IEEE (9), 1275–1296 (2001) 8. Lim, S., Kim, K.-M., Hong, J.-H., Cho, S.-B.: Interactive genetic programming for the sentence generation of dialogue-based travel planning system. In: 7th AsiaPacific Conference on Complex Systems, Cairns, Australia, Asia-Pacific Workshops on Genetic Programming, pp. 6–10 (2004) 9. Lim, S., Cho, S.-B.: Language generation for conversational agent by evolution of plan trees with genetic programming. In: Torra, V., Narukawa, Y., Miyamoto, S. (eds.) MDAI 2005. LNCS (LNAI), vol. 3558, pp. 305–315. Springer, Heidelberg (2005) 10. Kosorukoff, A.: Human-based genetic algorithm. In: 2001 IEEE International Conference on Systems, Man, and Cybernetics, pp. 3464–3469. IEEE Press, Tucson (2001) 11. Breukelaar, R., Emmerich, M.T.M., B¨ ack, T.: On interactive evolution strategies. In: Rothlauf, F., Branke, J., Cagnoni, S., Costa, E., Cotta, C., Drechsler, R., Lutton, E., Machado, P., Moore, J.H., Romero, J., Smith, G.D., Squillero, G., Takagi, H. (eds.) EvoWorkshops 2006. LNCS, vol. 3907, pp. 530–541. Springer, Heidelberg (2006) 12. Kubota, N., Nojima, Y., Sulistijono, I.A., Kojima, F.: Interactive trajectory generation using evolutionary programming for a partner robot. In: 12th IEEE International Workshop on Robot and Human Interactive Communication (ROMAN 2003), Millbrae, California, USA, pp. 335–340 (2003) 13. Babbar, M., Minsker, B.: A collaborative interactive genetic algorithm framework for mixed-initiative interaction with human and simulated experts: A case study in long-term groundwater monitoring design. In: World Environmental and Water Resources Congress (2006) 14. Quiroz, J.C., Banerjee, A., Louis, S.J.: Igap: interactive genetic algorithm peer to peer. In: Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation, GECCO 2008, pp. 1719–1720. ACM, New York (2008) 15. Quiroz, J.C., Louis, S.J., Banerjee, A., Dascalu, S.M.: Towards creative design using collaborative interactive genetic algorithms. In: IEEE Congress on Evolutionary Computation (CEC 2009), pp. 1849–1856. IEEE, Singapore (2009) 16. Takagi, H.: Active user intervention in an ec search. In: 5th Joint Conf. Information Sciences (JCIS2000), Atlantic City, NJ, pp. 995–998 (2000) 17. Deb, K., Chaudhuri, S.: I-mode: An interactive multi-objective optimization and decision-making using evolutionary methods. KanGal report 2007003, Kanpur Genetic Algorithms Laboratory (2007) 18. Deb, K., Kumar, A.: Interactive evolutionary multi-objective optimization and decision-making using reference direction method. KanGal report 2007001, Kanpur Genetic Algorithms Laboratory (2007) 19. Ohsaki, M., Takagi, H., Ohya, K.: An input method using discrete fitness values for interactive ga. Journal of Intelligent and Fuzzy Systems 6(1), 131–145 (1998)

Bio-inspired Combinatorial Optimization

355

20. S´ aez, Y., Vi˜ nuela, P.I., Segovia, J., Castro, J.C.H.: Reference chromosome to overcome user fatigue in IEC. New Generation Comput. 23(2) (2005) 21. Dozier, G.: Evolving robot behavior via interactive evolutionary computation: From real-world to simulation. In: 16th ACM Symp. Applied Computing (SAC2001), Las Vegas, NV, pp. 340–344 (2001) 22. Gong, D., Yao, X., Yuan, J.: Interactive genetic algorithms with individual fitness not assigned by human. Journal of Universal Computer Science 15(13), 2446–2462 (2009), http://www.jucs.org/jucs_15_13/interactive_genetic_algorithms_with 23. Inoue, T., Furuhashi, T., Fujii, M., Maeda, H., Takaba, M.: Development of nurse scheduling support system using interactive ea. In: IEEE Int. Conf. Systems, Man, and Cybernetics, vol. 5, pp. 533–537 (1999) 24. Klau, G., Lesh, N., Marks, J., Mitzenmacher, M.: Human-guided search. Journal of Heuristics 16, 289–310 (2010)