Information and Software Technology 43 (2001) 905±907
www.elsevier.com/locate/infsof
Software engineering using metaheuristic innovative algorithms: workshop report Mark Harman a,*, Bryan F. Jones b,1 a
Department of Information Systems and Computing, Brunel University, Uxbridge, Middlesex UB8 3PH, UK b School of Computing, University of Glamorgan, Pontypridd CF37 1DL, UK
Abstract This paper reports on the ®rst International Workshop on Software Engineering using Metaheuristic Innovative Algorithms, which was held in Toronto on the 14th of May 2001 as a part of the IEEE International Conference on Software Engineering. q 2001 Elsevier Science B.V. All rights reserved. Keywords: Software engineering; Metaheuristic innovative algorithms; Genetic algorithms
1. Introduction The ®rst International Workshop on Software Engineering using Metaheuristic Innovative Algorithms aimed to broaden awareness within the software engineering community of metaheuristic algorithms and their application to Software Engineering problems [4]. It brought together researchers and practitioners in Software Engineering and metaheuristics, building upon the embryonic community, which currently occupies the intersection of the two ®elds. Prior to the workshop, initial work on metaheuristics for Software Engineering had been conducted in the areas of testing [5±7,9,11,12] and cost estimation [2,3]. The workshop aimed to facilitate the extension of metaheuristic techniques into other areas of Software Engineering research and practice. The workshop included two sessions: a special session on the application of metaheuristics to software testing and a general session on the application of metaheuristics to the wider Software Engineering ®eld. The workshop attracted 26 participants from ®ve countries with representation from academia and industry and with work presented on theory, practice and evaluation of search-based Software Engineering. There were two keynote talks, and ®ve paper presentations, and a lively discussion of the issues. Papers presented at the workshop * Corresponding author. Tel.: 144-1895-274-000; fax: 144-1895-251686. E-mail addresses:
[email protected] (M. Harman),
[email protected] (B.F. Jones). 1 Tel.: 144-0-1443-482730; fax: 144-0-1443-482715.
are available on the workshop website at http://www.brunel.ac.uk/,csstmmh2/seminal2001/. This report brie¯y summarises (in Section 2) the talks presented and (in Section 3) the discussion of issues which took place. Details of future workshops and conferences on the application of metaheuristic search techniques to Software Engineering problems can be found on the SEMINAL 2 website at http://www.discbrunel.org.uk/seminal/seminalhome.html.
2. Papers presented 2.1. Keynote: an overview of genetic algorithms This talk was given by Darrell Whitley, Computer Science Department, Colorado State University, USA. Dr Whitley's talk provided an overview of Genetic Algorithms, indicating many useful and important theoretical results which software engineers seeking to exploit these techniques need to be aware of. He highlighted the issues involved in representation and selection operators, indicating that results which show binary encoding a poor second 2 SEMINAL is the EPSRC-funded Network on Software Engineering using metaheuristic innovative algorithms (GR/M87083).
0950-5849/01/$ - see front matter q 2001 Elsevier Science B.V. All rights reserved. PII: S 0950-584 9(01)00196-3
906
M. Harman, B.F. Jones / Information and Software Technology 43 (2001) 905±907
to real-number encoding can be misleading; grey coding should always be used in place of pure binary encoding. Dr Whitley produced an invited paper for the special issue of the Journal of Information and Software technology on Software Engineering using Metaheuristic Algorithms [14], providing a printed account of some of the insights into the theory and practical of genetic algorithms presented by Dr Whitley at the workshop.
design. Mr Hart pointed out that this approach might help to ameliorate the growing programming skills gap. Mr Hart accepted that this approach would not be suitable for all programs (most notably, it would be inappropriate for safety critical systems). However, he cited a number of examples where the approach could yield additional insight into the perceived requirements captured by the ®tness function.
2.2. Keynote: an overview on evolutionary testing
2.5. A prediction system for dynamic optimisation-based execution time analysis
This talk was given by Joachim Wegener, DaimlerChrysler, Germany. Dr Wegener provided a detailed and thorough overview of software testing using genetic algorithms and simulated annealing. The talk covered structural testing (using coverage analysis), mutation testing, worst-and best-case execution time testing and issues of boundary value analysis and partition testing. The talk also included a survey of empirical results and experience gained over seven years of the use of evolutionary testing techniques at DaimlerChrysler. The slides of Dr Wegener's talk are available on the workshop website. 2.3. A skeleton for the tabu search metaheuristic with applications to problems in software engineering This talk was given by Maria Blesa, Department de Llenguatges i Sistemes InformaÁtics, Universitat PoliteÁcnicade Catalunya, Spain. The talk presented a template tabu search engine, which can be instantiated to address problems in Software Engineering (and others). Ms Blesa illustrated the use of the template with the instantiation to handle project management scheduling and provided a brief overview of the tabu search technique. Most work on Search-Based Software Engineering focuses on the use of Genetic Algorithms, so the talk also contributed to this emergent research ®eld by presenting an alternative form of search. The provision of tools is a valuable resource for the community. Maria's work is part of the MALLBA project. The system proposed by Ms Blesa is available at the MALLBA project website at http://www.lsi.upc.es/,mallba. 2.4. Seizing the opportunity: how hardware advances and genetic programming may ease the software skills shortage? This talk was given by John Hart, School of Design, Engineering and Computing, Bourne mouth University, UK. Mr Hart presented an argument that software development be moved `upstream' from coding to design. Rather than coding programs, programmers could code ®tness functions, which could be used to code programs. Programming could thus become a question of ®tness function
This talk was given by Gerdi Gross, Fraunhofer Institute for Experimental Software Engineering, Germany. The talk addressed the problem of de®ning and evaluating metrics which determine how successful evolutionary testing is likely to be. The particular emphasis of the work was on test data generation for worst and best case execution time. The metrics provided a guide to the testability of the program under consideration. Dr Gross reported experience with the use of these metrics, suggesting that metrics needed to be combined to achieve a good correlation between predicated testability and actual testability. The de®nition of metrics to predict aspects of evolutionary test data selection is a sign of the growing maturity of this sub-area of Search-Based Software Engineering. Initial work on the feasibility of applying evolutionary search algorithms to test data selection was successful. This work is now being developed to address issues such as predicting and improving the testability of software using these techniques. 2.6. Improving heuristic software analysis with semantic information This talk was given by Christoph Michael, Cigital, USA.The talk described an approach used by Cigital to identify in¯uencing input domains. The technique, known as abduction, identi®es the conditions under which a portion of code will be executed. This work is aimed at helping to guide the choice of test data through partial analysis of program semantics. It thus supports evolutionary testing by narrowing the search space.The approach represents the combination of existing (by partial) analytic algorithms with metaheuristic search. 2.7. A genetic algorithm ®tness function for mutation testing This talk was given by Leonardo Bottaci, Department of Computer Science, University of Hull, UK.The talk presented a ®tness function for use by a genetic algorithm to search for test data for mutation testing. The ®tness function incorporates the necessity and suf®ciency conditions for a test to kill a mutant in addition to the usual reachability measure. For programs written in a procedural style language, the system under development generates mutants
M. Harman, B.F. Jones / Information and Software Technology 43 (2001) 905±907
that are instrumented for reachability, necessity and suf®ciency. 3. Issues raised in discussion The idea of genetic programming as a possible way forward was discussed at some length. Software engineers, and in particular those with extensive experience of software maintenance issues were concerned that the code created would present signi®cant comprehension and maintenance challenges. There was a discussion of issues in evolutionary testing. Joachim Wegener described the approach adopted by DaimlerChrysler, indicating that results always outperformed random testing on every execution and for every test object. The dif®culties associated with continuous data were discussed. This typically involves a larger search space, but initial results from the DaimlerChrysler evolutionary testing system were encouraging. Dr Wegener indicated that a large population was found to be helpful in improving the performance of evolutionary testing. Typically, the system used population sizes of 300±500 individuals. Dr Harman suggested the use of slicing [8,13] to reduce search space sizes and the use of imperative-style transformation [1,10] to make software more amenable to the de®nition of suitable ®tness functions. Remaining areas of dif®culty for evolutionary testing were de®ned and explored, including the problems associated with predicates over arbitrary strings (and in general arrays), for which ®tness functions are harder to de®ne. There are also problems where code contains implicit logical dependencies and constraints, which are not immediately obvious. The idea here was to combine evolutionary techniques with traditional source code analysis techniques, enabling the former to be informed by the latter, thereby avoiding searches within sub-spaces which could be predetermined to be unproductive. Generic issues involved in applying metaheuristics to Software Engineering problems were considered. The central issue here is that the results produced are not `explained' other than by the ®tness function. This suggests that the ®tness function will play a central role in any exploitation of search-based techniques within Software Engineering. The workshop ended with a general discussion of the advantages of the application of metaheuristic search to problems in Software Engineering. In particular, the robust nature of these techniques and their ability to cope with fuzzy, partially de®ned and possible inconsistent constraints
907
was felt to be signi®cant. The generic nature of the techniques was also thought likely to lead to portability and scalability. Many of those present reported experience of the ability of search based techniques to identify surprising and novel features. The initial results presented at the workshop were believed to be highly encouraging and motivated further development, experimentation and exploitation of metaheuristics for Software Engineering.
References [1] I.D. Baxter, Transformation systems: domain-oriented component and implementation knowledge, Proceedings of the Ninth Workshop on Institutionalizing Software Reuse, Austin, TX, USA, January 1999. [2] J.J. Dolado, A validation of the component-based method for software size estimation, IEEE Transactions on Software Engineering 26 (10) (2000) 1006±1021. [3] J.J. Dolado, On the problem of the software cost function, Information and Software Technology 43 (2001) 61±72. [4] M. Harman, B. Jones, SEMINAL: software engineering using metaheuristic innovative algorithms, 23rd International Conference on Software Engineering, ICSE 2001, Toronto, Canada, May 2001, IEEE Computer Society Press, Los Alamitos, CA, USA, 2001 pp. 762±763, Workshop report. [5] B. Jones, H.-H. Sthamer, D. Eyres, Automatic structural testing using genetic algorithms, The Software Engineering Journal 11 (1996) 299±306. [6] B.F. Jones, D.E. Eyres, H.H. Sthamer, A strategy for using genetic algorithms to automate branch and fault-based testing, The Computer Journal 41 (2) (1998) 98±107. [7] R.P. Pargas, M.J. Harrold, R.R. Peck, Test-data generation using genetic algorithms, The Journal of Software Testing, Veri®cation and Reliability 9 (1999) 263±282. [8] F. Tip, A survey of program slicing techniques, Journal of Programming Languages 3 (3) (1995) 121±189. [9] N. Tracey, J. Clark, K. Mander, Automated program ¯aw ®nding using simulated annealing, International Symposium on Software Testing and Analysis, ACM/SIGSOFT, March 1998, pp. 73±81. [10] M. Ward, Reverse engineering through formal transformation, The Computer Journal 37 (5) (1994). [11] J. Wegener, K. Grimm, M. Grochtmann, H. Sthamer, B.F. Jones, Systematic testing of real-time systems, Fourth International Conference on Software Testing Analysis and Review, EuroSTAR 96, 1996. [12] J. Wegener, H. Sthamer, B.F. Jones, D.E. Eyres, Testing real-time systems using genetic algorithms, Software Quality 6 (1997) 127± 135. [13] M. Weiser, Program slicing, IEEE Transactions on Software Engineering 10 (4) (1984) 352±357. [14] D. Whitley, An overview of evolutionary algorithms: practical issues and common pitfalls, Information and Software Technology Special Issue on Software Engineering using Metaheuristic Innovative Algorithms, vol. 43, 2001.
