Genetic Algorithms and the Fast Evolving World of ... - Semantic Scholar

The 10th IEEE International Conference on High Performance Computing and Communications

A Survey: Genetic Algorithms and the Fast Evolving World of Parallel Computing Masaharu Munetomo, Kiyoshi Akama Information Initiative Center Hokkaido University, Sapporo, Japan {munetomo,akama}@iic.hokudai.ac.jp

Asim Munawar, Mohamed Wahib Grad. School of Info. Science & Tech. Hokkaido University, Sapporo, Japan {asim,wahibium}@uva.cims.hokudai.ac.jp

Abstract

ficial evolution[2]. As a result, PGAs often lead to superior numerical performance (not only to faster algorithms), even when executed in a single processor environment[10]. Even though, GAs have a tremendous potential for parallelization, but up to the present time it is a very common practice to implement GAs in a serial fashion for medium complexity problems. This trend prevailed due to a continuous increase in clock speeds over the last two decades but, as rising clock speeds mean rise in power consumption, in this decade multicore technology with low clock speeds is becoming more prominent. Multicore strategy helps to achieve high performance with low clock speeds, resulting in low power consumption. Other trends that are clearly visible during this decade include:

This paper gives a survey about the impact of modern parallel/distributed computing paradigms over Parallel Genetic Algorithms (PGAs). Helping the GA community to feel more comfortable with the evolving parallel paradigms, and marking some areas of research for the High-Performance Computing (HPC) community is the major inspiration behind this survey. In the modern parallel computing paradigms we have considered only two major areas that have evolved very quickly during the past few years, namely, multicore computing and Grid computing. We discuss the challenges involved, and give potential solutions for these challenges. We also propose a hierarchical PGA suitable for Grid environment with multicore computational resources.

1. Distributed Computing: Distributed computing allows the use of resources distributed at global distances. We will only concentrate on the most distributed form of computing called Grid computing. Grid allows secure and coordinated sharing of globally distributed resources spanning several physical organizations [7].

1 Introduction A Genetic Algorithm (GA) is a search technique used to find exact or approximate solutions to optimization and search problems. GAs are categorized as global search heuristics. They are a particular class of Evolutionary Algorithms (EAs) that use techniques inspired by evolutionary biology such as inheritance, mutation, selection, and crossover. GAs construct a population of probable solutions and evolve it over the generations to find a better individual that is close to the optimal solution. GAs usually require a bigger population for solving nontrivial problems, this leads to higher computational cost [3]. Parallel GAs (PGAs) is a technique used to reduce this computational cost. As GAs mimic the process of natural evolution. In nature the individuals do not operate as a single population in which a given individual can mate with any other partner in the population. Rather, species live in and reproduce within subgroups or neighbors only. PGA introduces this concept of interconnected sub populations in arti-

978-0-7695-3352-0/08 $25.00 © 2008 IEEE DOI 10.1109/HPCC.2008.77

2. Data Deluge: Moore’s Law also applies to the sensors, therefore amount of data is also increasing at same rate as the increase in computer’s performance. This phenomenon leads to a flood of data or “data deluge”[9]. All these changes make parallel and distributed algorithms more vital than ever before, and require us to rethink GA design and simulation strategies. We believe that with all its advantages, modern parallel computing paradigms bring complexity in the algorithm/simulation design (see Sect. 3), therefore, designers and programmers are reluctant to embrace the inevitable change in parallel computing paradigms. To bring this trend to an end is the basic inspiration behind this paper. We want the algorithm developers to feel more comfortable with the new parallel paradigms, at the same time we want to encourage the development of tools that can make this transition as smooth as possible for the application developers.

897

Figure 1. Different models of PGAs: (a) global parallelization, (b) coarse grain, (c) coarse grain without controller, (d) fine grain, (e) coarse grain and global parallelization (hybrid), (f) coarse grain and coarse grain (hybrid), (g) coarse grain and fine grain (hybrid), (h) peer-to-peer, (i) coarse grain, fine grain and global parallelization (hybrid).

2 Classification of PGAs

(b) Alogrithms should support late-binding. (c) Alogrithms must be able to take full advantage of any kind of multicore architecture available. (d) Alogrithms should be developed to support the interconnections of loosely coupled distributed applications. (e) Alogrithms must be able to tolerate communication delays up to 100’s of milliseconds. (f) Alogrithms must be able to rely on external remote data sources whenever needed.

PGAs are widely classified into two main classes. This classification relies on the computation/communication ratio. If this ratio is high we call the PGA a coarse grain GA (also known as distributed or island GAs); if it is low we call it a fine grain parallel GA[2]. Many models of PGAs are based on hybrids of these two approaches. Figure 1 shows the most famous models of PGAs [4]. The model shown in Fig. 1-(i) is particularly suitable for modern distributed computing paradigms (see Sect. 4).

GAs are embarrassingly parallel algorithms and therefore they can handle most of these challenges quite easily as discussed in Sect. 5.

3 Challenges Many technical challenges are involved in an efficient implementation of GAs over the modern parallel computing paradigms. Some challenges come into the domain of GA community while the others are related with the HPC community. We will discuss these challenges separately:

2. Challenges for the HPC community: HPC community faces even greater challenges. Due to the long time legacy of using MPI and OpenMP for parallel algorithms, application developers are used to these mature tools. Algorithm developer will be reluctant to follow a very complex programming model that is difficult to design, and very error prone. Moreover, due to the slow learning curve of Grids application developers would never want to learn about Grids in detail. They just want the best performance for their algorithm with a minimum effort. It is a responsibility of the HPC community to provide an easy to use environment for algorithm development. We will consider the following challenges faced by HPC community:

1. Challenges for the GA community: Algorithm developers face a lot of challenges regarding implementation over modern computing paradigms. These challenges are not specific to GAs and apply to any kind of algorithm. Unfortunately, solutions to some of these problems are out of their domain and therefore HPC and algorithm development communities must collaborate with each other to solve these problems. The challenges that come in the domain of algorithm developer’s community can be listed as follows:

(a) Provide a way to deal with the heterogeneous resources.

(a) Alogrithms must be fault tolerant.

898

(b) Provide simulators. (c) Provide a unified way to program hybrid parallel environments. (d) Provide easy and reliable method to access the remote archival and real time data sources. (e) Provide a certain QoS to the application developer despite all the uncertainties involved.

Figure 2. Classification of multicores.

(f) Provide advanced reservation of the resources. (g) Provide an easy to use environment.

may give n times better performance for n number of processing cores. In this decade, however, we see at least one big success of the asymmetric architecture in the form of STI Cell BE. Cell BE is commercially available in IBM Blade Server and Sony Play Station 3 (PS3). With the great success of Cell BE, asymmetric multiprocessor architecture may soon become common once again. GAs can be implemented over multicore architecture using MPI or OpenMP. MPI is expected to perform better on the distributed memory case and OpenMP on the shared memory architecture. All the fine grain GAs (Sect. 2) are well suited for the synchronous multiprocessors. Most of the asynchronous multiprocessors do not have enough local storage to satisfy the demands of genetic algorithms, making them virtually unusable by GAs. However, in Munawar et al. (2008) [20] we propose a way of using such architectures to implement GAs efficiently. The basic idea is to run the GA on a single processor with access to the main memory and run the local search on the distributed processors as shown in Fig. 3. Local search is not a part of GA but GAs

Other challenges not discussed in this paper (due to limitation of space) include: security, fault tolerance, scalability, load balancing, message passing, data storage, communication protocols & architecture, accounting, economic models, debugging tools etc.

4 GAs over different Parallel Paradigms This section presents a survey of the previous work done in the area of PGAs over different kinds of parallel computing environments.

4.1

GAs over Clusters and MPPs

Most of the existing parallel implementations of PGAs use either cluster or Massive Parallel Processor (MPP) for execution. Communication delays in this is less than 10 ms. Approximately 74.6% of the worlds supercomputers are clusters and 21.4% are MPPs [1]. MPI is very commonly used for such environments. Most famous model for PGAs running over clusters is the island model [17, 21]. Clusters and MPPs consist of homogenous nodes, hence making them a haven for most of the parallel algorithms. However, the world is changing and the trend of having a single dedicated cluster of computing nodes is becoming obsolete. The developer in the near future will not be able to enjoy this homogeneity and dedicated resources provided by the cluster. Therefore, we need a way to make the algorithms designed for homogeneous clusters to run efficiently over the modern computing paradigms.

4.2

Figure 3. Implementation of GAs over asynchronous multiprocessor (Munawar et al.).

are often combined with local search techniques to build hybrid algorithms (in some cases local search is mandatory). Similarly for GAs with very complex time consuming fitness functions, simple master-slave architecture can be used to implement fitness function over the available cores.

GAs over Multicore

With the multicore processors getting cheaper and common, we cannot ignore their importance anymore. Multicore systems have multiple processing cores on same chip while multiprocessor systems have multiple chips inside a system. Hybrid of the two is also very common. A further classification is shown in Fig. 2. Most of the systems use symmetric multiprocessors. For embarrassingly parallel problems a symmetric processors

4.3

GAs over Grids

Most important and quickly evolving parallel paradigm is the distributed computing paradigm. Within distributed computing we will discuss only Grid, which is widely accepted to be the most distributed form of computing. Two

899

Figure 4. An example of Hierarchical GA for Grid. very common misconceptions regarding the Grids are: (1) Grid is an alternative for cluster and it helps in reducing the overall execution time by using more resources, (2) Grid is a cluster of clusters. These misconceptions are not completely wrong as reducing the execution time by using distributed resource of Grid will remain an important style of using Grid. However, Grid is much more than that, and provides a lot of advantages over clusters or cluster of clusters. Some of the advantages include: no centralized control, security, access to distributed heterogeneous resources, easy and reliable access to remote data sources, services available to any application etc. In the field of Grid computing, Grid Oriented Genetic Algorithm (GOGA) term was first introduced by [12]. Since then, work has been done on the GOGAs[13, 11] but still it is far from mature. We need hybrid hierarchical PGAs for such kind of environments as shown in Fig. 1(i). Several hierarchical algorithms have been suggested in the recent past, eg. GE-HPGA [15], but most of them ignore the importance of multicores and multiprocessors in the computational nodes. In very near future clusters will consist of nodes with multiprocessors; therefore, we cannot neglect this scenario anymore. We propose a GA suitable for a real Grid environment as shown in Fig. 4. Such GAs are called hierarchical GAs and they are suitable for Grid like distributed environments as they combine the coarse-grained, medium-grained, and fine-grained algorithms.

GAs are known for their modularity (i.e. we can combine different operators of GA in a “plug & play” manner) [18]. This property of the GAs helps in constructing an algorithm with late-binding support. In case of GAs it is also possible to change the operators on the fly. Similarly migration in case of any errors is also very convenient, as the state of the algorithm is represented by the population only. GAs are non deterministic algorithms and each step does not depend entirely on the previous step. Therefore, in case of any error or un-received messages the GAs recover very quickly from the error. Moreover, coarse grain GAs can support the delay of 100’s of milliseconds without affecting the overall performance of the algorithm. Therefore, GAs have all the ability to fit into the new computing paradigms. GAs can also be broken very easily to support hierarchical computation model as shown in Fig. 4. However, in order to design an efficient algorithm for a Grid like environment an algorithm developer must know at least the basics of the Grids and Grid middlewares. Grid middleware is a piece of software that sits between the OS and the application layer to hide most of the low lying details from the application developers. A lot of middlewares have been developed in the past (many of them not compatible with others). However, under the supervision of Global Grid Forum (GGF) an effort started to streamline all these projects, this effort resulted in a standard for middlewares called Open Grid Services Architecture (OGSA)[8]. Core services offered by the OGSA infrastructure, includes: execution management services, data services, resource management services, security services, and information services. OGSA has been adopted as Grid architecture by a number of low-level grid middleware projects including Globus[6]. However, there is a large semantic gap between the low level Grid middleware and application level therefore, it is important that abstraction tools be built based on top of the low-level Grid middleware. For the HPC community the things are much more com-

5 Potential Solutions In this section we will give a brief survey of potential solutions to the challenges mentioned in Sect. 3. We will again divide this section into two parts. First we will give some recommendations to the GA community, and then we will make some recommendations to the HPC community.

900

famous Grid simulators includes GridSim (most famous), SimGrid, Bricks, GangSim, and OptorSim [22]. Another difficult challenge is to provide the user with a unified programming model. Grid involves so many different technologies that it is really difficult for an application programmer to master all these technologies before implementing an algorithm over a Grid. In order to implement an algorithm efficiently over a Grid the developer needs to know at least the basic tools that involve: GridMPI, GridRPC, Schedulers, Workflows, and Web services. To master all these tools before algorithm development over Grid is not only difficult but impossible for application developers. Therefore, we suggest the development of APIs similar to the one presented in Munawar et al. (2008) [19] or intonation based approaches in order to provide a unified programming model that handles all the complexity by itself. Use of graphical programming languages solves some of the problems but many issues still remains to be resolved. Therefore this can be considered one of the hottest areas of research.

Figure 5. Different scenarios of using VMs to solve the problem of deployment.

plicated as discussed in Sect. 3. A lot of research is being done in the area of modern parallel paradigms but the challenges discussed in this paper are often ignored by making some assumptions. The most important problem that comes with the Grids is that of heterogeneous resources. Up till today it is a very common to manually deploy the application before starting a simulation[23]. This is against the true spirit of a Grid and a solution must be found. A potential solution to this problem is the use of Virtual Machines (VMs). With the advent of multicore powerful systems and hardware level support for VMs, we see a sharp increase in number of V irtualM achines as compared to the P hysicalM achines. VMs have their own overhead but they provide a virtual homogeneous environment for the Grid. VMs can be used in two different ways to solve the problem of dynamic deployment as shown in Fig. 5. In the scenario shown in Fig. 5-(a) a minimum OS kernel with Grid middleware support is installed on all the nodes participating in the Grid [14] (also known as virtual workspaces). Another scenario can be the use of multiple VMs on all the gateway nodes in order to provide interfaces of any required kind [5]. Both the methods have their own pros and cons. Many projects are using JavaVM to achieve the desired homogeneity. However, it forces the users to use Java Technology to implement the simulations, which is a severe limitation. Some other famous projects like Folding@Home and SETI@Home have dedicated clients for each type of base resources. This client must be installed before that resource can be used for computations. In its scope this is somewhat analogous to using Java VM and limits the flexibility promised by Grids.

Figure 6. Cost and Realism tradeoff for Grids.

Moreover, despite all the uncertainties involved in a Grid the application developer will require a certain QoS for their algorithm. This can be done by having some kind of agreement between the algorithm and the system itself. Service Level Agreement (SLA) is an active area of research that considers a matrix to make certain promises to the algorithm or the application. SLA can also be used to make advanced reservation of the resources for a specific application. Load balancing and scheduling tools also play an important role in dynamic migration of programs in case of any error in the system. Another major challenge is the reliable and secure transfer of large amount of archival or real time data to remote locations. Fortunately this challenge has more or less been solved in the shape of GridFTP [16]. GridFTP is an FTP based high-performance, secure, reliable data transfer protocol optimized for high-bandwidth wide-area networks. Some major features of GridFTP includes: Grid security infrastructure (GSI) and Kerberos support, third-party control of data transfer, parallel data transfer, striped data transfer, partial file transfer, automatic negotiation of TCP buffer/window sizes, and support for reliable & restartable data transfers.

Due to the costly, and time consuming installation of an actual Grid system, Grids are often simulated for algorithm implementation and testing. Different tools are available for study of a Grid as shown in Fig. 6. However, the user has to make a tradeoff between the cost and realism. A lot of tools have been developed over the past few years [22] however, each one of them have their own limitations. Therefore, it still remains an active area of research. Among all the above mentioned tools simulators are used most commonly,

901

6. Conclusions

[10] V. Gordon and D. Whitley. Serial and parallel genetic algorithms as function optimizers. In S. Forrest, editor, Proceedings of the Fifth International Conference on Genetic Algorithms, pages 177–183, San Mateo, CA, 1993. Morgan Kaufman. [11] J. Herrera, E. Huedo, R. Montero, and I. Llorente. A gridoriented genetic algorithm. In Advances in Grid Computing - EGC 2005, pages 315–322, 2005. [12] H. Imade, R. Morishita, I. Ono, N. Ono, and M. Okamoto. A grid-oriented genetic algorithm for estimating genetic networks by s-systems. SICE 2003 Annual Conference, 3(46):2750–2755, August 2003. [13] H. Imade, R. Morishita, I. Ono, N. Ono, and M. Okamoto. A grid-oriented genetic algorithm framework for bioinformatics. New Gen. Comput., 22(2):177–186, 2004. [14] K. Keahey, I. Foster, T. Freeman, X. Zhang, and D. Galron. Virtual workspaces in the grid. In Europar 2005, Lisbon, Portugal., September 2005. [15] D. Lim, Y. Ong, Y. Jin, B. Sendhoff, and B. Lee. Efficient hierarchical parallel genetic algorithms using grid computing. Future Gener. Comput. Syst., 23(4):658–670, 2007. [16] S. Lim, G. Fox, S. Pallickara, and M. Pierce. Web service robust gridftp. In H. Arabnia and J. Ni, editors, PDPTA ’04, volume 2, pages 725–730, Las Vegas, Nevada, USA, 2004. [17] S. Lin, W. Punch, and E. Goodman. Coarse-grain parallel genetic algorithms: categorization and analysis. In IEEE Symposium on Parallel and Distributed Processing, pages 27–36, 1994. [18] A. Munawar, M. Wahib, M. Munetomo, and K. Akama. A grid based unified framework for optimization. In Proceedings of IPSJ-MPS 69, Kyoto, Japan, May 2008. [19] A. Munawar, M. Wahib, M. Munetomo, and K. Akama. Parallel GEAs with Linkage Analysis over Grid (to appear), chapter Linkage in Evolutionary Computation. Springer, 2008. [20] A. Munawar, M. Wahib, M. Munetomo, and K. Akama. Solving large instances of capacitated vehicle routing problem over cell be. In Proccedings of HPCC 2008 (not accepted yet), Dalian, China, September 2008. [21] C. Pettey, M. Leuze, and J. Grefenstette. A parallel genetic algorithm. In J. G. (Ed.), editor, Proceedings of the 2nd International Conference on Genetic Algorithms and their application (ICGA). Lawrence Erlbaum Associates Publishers, 1987. [22] B. Quetier and F. Cappello. A survey of grid research tools: simulators, emulators and real life platforms. In Proceedings of the 17th IMACS World Congress (IMACS 2005), Paris, France, 2005. [23] H. Takemiya, Y. Tanaka, S. Sekiguchi, S. Ogata, R. Kalia, A. Nakano, and P. Vashishta. Sustainable adaptive grid supercomputing: multiscale simulation of semiconductor processing across the pacific. In SC ’06: Proceedings of the 2006 ACM/IEEE conference on Supercomputing, page 106, New York, NY, USA, 2006. ACM.

The paper gives a brief survey of GAs over the parallel/distributed computing paradigm of the future. We discuss the impact of modern parallel paradigms over the algorithms (especially GAs). GAs are embarrassingly parallel and modular, these properties make GAs an ideal candidate for implementation over modern parallel paradigms. We also presented a flow diagram of a hypothetical hierarchical GA, suitable for the Grid environment. We believe the GAs are capable of dealing with all the challenges posed to algorithm development by the modern parallel computing paradigms. However, the challenges faced by the HPC community are enormous and need a lot of research in the coming years. We highlighted some of the most important issues related with the Grid and multicore computing. Out of these the most important one is the problem of deployment, and the development of unified framework for algorithm development. We also give the potential solutions to some of the challenges discussed in the paper. Especially the use of VMs to implement virtual workspaces. This seems to be a very feasible and practical solution for solving the issue of deployment. Similarly, other potential solutions are also discussed in the paper.

References [1] http://www.top500.org, June (2007). [2] E. Alba and J. Troya. A survey of parallel distributed genetic algorithms. Complex., 4(4):31–52, 1999. [3] E. Cant´u-Paz. A summary of research on parallel genetic algorithms. Technical report, IlliGAL 95007, University of Illinois at Urbana-Champaign, 1995. [4] E. Cant´u-Paz. A survey of parallel genetic algorithms. Technical report, IlliGAL 97003, University of Illinois at Urbana-Champaign, 1997. [5] S. Childs, B. Coghlan, D. Callaghan, G. Quigley, and J. Walsh. Deployment of grid gateways using virtual machines. In Advances in Grid Computing - EGC 2005, pages 761–770. Springer, 2005. [6] I. Foster. Globus toolkit version 4: Software for serviceoriented systems. In IFIP International Conference on Network and Parallel Computing, pages 2–13. Springer-Verlag LNCS 3779, 2006. [7] I. Foster and C. Kesselman. The Grid: Blueprint for a New Computing Infrastructure. Morgan-Kaufman, 1999. [8] I. Foster, H. Kishimoto, A. Savva, D. Berry, A. Djaoui, A. Grimshaw, B. Horn, F. Maciel, F. Siebenlist, R. Subramaniam, J. Treadwell, and J. V. Reich. The open grid services architecture, version 1.0. In GGF informational document. Global Grid Forum(GGF), January 2005. [9] G. Fox, M. S. Aktas, G. Aydin, H. Gadgil, S. Pallickara, E. Pierce, and A. Sayar. Algorithms and the grid. In Computing and Visualization in Science (CVS), March 2005.

902