Emergent Social Structures in Cultural Algorithms Robert G. Reynolds Dept. of Computer Science, Wayne State University
[email protected] Bin Peng Dept. of Computer Science, Wayne State University
[email protected] Robert Whallon Museum of Anthropology, University of Michigan
[email protected] Abstract Abstract - Various biologically inspired approaches to problem solving using a social metaphor have been proposed. For example, both Particle Swarm Optimization (PSO) and Ant Colony Optimization (ACO) have been employed to solve problems in optimization and design. Both approaches employ simple social interactions between agents to produce emergent social structures that are used to solve a given problem. In this paper we investigate the emergence and power of more complex social systems based upon principles of cultural evolution. Cultural Algorithms employ a basic set of knowledge sources, each related to knowledge observed in various social species. These knowledge sources are then combined to direct the decisions of the individual agents in solving optimization problems. Here we develop an algorithm based upon an analogy to the marginal value theorem in foraging theory to guide the integration of these different knowledge sources to direct the agent population. Various phases of problem solving emerge from the combined use of these knowledge sources and these phases result in the emergence of individual roles within the population in terms of leaders and followers. These roles result in organized swarming in the population level and knowledge swarms in the social belief space. Application to real-valued function optimization in engineering design is used to illustrate the principles.
Contact: Dr. Robert G. Reynolds Dept. of Computer Science Wayne State University Detroit, MI 48202 Tel: 1-313-577-0726 Fax: 1-313-577-6868 Email:
[email protected] Key Words: Cultural Algorithms, Swarm Intelligence, Computational social theory, memetic algorithms.
Emergent Social Structures in Cultural Algorithms Robert G. Reynolds, Bin Peng, and Robert Whallon Introduction Recently, a number of socially motivated algorithms have been used to solve complex optimization problems. Some of the example algorithms are the Particle Swarm Optimization (PSO) [Kennedy & Eberhart, 1995], the Ant Colony Optimization (ACO) [Dorigo, Maniezzo & Colorni, 1996], and the Cultural Algorithm [Reynolds 1978, 1994]. These three algorithms all use a population-based model as the backbone of the algorithm and solve problems by sharing information via social interaction among agents in the population. Figure 1 expresses each of these approaches in terms of both a space and a time continuum over which the social interactions take place. Notice that both the ant and particle swarm approaches can be found near the lower left end of this continuum, with the social interaction between individuals taking place within limited temporal and spatial dimensions. For example, in particle swarm, agents can exchange their direction of movement and velocity locally with other agents. In the Ant Algorithm, agents locally exchange information in terms of the density and gradient of a “pheromone” substance that marks their trial. The pheromone is deposited by an ant moving along a trail. The frequency of use of a trail is indicated by the amount of pheromone that has been deposited relative to its degradation in the environment over time. 100 years
Cultural Algorithm
Temporal Scale Ants
Chimps & hominids
day minute meter
Particle Swarm
Spatial Scale
global
Figure 1: Scale of Social Interaction Cultural Algorithms on the other hand allow agents to interact in many different ways using various forms of symbolic information reflective of complex cultural systems. The basic Cultural Algorithm allows individuals to communicate via a shared belief space. The shared space stores five basic types of information that can be shared cognitively or symbolically. It is well known that the scale of interaction within complex systems effect the nature of the structures that emerge from the interaction of agents within that system [Holland 1998]. We now briefly examine each of the three social models for problem solving in terms of the nature of their social interactions and their emergent properties.
Particle Swarm Optimization Particle Swarm Optimization (PSO) is a population based stochastic optimization technique developed by Eberhart and Kennedy in 1995 [Kennedy & Eberhart 1995, Kennedy 1999], inspired by social behavior of bird flocking or fish schooling. In PSO, the potential solutions, called particles, move through the problem space by following the paths of its successful neighbors. Each individual / particle keeps track of its past best performance / fitness and that of its neighbors’ (social view) within a fixed radius and uses this information to determine its next direction and velocity (cognitive view). PSO is initialized with a group of random particles (solutions) and then searches for optima by updating generations. In every iteration, each particle is updated by following two "best" values. The first one is the best
solution (fitness) it has achieved so far (the fitness value is also stored). This value is called pbest. Another "best" value that is tracked by the particle swarm optimizer is the best value obtained so far by any particle in the population. This second best value is a global best and called gbest. When a particle takes part of the population as its topological neighbors, the second best value is a local best and is called lbest. After finding the two best values, the particle updates its velocity and positions with following equation (a) and (b). v[] = v[] + c1 * rand() * (pbest[] - present[]) + c2 * rand() * (gbest[] - present[]) (a) present[] = present[] + v[] (b) v[] is the particle velocity, present[] is the current particle (solution). pbest[] and gbest[] are defined as stated before. rand() is a random number between (0,1). c1, c2 are learning factors. Usually c1 = c2 = 2. The pseudo-code of the initial version of PSO for real-valued variables is given in [Kenneday, Eberhart, & Shi 2001] as follows: For each particle initialize particle End For Do For each particle calculate fitness value if the fitness value is better than the best fitness value (pBest) in history set current value as the new pBest End choose the particle with the best fitness value of all the particles as the gBest For each particle calculate particle velocity according equation (a) update particle position according equation (b) End While maximum iterations or minimum error criteria is not attained
By simulating individual learning and inter-personal (social) cultural transmission, PSO attains both simplicity and efficiency (speed of convergence). It has been demonstrated to perform well on a variety of benchmark problems such as Schaffer f6 function [Kennedy & Eberhart 1995] as well as a wide range of applications such as neural network optimization, and engineering optimization problems such as minimizing the weight of a tension spring [Hu, Eberhart, & Shi 2003]. They generally operate over a two-dimensional landscape but can be extended to multiple dimensions theoretically. Because of the simplicity of their social behavior certain results regarding convergence, convergence rates, and other emergent properties have been produced [Kennedy & Eberhart 1995, Kennedy, Eberhart, & Shi 2001]. The basic emergent property is the particle swarm, or coordinated movement of individuals through the search space towards the optimum.
Ant Colony Algorithm Ant Colony Optimization (ACO) was based on the observation of laboratory ant colonies [Beckers, Deneubourg & Goss, 1992]. These researchers found that ants are capable of finding the shortest path from a food source to the nest without using visual cues [Hölldobler & Wilson, 1990]. Ants were also observed to be capable of adapting to changes in the environment, for example, finding a new shortest path once the old one is no longer feasible due to a new obstacle. It is well known that the primary means for ants to form and maintain the line is a pheromone trail. In the Ant Colony Optimization Algorithms proposed by Dorigo et. al. [1996], an ant is defined to be a simple computational agent, which iteratively constructs a solution to a problem, frequently a path planning problem. In the model, ants deposit a certain amount of pheromone, a chemical substance, while walking, and each ant probabilistically prefers to follow a direction rich in pheromone. Thus, the pheromone and the density of the pheromone along the trail is the knowledge that the ACO shares among its individual ants. Partial problem solutions are seen as states and each ant moves from a state ι to another one ψ corresponding to a more complete partial solution. At each step σ, each ant k computes a set of feasible expansions to its current state, and moves to one of these in probability, according to a probability distribution specified as follows:
k p!" =
* $+&( + (1 % * ) $)&( , if éø " tabuk, éí ! tabuk # (* $+&' + (1 % * ) $)&' )
0, otherwise Here the set tabuk represents a set of feasible moves for ant k and parameter defines α defines the relative importance of trail with respect to attractiveness. After each iteration t of the algorithm, trails are updated using the following formula,
#!" (t ) = $#!" (t & 1) + %#!" where ρ is a user-defined coefficient and Δτιψ represents the sum of the contributions of all ants that use move (ιψ) to construct their solution. The update process functions to increase the level of those cells related to moves that were part of “good” solutions, while decreasing all others. The pseudo-code from Maniezzo [2000] describes how the basic Ant Colony Optimization works: 1. (Initialization) initialize τιψ, ∀ι, ψ 2. (Construction) For each ant k do repeat compute ηιψ, ∀ι, ψ choose in probability the state to move into append the chosen move to the k-th ant’s set tabuk until ant k has completed its solution [apply a local optimization procedure] enddo 3. (Trail update) For each ant move (ι,ψ) do compute Δτιψ and update the trail values 4. (Terminating condition) If not (end_condition) go to step 2
ACO is widely applied to path planning optimization problems in many areas such as the symmetric and asymmetric traveling salesman problems, the sequential ordering problem, the vehicle routing problem as well as the scheduling and partitioning problems associated with telecommunications networks. What emerge from the social interaction of the ants in the population are paths of high performance as defined in terms of some performance function. Some properties of the emergent paths have been demonstrated to hold for specific types of problems such as finding the minimum cost (shortest) paths in a graph in general [Dorigo, Maniezzo & Colorni, 1996].
Cultural Algorithms The Cultural Algorithm (CA) is a class of computational models derived from observing the cultural evolution process in nature [Reynolds 1978, 1994]. CA has three major components: a population space, a belief space, and a protocol that describes how knowledge is exchanged between the first two components. The population space can support any population-based computational model, such as Genetic Algorithms, and Evolutionary Programming. The basic framework is shown in Figure 2. update()
Belief Space accept() select()
influence()
Population Space
obj()
generate()
Figure 2: Framework of Cultural Algorithm
The Cultural Algorithm is a dual inheritance system that characterizes evolution in human culture at both the macro-evolutionary level, which takes place within the belief space, and at the micro-evolutionary level, which occurs in the population space. Knowledge produced in the population space at the micro-evolutionary level is selectively accepted or passed to the belief space and used to adjust the knowledge structures there. This knowledge can then be used to influence the changes made by the population in the next generation. What differentiates Cultural Algorithms from the PSO and ACO algorithms is the fact that the CA uses five basic knowledge types in the problem solving process rather than just one or two locally transmitted value. There is evidence from the field of cognitive science that each of these knowledge types is supported by various animal species [Wynne 2001; Clayton, Griffiths, & Dickinson, 2000] and it is assumed that human social systems minimally support each of these knowledge types as well. The knowledge sources include normative knowledge (ranges of acceptable behaviors), situational knowledge (exemplars or memories of successful and unsuccessful solutions etc.), domain knowledge (knowledge of domain objects, their relationships, and interactions), history knowledge (temporal patterns of behavior), and topographical knowledge (spatial patterns of behavior). This set of categories is viewed as being complete for a given domain in the sense that all available knowledge can be expressed in terms of a combination of one of these classifications.
Problem Statement Cultural Algorithms have been studied with benchmark problems [Chung & Reynolds 1998] as well as applied successfully in a number of diverse application areas such as modeling the evolution of agriculture [Reynolds 1986], concept learning [Reynolds 1994], real-valued function optimization [Jin & Reynolds1999, Reynolds & Saleem 2005] and re-engineering of knowledge bases for the manufacturing assembly process [Rychlyckyj 2003], and agent-based modeling of price incentive systems [Ostrowski & Reynolds, 2002] among others. While successful, the relative complexity of the knowledge sources and their interaction made it difficult to determine why the Cultural Algorithms worked so well. Alternatively stated, under what conditions will such systems successfully solve a given problem and what social structures will emerge along the way? The emergence of these structures in both the population and belief space can be viewed as signs of a successful problem solving process. This paper attempts to develop answers to these questions. In order to do this we begin by examining how Cultural Algorithms solve resource optimization problems within an experimental environment. In our investigation, we first employ a simulated cones world environment developed by Morrison and De Jong [1999] and extended here. Within this world, resources are viewed as being distributed in piles (cones) on the landscape (Sugarscape style [Epstein & Axtell, 1996]). We can place arbitrary numbers of cones of varying sizes on the landscape to produce foraging surfaces of varying complexity. The distribution of cones can be static, dynamic, and deceptive (in that the positioning of some cones can hide better areas from hill climbing agents). Agents then interact socially via these various knowledge sources to find the optimum, and in dynamic environments keep track of its changing position over time. We then investigate the emergence of social patterns in both the population space and the belief space when the problem is successfully solved. Next we use what we have learned from this experimental environment to solve complex problems in engineering design. We will then observe whether similar social structures emerge there. The comparison of our systems performance with particle swarm optimization will be made. Since our cones world problems can be described as foraging problems within a search space, we use a framework inspired by theoretical results from studies of foraging theory in population biology. Specifically, agents select different knowledge sources based on what we characterize as "the marginal value of information". The inspiration for this comes from the classic work by Charnov [1976] concerning the "marginal value theorem". In certain situations, agents using the marginal value theorem were able to optimize their resource intake within an environment. Simply stated, the marginal value theorem says that an agent stays within a location on the landscape until the current resource gain is less than the average expected value. It then moves to another cell that satisfies this marginal value constraint. Here we employ an approach to knowledge integration that uses a corresponding “marginal knowledge value” principle. With this approach, an individual is more likely to use a strategy that is above average in performance and will try another if its performance falls below the average for the other different knowledge sources. We then observe the social organizations that emerge in both the population space and the belief space the overall success of the system. We use this marginal value-based integration approach coupled with an evolutionary programming model for the population, and the five knowledge sources as the basic framework to solve problems within the simulated and
real-world environments. We will show that use of the marginal value approach to knowledge integration produced the following emergent structures and behaviors: 1. The emergence of certain problem solving phases in terms of the relative performance of different knowledge sources over time. We label these phases as coarse grained, fine grained, and backtracking phases. Each phase is characterized by the dominance of a suite or subset of the knowledge sources that are most successful in generating new solutions in that phase. In fact, the dominant subset of knowledge sources is often applied in a specific sequence within each phase. It appears that one type of knowledge produces new solutions that are consequently exploited in by another knowledge source. Transitions between phases occur when the solutions produced by one phase can be better exploited by knowledge sources associated with the next phase. 2. The emergence of swarms of individuals moving within the problem space as a result of the interaction of the cultural knowledge. Since these phases continually emerged in static, dynamic, and deceptive environments when the marginal value integration approach was used. We called these “Cultural or Population Swarms” [Peng, Reynolds, and Brewster 2003]. 3. We then observed the “swarming of knowledge” at the meta-level. These were called “knowledge swarms”. Thus, the swarming of knowledge sources at the meta-level was produced by the interaction of knowledge sources via the marginal value theorem and this serves to induce a swarming at the population level.
References [Beckers, Deneubourg & Goss, 1992] Beckers, R., Deneubourg, J.L. & Goss, S., 1992, “Trails and U-turns in the Selection of the Shortest Path by the Ant Lasius Niger.” Journal of Theoretical Biology, 159, 397–415. [Charnov 1976] Charnov, E. L. (1976). “Optimal Foraging: the Marginal Value Theorem”. Theoretical Population Biology, 9, 129-136. [Chung & Reynolds 1998] Chung, C. & Reynolds, G. R., 1998, “CAEP: An Evolution-based Tool for RealValued Function Optimization using Cultural Algorithms.” International Journal on Artificial Intelligence Tools, 7(3), 239-291. [Clayton, Griffiths, & Dickinson, 2000] Clayton, N. S., Griffiths, D. P., & Dickinson A., 2000, "Declarative and Episodic-like Memory in Animals: Personal Musings of a Scrub Jay." In The Evolution of Cognition. Edited by Heyes, C. and Huber, L. the MIT Press, Cambridge, Massachusetts. [Dorigo, Maniezzo & Colorni, 1996] Dorigo, M., Maniezzo, V. & Colorni, A., 1996, “Ant System: Optimization by a Colony of Cooperating Agents.” IEEE Transactions on Systems, Man, and CyberneticsPart B, 26(1), 29-41. [Epstein & Axtell, 1996] Epstein, J., & Axtell R., 1996, Growing Artificial Societies. MIT Press/Brookings Institute, Cambridge, MA. [Holland 1998] Holland, J. H., 1998, Emergence. Addison-Wesley Press, Reading, MA, 1-10. [Hölldobler & Wilson, 1990] Hölldobler, B. & Wilson, E.O., 1990, The Ants. Springer-Verlag, Berlin. [Hu, Eberhart, & Shi 2003] Hu, X., Eberhart, C. R. & Shi, Y., 2003, “Engineering Optimization with Particle Swarm.” In Proceedings of the 2003 IEEE Swarm Intelligence Symposium (Indiana, U.S.A.), IEEE Press, 53-57. Iacoban, R., Reynolds. R. G., Brewster, J. (2003) “Cultural Swarms: Modeling the Impact of Culture on Social Interaction and Problem Solving, ” in Proceedings of 2003 IEEE Swarm Intelligence Symposium, IEEE Press, 205-211. [Jin & Reynolds, 1999] Jin, X. & Reynolds, G. R., 1999, “Using Knowledge-Based Evolutionary Computation to Solve Nonlinear Constraint Optimization Problems: a Cultural Algorithm Approach.” In Proceeding of the 1999 Congress on Evolutionary Computation, 1672-1678.
[Kennedy & Eberhart, 1995] Kennedy, J. & Eberhart, R. C., 1995, “Particle Swarm Optimization.” In Proceeding of the IEEE International Conference on Neural Networks, Perth, Australia, IEEE Service Center, 12-13. [Kennedy 1999] Kennedy, J., 1999, “Small Worlds and Mega-Minds: Effects of Neighborhood Topology on Particle Swarm Performance.” In Proceedings of the 1999 IEEE Congress on Evolutionary Computation, IEEE Service Center, Piscataway, NJ, 22-31. [Kennedy, Eberhart, & Shi 2001] Kennedy, J., Eberhart, R. C., & Shi, Y., 2001, Swarm Intelligence. Morgan Kaufmann Publishers, San Francisco, CA. [Maniezzo 2000] Maniezzo, V., 2000, Ant Colony Optimization: An Overview. http://www3.csr.unibo.it/~maniezzo/didattica/Vienna/ACOintro.pdf. Retrieved on March 12, 2005. [Morrison & De Jong, 1999] Morrison, R. & De Jong, K., 1999, “A Test Problem Generator for Non-Stationary Environments.” In Proceedings of Congress on Evolutionary Computation, IEEE Press, 2047-2053. [Ostrowski 2002] Ostrowski, D. A., Tassier, T., Everson, M., and Reynolds, R. G. (2002). “Using Cultural Algorithm to Evolve Strategies in Agent-Based Models”. In Proceedings of the 2002 IEEE World Congress on Computational Intelligence, Wakiki, Hawaii, 2-7. [Peng, Reynolds, & Brewster, 2003] Peng, B., Reynolds, R. G., & Brewster, J. J., 2003, “Cultural Swarms.” In Proceedings of IEEE The Congress on Evolutionary Computation, Canberra, Australia, 1965-1971. [Reynolds 1978] Reynolds, R. G., 1978, “On Modeling the Evolution of Hunter-Gatherer Decision-Making Systems.” Geographical Analysis, 10(1), 31-46. [Reynolds 1986] Reynolds, R. G., 1986, “An Adaptive Computer Model of Plan Collection and Early Agriculture in the Eastern Valley of Oaxaca.” In Guila Naquitz: Archaic Foraging and Early Agriculture in Oaxaca, Mexico, K. V. Flattery Ed, Academic Press, 439-500. [Reynolds 1994] Reynolds, G. R., 1994, “An Introduction to Cultural Algorithms.” In Proceedings of the 3rd Annual Conference on Evolutionary Programming, World Scienfific Publishing, 131-139. [Reynolds & Saleem 2005] Reynolds, R. G., & Saleem, S. M., 2005, “The Impact of Environmental Dynamics on Cultural Emergence.” Perspectives on Adaptions in Natural and Artificial Systems. Oxford University Press, 253-280. [Rychlyckyj 2003] Rychlyckyj, N., Ostrowski, D., Schleis, G. & Reynolds, R. G., 2003, "Using Cultural Algorithms in Industry." In Proceeding of the 2003 IEEE Swarm Intelligence Symposium, 187-192. [Wynne 2001] Wynne C. D., 2001, Animal Cognition - The Mental Lives of Animals. Palgrave Macmillan, Great Britain.
