A Multi-Agent Simulation for Social Agents - CiteSeerX

A Multi-Agent Simulation for Social Agents Yu Zhang, Phillip Coleman, Michael Pellon and Jason Leezer Department of Computer Science Trinity University San Antonio, TX 78216 {yzhang, pcoleman, mpellon, jleezer}@trinity.edu

ABSTRACT Agent-based modeling of human social behavior is an increasingly important research area. Many fully functional multi-agent systems have been developed and have been put to use over the years. But efficient scalable and robust social systems are difficult to engineer, both from the modeling perspective and the implementation perspective. There exist three important difficulties. First, the system needs to have an adaptable agent framework that can successfully make intuitive and deliberative decisions much like a real social participant would. Secondly, the system must have a robust architecture that not only ensures its functioning no matter the simulation, but also provides an easily understood interface that researchers can interact with while running their simulations. Finally, the system must be effectively distributed to handle the necessary number of agents that social research requires to obtain meaningful results. This paper presents our work on creating a multi-agent simulation for social agents that would overcome these three difficulties. Lastly, we present a simulation to test the system and provide the results of that simulation.

Keywords Cognitive Modeling, Social Agents, Agent Simulation

1. INTRODUCTION Multi-agent systems have been widely used to simulate human behaviors in social systems [6, 7, 10, 13, 18]. The systems are often used by social and economic researchers to help them analyze or predict the complex phenomena that emerge from the interactions of massive autonomous agents, especially for the domain that often requires a long time to evolve or requires exposing real people to a dangerous environment. Therefore, the realism of the systems is an important issue in their development. However, efficient, scalable, and robust social systems are difficult to engineer [1], both from the modeling perspective and the implementation perspective.

From the modeling perspective, there exist two difficulties in the societal side and individual agent side respectively. First, in social environments, people interact with each other and form different societies (or organizations or groups). These societies are embedded in multiple structures at the same time such as social structure and space structure. Often, the agent interactions are limited to one structure and correspondingly the drivers of the interactions come from one dimension too. Therefore a real system needs to have an adaptable framework that can capture the multi-dimensional drivers of the evolvement of society and agent. Second, when an individual agent makes a decision, this is a highly cognitive process that sometimes follows intuition and bounded rationality [14]. Therefore, we present a computational model, CASE (Cognitive Agent for Social Environments) [17], which is designed to address these two difficulties. CASE agents act in multidimensional spaces and are supported by a computational descriptive decision model of intuitive and deliberative decision-making. From the implementation perspective, the system must have a robust architecture that not only ensures its functioning no matter the domain, but also provides an easily understood interface that researchers can interact with while running their simulations. Further, the system must be effectively distributed to handle the necessary number of agents that social research requires to obtain meaningful results. Therefore, we develop a multi-agent simulation for CASE agents. Our simulation provides a visualization tool called MASVis (Multi-Agent Systems Visualization) for data analysis and plotting, and supports GIS (Geographic Information Systems). It is deployable on all machines, capable of being controlled by a single GUI on one machine, and is easily scalable. The rest of the paper is organized as follows. In Section 2, we review the CASE agent model. Section 3 presents a simulation for CASE agents. Section 4 reports the experiments. Section 5 is related work and Section 6 concludes the paper.

2. COGNITIVE AGENTS FOR SOCIAL ENVIRONMENTS Cognitive Agents for Social Environments (CASE) is a multi-agent architecture for simulating social agents [17]. CASE is designed to achieve two goals: 1) it aims to model the multi-dimensional drivers for agent interactions, and 2) it provides a computational descriptive decision model of the highly cognitive process wherein an individual agent’s decision-making.

2.1 Agent-Society Evolvement Any society is the result of an interaction between agents, and the behavior of the agents is constrained by the assembly of societal structures [5]. For this reason, a society is not necessarily a static structure, that is, an entity with predefined characteristics and actions. Therefore in our view, a society is the emergence of properties of individual interactions, without it being necessary to define a specific objective which represents such an outcome. The agent/society duality, shown in Fig. 1, characterizes the processes that take place between the agents and the societies which result from them. We are dealing with dynamic interaction, i.e., on one hand, agents have their own goals and are capable of performing various actions; on the other hand, according to social scientists, agents interact with each other and the interactions are embedded in three social structures [12]: 1. 2. 3.

Group: represents social connections. Neighborhood: represents space connections. Network: represents connections that span social and space category. Group

•Social Rules

Network

•Bounded rationality •Intuitive/deliberative

Communication

decision •Perception •Space constraints

•Actions

Neighborhood

Fig. 1 Agent-Society Duality

The purpose of these structures is to reproduce the way information and social strategy are passed and the way people influence each other. A group is a collection of agents who “think alike”, or make similar decisions. It allows for a diffusion of social strategies through social space. An agent can belong to multiple groups at a time and can change groups over time. Each agent also has its own neighborhood and network. A neighborhood includes the agents that this agent can observe their behaviors within a

predefined physical distance. It allows for a diffusion of information through physical space. One agent only has one neighborhood which includes the agents that it can directly observe. However, these neighbors can be different at every step because the agent moves. A network includes the agents that this agent chooses to communicate and interact with. Agents can’t choose who is in their neighborhood but they can choose who they want to interact with. Therefore, the network connection allows for selective diffusion of information. Each agent also only has one network but its members can change over time. For example, if the agent has not heard from one agent over a certain step, the communication connection to that agent will be dropped. Let agent a∈A where A is the set of all agents. Each agent has a social strategy. This social strategy can be either ordinal or cardinal. We denote the social strategy for agent a by Sa. Let g∈G where G is the set of all groups. Each group also has a social strategy, denoted by Sg. Groups are formulated on the basis of a common preference. Each agent identifies itself with any group such that the agent’s preference falls within some threshold of the group’s preference.

∀a∈A and g∈G, a∈g if diff(Sa, Sg) D

(7)

MASVis 

Quantum GIS 

Server  Control 

Client Tier

Simulation  Control  

Middleware Tier 

Service Providers (stateless session beans)

Foreclosure 

Reader 

Loader

MAS  * Entity  * Agent  * SlaveServer  * MasterServer

Communication

Sugarscape  *Experiment domains  are loaded dynamically  at runtime using Java  Reflection API 

Decision

Metadata  Metadata  Generator 

File servers

Data Tier

Fig. 3 A Three Tier SOA Diagram

3. A SIMULATION FOR CASE AGENTS We have developed a simulation for the above agent system. Our system is capable of scaling up to huge simulations, to be capable of being deployed on many machines with the ability to control what’s happening in the simulation through the use of a single GUI running on one machine, and it is able to process large amounts of data and perform operations on that data.

3.1 Sever-Oriented Architecture The Service-Oriented Architecture (SOA) [3] is a design pattern commonly used in many large corporations, such as NASA. It provides a flexible and stable architecture for large scale software systems. For this reason we felt it would be a good fit for the distributed multi agent system we were developing. SOA requires that services should be “loosely coupled”, in other words encapsulated and have as few dependencies on other services as possible. This allows for services to be easily swapped and their implementation to be easily upgraded. One service could easily be swapped out for another service that accomplishes the same result. For our system we use a three tier approach as seen in Fig. 3, which consists of: • The Client Tier • The Middleware Tier • The Data Tier The client tier consists of the user interface. As defined by the nature of the SOA design pattern, the user interface maintains service encapsulation and could theoretically be used with any core system. All that would need to be ensured was that the core followed the service contract as

far as how to communicate with the client tier. The first two services that the interface provides are server control and simulation control. This allows the user to set up their own system distribution and experiment. The other two services the interface provides are MASVis and Quantum GIS. MASVis is a data analysis and visualization package we developed for social simulations. It is built on the fundamental design of having data sources that connect through multiple filters to process elements of data and which can be plotted to display the data. Our simulation also supports GIS. The GIS system we use to display our data is Quantum GIS1. Quantum GIS is suitable for our needs because it allows for reprogramming. It is quite light and allows for plug-ins so it is easily expandable. It also requires few resources, being able to run on very little RAM and consume little processing power, which is necessary for our simulations since our simulations use the processor heavily. Further, it is completely compatible on all operating systems and has a very large community of active developers. Both MASVis and Quantum GIS are still work on progress. The middleware tier consists of the core CASE agent system as described earlier. The core system is made up of the collection of master and slave servers as well as the CASE agents that occupy them. The middleware tier will also consist of the service providers that help the middleware tier communicate with both the client tier as well as the data tier. We have developed two experimental domains: Sugarscape and Foreclosure. Sugarscape is a classical test-bed for growing agent-based societies [4]. Foreclosure is a domain we developed for helping social

1

http://www.qgis.org

scientists to analyze the nationwide “foreclosure crisis” problem (refer to section 4 for more details). The data tier helps encapsulate the data, and stores it separately from the simulation running in the core system; this could entail having the data tier on a separate server. Not only has this helps ensured the stability of the data, it also allows for the use of metadata. In our simulation, the data tier records its data using XML specifications specifying important meta information such as date, author, simulation, etc. This allows for a more robust data system, which can easily be searched using filters, such as range of dates or author. Using these three tiers will help ensure that our simulation is robust enough to allow for any part of it to be updated with little or no difficulty. Also the modular nature of the SOA allows for other researchers to use only the parts of the system that they find to be appropriate for their own project.

•

Secondly it has each of its agents implement its decided action for that step.

The agents are run in the maximum number of threads the server can handle; this ensures optimal performance through parallelization. To ensure effective load balancing the master keeps track of the rest of the server by using a KD tree (kdimensional tree). The KD tree works by splitting the bounded region of the simulation on a different axis each time, until there is a single leaf per slave server, as seen in Fig. 5. Splitting the region up in this manner allows the master server to go in and shift these splits one way or the other to balance the load between slaves. By shifting a KD split, the master server reduces the amount region and the amount of agents that a slave server is responsible for, and by doing so alleviates that server’s load.

Next, we explain in detail the middleware tier which focuses on the system distribution and the client tier which describes the user interface.

3.2 System Distribution For the system to be effective at dealing with the large number of agents required for social research, it has to be highly distributed both in processing as well as memory. To achieve this we developed a Master/Slave system, as showed in Fig. 4. Master Server -Load Balancing -Server Management

Slave Server -Updating Agents Perception -Decision-Making -Updating Agents Action -Communication

Slave Server

…

-Updating Agents Perception -Decision-Making -Updating Agents Action -Communication

Fig. 5 A Graphical Representation of the KD Tree To ensure effective load balancing the master keeps track of the rest of the server by using a KD tree. The KD tree works by splitting the bounded region of the simulation on a different axis each time, until there is a single leaf per slave server, as seen in figure 2. Splitting the region up in this manner allows the master server to go in and shift these splits one way or the other to balance the load between slaves. By shifting a KD split, the master server reduces the amount region and the amount of agents that a slave server is responsible for, and by doing so alleviates that server’s load. Fig. 6 is a screenshot of the server control interface.

Fig. 4 Master/Slave Relationship

The system has a single master server in charge of initializing and synchronizing the other servers (slaves). The master’s responsibilities include initializing each step, facilitating agent communication and agent movement from one server to another, and most importantly load balancing all of the servers to ensure optimal performance. The slave is originally initialized with a given bounds and a set of agents. The slave’s responsibility is to update all of its agents each step, this is a two step process: •

First it updates all of the agents’ knowledge for what they perceived that step

Fig. 6 A Screenshot of the Server Control Interface

3.3 User Interface In creating the user interface, the primary consideration is that users will not be concerned with all of the system’s features at once. Furthermore, there will not be enough

screen space to conveniently and aesthetically portray all of these features, specifically the settings, server, data, graphical, and simulation windows, at once. To house each element of the system’s features in detached windows, able to be hidden and then restored whenever needed, is therefore clearly plausible and arguably necessary for such an open-ended agent simulation. This can be argued for each individual element of the system’s features. Once a user calibrates the appropriate settings for his simulation implementation, he may not need to keep these settings in focus throughout the running of the simulation; rather, the user may be more focused on the system’s output and the results of the simulation. To keep these settings on the screen for the duration of the simulation would force the user to work with cluttered space, constantly having to move windows around in order to see and collect data. To house all of the simulation settings in a detached window, able to be hidden and then restored whenever needed, is, therefore, clearly plausible and arguably necessary for such a wide-breadth multi-agent simulation. This same logic can apply to the server window, where server hosts can be added, managed, or booted. Clearly, server management is not the primary focus of any type of simulation. While servers need to be added at the beginning of a simulation and monitored and/or booted throughout the duration of the simulation, the server window need not be opened and visible the entire time. Again, the detached and hide-able window system works effectively in this situation. This detached-window argument can also be applied to output-based elements of the simulation user interface. Consider when a user is focused on only the raw textual data output of a simulation. Clearly, any visual representation of the data in question, whether it is a graphical representation of this data or a step-by-step visualization of the individual agents in the system, would be extraneous and unneeded in the given situation. Obviously, this scenario can be flipped into a situation in which the user is only focused on visuals, in which case the raw textual data output would be unneeded. Similar to the case of the settings window, housing both graphical data representations and textual data output in detached, escapable and restorable windows, is a necessary feature. To keep almost every aspect of the simulation in separate, hide-able windows requires the use of a main menu, or a master controller, that cannot be closed. This controller handles not only the core, system-wide commands for MASVis but also the hide and restore functions for all other windows, while still remaining as small as possible and preserving screen space. Without a doubt, users of MASVis advantage greatly from a user interface system with separate, moveable, hide-

able windows. Such a system allows for a maximization of the valuable resource of screen real estate and also puts the system’s focus on what currently matters to the user. Fig. 7 is a screenshot of the Sugarscape interface.

Fig. 7 A Screenshot of the Sugarscape Interface

4. EXPERIMENT Multi-agent systems are increasingly used to identify and analyze the social and economic problems of urban areas and provide solutions to these problems [5]. We tested the CASE architecture in the urban dynamics field. We did not use the classical benchmarks such as Santa-Fe Artificial Stock Market (SF-ASM) which is commonly used by the Agent-Based Simulation (ABS) community of social science and economics. This is because such classical testing domains were originally designed for game theory, which uses reactive agents and relies on the concept of equilibrium that is rarely seen in real-world environments, and decision theory, which was grounded on the level of a single individual with a lack of social interests. Our simulation in urban dynamics focuses on the nationwide “foreclosure crisis” problem. With more than 430,000 foreclosure filings reported nationwide, the nation’s rate of foreclosure is at an all time high.2 This experiment simulated mortgage default in San Antonio, Texas. By the first quarter 2007, San Antonio’s rate of foreclosures is nearly twice the national average and is among the top 20 cities with the highest foreclosure rates in the U.S. city. The purpose of this simulation is to examine the relationship between an individual agent’s strategy, and the diffusion of those strategies to its society (groups, neighborhood and network), and finally the overall foreclosure rate of the city. Our initial simulation used only 100 homeowners (agents) and 100 homes that were both randomly generated within several carefully selected statistical parameters to reflect demographic realities of the city. We defined two groups of agents within the system: an aggressive group and a conservative group. Agents in the aggressive group are more adapt to “over-extend” their 2

Data provided by Realtytrac. Data Accessed May, 2007.

Fig. 8 The Result of Team A

Fig. 9 The Result of Team B

credit lines and purchase a home that may or may not be within their budget. While agent in the conservative group are more apt to purchase a home that is well within their budget. Initially each agent is given their own strategy that falls within some statistical range of one of the two groups. This initial strategy defines an each agent's propensity to join one of the two groups. As time passes, agents are influenced in a variety of ways: either through member s of their own groups, their neighbors who may or may not belong to the same group and their extended social network. An agent’s neighborhood is a direct function of its observability, i.e. the amount of agents it can observe at a given time. An agent's network is not a physical construct, but rather a meta-physical medium constructed from the agent's it decides to frequently communicate with. Hence as size of an agent's neighborhood, social network and group membership grows its strategy becomes more dominant within society as a whole. We use three teams. Except for the decision rules, conditions of all teams were exactly the same. • Team A uses Reactive decision rules. • Team B uses Intuitive & Deliberative decision rules. • Team C uses Intuitive & Deliberative decision rules and adds the three social structures to it (group, neighborhood and network). We report three experiments. For each experiment, we ran the system for 100 steps with each single step simulating one month in reality. Experiment 1 involves only Team A. These agents do not utilize any social structure (neighborhoods, groups or networks) and hence have no ability to learn through time or adapt to environmental changes. Fig. 8 is the result of the reactive team. It shows that the foreclosure rate is extraordinarily high (~%60) and is rather unstable, moving up and down 15 points between some time steps. These large variations in the foreclosure rate are a product of both the agent’s simplistic decision making mechanisms and their inability to diffuse successful strategies to other agents. Indicative of the fact that we do not see this high or variable of foreclosure rates in reality is the underlying fact that humans utilize a much more complex and sophisticated means for making decisions.

Fig. 10 The Result of Team C

Experiment 2 tests Team B. Fig. 9 illustrates the results of adding only intuition and deliberation capability to reactive agents. The upper line shows the number of agents who use intuitive decision-making. At the beginning of the simulation, this number is quite low. But agents are able to quickly learn the decisions they made before and use them to solve the new decision problems. The low line is the number of foreclosures. It is lower than the Reactive Team in experiment 1 as the agents are able to recognize recurring instances of potential default and avoid them if possible. However, this adaptability is limited only to an individual agent and there are no mechanisms for one agent to diffuse its successful strategy onto other agents. This limitation is indicated by the presence still of variability in the foreclosure rate. The inability of individual agent leads to decreased stability of the system as a whole. To further explore this, in experiment 3 we test Team C. We add the three social structures (neighborhood, group and network) on top of the existing individual agent decision-making mechanisms. In doing so we allowed the agents to diffuse their successful strategies through physical (neighborhood) and non-physical (group and social network) space. As Fig. 10 indicates we see the same adaptability present in experiment 2 as the knowledge base of the individual agent's expands allowing them to better predict future events based on past experiences, in addition, to an increased level of stability (measured by less variability in the foreclosure rate) as the successful strategy is diffused through various social structures. Experiments 2 and 3 confirm our hypothesis that human decision-making is “embedded” in a social context.

5. RELATED WORK There have been many successful multi-agent systems that all possess beneficial features. Due to the lack of space, we limit this discussion to several of the most relevant systems. We review them from two categories: agent modeling and agent simulation. Agent Modeling. COGnitive agENT (COGENT) [2] is a cognitive agent architecture based on the RPD model [11]. However, it focuses on providing decision-aiding at multiple levels of information processing, such as

information filtering and situation assessment, and not a direct process of real human social behaviors. COgnitive Decision AGEnt (CODAGE) [10] is an agent architecture that derived its decision model from cognitive psychological theories to take bounded rationality into account. But, CODAGE does not consider an agent’s influence on other agents and there is no communication between agents, while we consider communication important since it permits individuals to expand their spheres of interest beyond the self. PsychSim [13] is a multi-agent simulation for human social interactions and gives each agent full decision models of other agents. In PsychSim, bounded rationality is implemented as the finite horizon of an agent’s look-ahead and the possible error in their belief about others, while we treat bounded rationality as a human tendency to anchor on one trait or piece of information when making decisions. Agent Simulation. RePast3, perhaps one of the most feature filled packages, provides templates for easy construction of behavior for individual agents and integrates GIS support, which is a feature that our simulations will need. It is also fully implemented on all systems. The multi agent system named MASON4 is a lightweight system with a good amount of functionality. It has the ability to generate videos and snapshots as well as charts and graphs. JADE5 is a project that we looked at for its ability to be distributed across multiple machines that don’t need to be running the same operating system, which is a feature of our system. It also allows configuration of a distributed model to be controlled by a remote GUI, which is also a feature we implement in our package. Cougaar6, developed for use by the military is influential in that it allows for huge scaling of projects to simulate many agents working together which is a very appealing feature for our simulations.

6. CONCLUSION The work in this paper is the first step in developing a robust and flexible multi-agent simulation that can be used by social scientists and economists. Our future work includes expanding on the data analysis capabilities to allow for a much more streamlined research process. We are also expanding the scale of the foreclosure experiment in order to draw more conclusions to the question regarding the social and economic factors that drives an increase in the foreclosure rate, and predict the likelihood and location of future foreclosures.

3

repast.sourceforge.net cs.gmu.edu/~eclab/projects/mason 5 jade.tilab.com 6 www.cougaar.org 4

7. ACKNOWLEDGMENTS We would like to thank Claudia Scholz and Christine Drennon in the Department of Sociology and Anthropology for helpful discussions. We also thank Mark Lewis, Matt Maly and James McNiff for helping develop the simulation.

8. REFERENCES [1] Castelfranchi, C., Engineering Social Order, Lecture Notes In Computer Science; 1972: 1-18, 2000. [2] Das, S. and Grecu, D., COGENT: Cognitive Agent to Amplify Human Perception and Cognition, AAMAS’00, pp. 443-450, 2000. [3] Erl, T.,Service-Oriented Architecture:Concepts, Technology, and Design, Upper Saddle River: Prentice Hall PTR, 2005. [4] Epstein, J. and Axtell, R., Growing Artificial Societies: Social Science from the Bottom Up, The MIT Press, 1996. [5] Ferber, J., Multi-Agent System: An Introduction to Distributed Artificial Intelligence, Harlow: Addison Wesley Longman, 1999. [6] Goncalves, B. and Esteves, S., Cognitive Agents Based Simulation for Decision Regarding Human Team Composition, AAMAS’06, pp. 34-41, 2006. [7] Hogg, L. and Jennings, N. R., Socially Intelligent Reasoning for Autonomous Agents, IEEE Transactions on Systems, Man and Cybernetics - Par t A, 31(5):381-399, 2001. [8] Kahneman, D. and Tversky, A., Prospect Theory: An Analysis of Decision under Risk, Econometrica, 47(2): 263292, 1979. [9] Kahneman, D., Maps of Bounded Rationality: A Perspective on Intuitive Judgment and Choice, Les Prix Nobel, 2002. [10] Kant, J. and Thiriot, S., Modeling One Human Secision Maker with A Multi-agent System: The CODAGE Approach, AAMAS’06, pp. 50-57, 2006. [11] Klein, G., A Recognition-Primed Decision Making Model of Rapid Decision Making, in Klien, G., Orasanu, J., Calderwood, R. and Zsambok, C. (eds.), Decision Making In Action: Models and Methods, pp. 138-147, 1993. [12] Portes, A. and Sensenbrenner, J., Embeddedness and Immigration: Notes on the Social Determinants of Economic Action, The American Journal of Socioligy, 98(6): 13201350, 1993. [13] Pynadath, D. V. and Marsella, S. C, PsychSim: Modeling Theory of Mind with Decision-Theoretic Agents, IJCAI’05, pp. 1181-1186, 2005. [14] Simon, H., A Behavioral Model of Rational Choice, in Models of Man, Social and Rational: Mathematical Essays on Rational Human Behavior in a Social Setting, New York: Wiley, 1957. [15] Stirling, W. C., Satisficing Games and Decision Making: with Applications to Engineering and Computer Science, Cambridge University Press, 2003. [16] Yoon, P. K. and Hwang, C., Multiple Attribute Decision Making: An Introduction, Sage Publications, 1995. [17] Zhang, Y., Pellon, M., Leezer, J., Coleman, P., CASE: Cognitive Agents for Social Environment, AAMAS, 2008 (under review). [18] Zhang, Y., Pellon, M. and Coleman, P., Decision Under Risk in Multi-Agent Systems, International Conference on System of Systems Engineering, pp. 133-138, 2007.