duce fundamental concepts of intelligent agents, describe soft computing ..... to intuitive heuristics, used GAs to generate better rule base, and finally. 14 ...
A Soft Computing Framework for Intelligent Agents: Application to Software and Hardware Agents
Seung-Ik Lee
The Graduate School Yonsei University Department of Computer Science and Industrial Systems Engineering
A Soft Computing Framework for Intelligent Agents: Application to Software and Hardware Agents A Dissertation Submitted to the Department of Computer Science and Industrial Systems Engineering and the Graduate School of Yonsei University in partial fulfillment of the requirements for the degree of Doctor of Philosophy
Seung-Ik Lee
December 2001
This certifies that the dissertation of Seung-Ik Lee is approved.
Thesis Supervisor: Sung-Bae Cho
Yoon-chul Choy
Yillbyung Lee
Sung-Bong Yang
Sang-Rok Oh
The Graduate School Yonsei University December 2001
c Copyright by Seung-Ik Lee, 2001. All Rights Reserved.
iii
To my mother and family for their patience and understanding.
/ ?< ü s K¦ \ KÒ Å Q #Qm < ü 7 ¤ á \ > sH 7 H ë¦ `¼ × n w m .
iv
CONTENTS ABSTRACT 1
2
3
xii
INTRODUCTION
1
1.1
Background and Motivation . . . . . . . . . . . . . . . .
2
1.1.1
Fundamental Concepts of Intelligent Agents . . . .
3
1.1.2
What is Soft Computing? . . . . . . . . . . . . . .
7
1.1.3
Need for New Soft Computing Framework . . . .
8
1.2
Research Objectives . . . . . . . . . . . . . . . . . . . . .
10
1.3
Organization of the Dissertation . . . . . . . . . . . . . .
12
RELATED WORKS
13
2.1
Conventional Soft Computing Frameworks . . . . . . . .
13
2.2
Analysis of Evolution . . . . . . . . . . . . . . . . . . . .
18
SOFT COMPUTING FRAMEWORK
21
3.1
Soft Computing Techniques and Frameworks . . . . . . .
22
3.1.1
Comparison of Soft Computing Techniques . . . .
22
3.1.2
Conventional Soft Computing Frameworks . . . .
25
Soft Computing Framework for Intelligent Agents . . . . .
27
3.2
v
3.3
3.4
3.5
4
Rule-based Systems . . . . . . . . . . . . . . . . . . . . .
30
3.3.1
Fuzzy Logic . . . . . . . . . . . . . . . . . . . .
31
3.3.2
Fuzzy Sets and Rule Base . . . . . . . . . . . . .
32
3.3.3
Fuzzy Reasoning and Defuzzification . . . . . . .
35
Evolutionary Algorithms . . . . . . . . . . . . . . . . . .
37
3.4.1
Genetic Algorithms . . . . . . . . . . . . . . . . .
38
3.4.2
Genetic Operations . . . . . . . . . . . . . . . . .
40
3.4.3
Learning with GA . . . . . . . . . . . . . . . . .
44
Analysis Methods . . . . . . . . . . . . . . . . . . . . . .
46
3.5.1
Evolutionary Activity Statistics . . . . . . . . . .
46
3.5.2
Observational Emergence . . . . . . . . . . . . .
50
3.5.3
Schema Analysis . . . . . . . . . . . . . . . . . .
52
APPLICATION TO A HARDWARE AGENT
54
4.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . .
56
4.2
Hardware Agent: KHEPERA . . . . . . . . . . . . . . . .
57
4.3
Rule-Based System: Fuzzy Logic Controller . . . . . . . .
59
4.4
Evolution of Behavior Controller . . . . . . . . . . . . . .
61
4.5
Evolution Analysis . . . . . . . . . . . . . . . . . . . . .
65
4.5.1
Adaptive Evolution . . . . . . . . . . . . . . . . .
65
4.5.2
Adaptive Behaviors . . . . . . . . . . . . . . . . .
71
4.5.3
Evolutionary Pathways to Solutions . . . . . . . .
76
Emergence Analysis . . . . . . . . . . . . . . . . . . . .
85
4.6.1
Turning Around . . . . . . . . . . . . . . . . . . .
85
4.6.2
Smooth Cornering . . . . . . . . . . . . . . . . .
89
Summary . . . . . . . . . . . . . . . . . . . . . . . . . .
92
4.6
4.7
vi
5
6
APPLICATION TO A SOFTWARE AGENT
94
5.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . .
96
5.2
Software Agent: Conversational Agent . . . . . . . . . . .
98
5.2.1
Dialogue Acts Classification . . . . . . . . . . . .
99
5.2.2
Knowledge Representation . . . . . . . . . . . . . 103
5.3
Behavior Controller : Rule-based Pattern Matcher . . . . . 106
5.4
Experimental Results . . . . . . . . . . . . . . . . . . . . 107
5.5
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . 109
CONCLUSION
111
BIBLIOGRAPHY
116
ABSTRACT (KOREAN)
128
vii
LIST OF FIGURES 1.1
Categories of intelligent agents [16]. . . . . . . . . . . . .
3
3.1
Conventional soft computing framework. . . . . . . . . .
25
3.2
Possible evolutionary pathways. . . . . . . . . . . . . . .
26
3.3
Proposed soft computing framework for agents. . . . . . .
28
3.4
Block diagram of a fuzzy controller. . . . . . . . . . . . .
31
3.5
Single point crossover. . . . . . . . . . . . . . . . . . . .
41
3.6
A single mutation. . . . . . . . . . . . . . . . . . . . . . .
42
3.7
General procedure of GA. . . . . . . . . . . . . . . . . . .
43
4.1
A framework for hardware agents. . . . . . . . . . . . . .
55
4.2
(a). Khepera mobile robot (b). IR sensors and motors [46].
57
4.3
Position of some parts of the robot [46]. ➀ LEDs. ➁ Serial line (S) connector. ➂ Reset button. ➃ Jumpers for the running mode selection. ➄ Infra-Red proximity sensors. ➅ Battery recharge connector. ➆ ON-OFF battery switch. ➇ Second reset button. . . . . . . . . . . . . . . . . . . . . .
58
4.4
Gene representation of an individual. . . . . . . . . . . . .
61
4.5
Three out of five membership functions need to be encoded.
62
viii
4.6
Encoding of a rule. . . . . . . . . . . . . . . . . . . . . .
62
4.7
Environment. . . . . . . . . . . . . . . . . . . . . . . . .
64
4.8
(a). Fitness changes. (b). Trajectory of the best individual.
65
4.9
Fuzzy sets obtained by evolution. . . . . . . . . . . . . . .
66
4.10 Activity distribution function
.
. . . . . . . . . . .
68
. (c). Mean cumulative evolutionary activity . (d). New activity . . . . . . . . . . . . .
69
4.11 (a). Diversity
. (b). Total cumulative evolutionary ac-
tivity
4.12 Log-log plot of the component activity distributions for the fuzzy model and its neutral shadow. . . . . . . . . . . . .
70
4.13 Evolutionary activities of salient waves. . . . . . . . . . .
72
4.14 Behaviors of S and corresponding data. . . . . . . . . . .
73
4.15 Behaviors of S
75
4.16
77
4.17 4.18 4.19
S and corresponding data. . . . . . . . . Behaviors of S S S and corresponding data. . . . . . Analysis of schemata S ! #" and B ! #" . . . . . . . . . . . . . Analysis of the schemata S ! $" and B ! $" . . . . . . . . . . . Analysis of schemata S !#% " and B !#% " . . . . . . . . . . . . .
4.20 The lineage of the best individual. . . . . . . . . . . . . . 4.21 4.22 4.23 4.24 4.25 4.26
&('*) of the three first order structures: (a). &+'*),#- . . (b). &+'*),#- % . . (c). &+'*),#- / . . . . . . . . . . . . . . . . . . . . Analysis of &+'*) . of turning around. . . . . . . . . . . . . &('*) of turning around. . . . . . . . . . . . . . . . . . . . Analysis of &+'*) of turning around. . . . . . . . . . . . . &('*) of the two first order structures of smooth cornering: (a). &+'*),#- . (b). &+'*),#- / . . . . . . . . . . . . . . . . . . Analysis of &+'*) . of smooth cornering. . . . . . . . . . . ix
80 81 82 84 86 87 87 88 89 90
4.27 Smooth cornering. . . . . . . . . . . . . . . . . . . . . . .
90
4.28 Analysis of
of smooth cornering. . . . . . . . . . . .
91
5.1
A framework for software agents. . . . . . . . . . . . . .
95
5.2
Structure of a conversational agent. . . . . . . . . . . . . .
98
5.3
Automata for location question. . . . . . . . . . . . . . . 101
5.4
Dialogue acts classification using subsumption architecture. 102
5.5
Internal representation of script. . . . . . . . . . . . . . . 106
5.6
Overall response matching procedure. . . . . . . . . . . . 108
5.7
Location question 1. . . . . . . . . . . . . . . . . . . . . . 108
5.8
Location question 2. . . . . . . . . . . . . . . . . . . . . . 108
&+'*)
x
LIST OF TABLES 4.1
Rules of the best individual. . . . . . . . . . . . . . . . .
66
4.2
Salient rules. . . . . . . . . . . . . . . . . . . . . . . . .
74
4.3
Test rule set for S . . . . . . . . . . . . . . . . . . . . . .
74
4.4
Test rule set for S . . . . . . . . . . . . . . . . . . . . . .
76
4.5
Schemata of the salient rules. . . . . . . . . . . . . . . . .
78
4.6
Schemata of the best individual’s rules. . . . . . . . . . . .
4.7
- . , - % . , and - / . .
78
. . . . . . . . . . . . . . . . . . . . . .
85
5.1
Dialogue acts. . . . . . . . . . . . . . . . . . . . . . . . . 100
5.2
A part of BNF grammar for the query-response script database.104
5.3
Comparison functions. . . . . . . . . . . . . . . . . . . . 105
5.4
A part of script. . . . . . . . . . . . . . . . . . . . . . . . 105
5.5
Score assignment. . . . . . . . . . . . . . . . . . . . . . . 107
xi
ABSTRACT A Soft Computing Framework for Intelligent Agents: Application to Software and Hardware Agents Seung-Ik Lee Dept. of Computer Science and Industrial Systems Engineering The Graduate School Yonsei University The old ways of interactions between human and computer require more time and effort of users and make untrained users to meet more difficulties. Recent research on agents has opened a new way for solving those problems, especially utilizing soft computing techniques. Two main frameworks are used to apply soft computing techniques to the construction of agents. One is to apply each soft computing technique independently. It has some drawbacks in determining the internal parameters of the technique used. The other one is to combine two or more soft computing techniques to relieve the difficulties of tuning internal parameters. Especially, the combination of an evolutionary algorithm and other techniques has been extensively attempted. Despite the success of the conventional framework, it has at least three shortcomings. First, it cannot guarantee that evolved solutions are from
xii
adaptation, since evolutionary phenomena contain not only adaptation but also others such as chances, necessity, and random genetic drift. Second, it cannot show what, when, how, and why some genes have successfully evolved into good solutions while the others not. These questions are critical to show causes of adaptation and evolution. Finally, it is not able to analyze the evolved solutions. Evolution necessarily causes the changes of simple structures into complex ones that may be viewed as emergent behaviors or emergent properties. However, the conventional frameworks cannot tell whether certain behaviors are emergent because they do not have any formal definition of emergence. This dissertation proposes a soft computing framework for intelligent agents to overcome the shortcomings of the conventional frameworks. The framework consists of two parts: construction and analysis part. In the construction part, the combination of rule-based systems and evolutionary algorithms is used to construct an intelligent agent. In the analysis part, evolutionary activity statistics, schema analysis, and observational emergence are applied to the analysis of the evolved agent in both evolutionary and behavioral perspectives. Evolutionary activity statistics is a quantitative measure for adaptive evolution and can tell objectively and quantitatively whether an evolution is adaptive. Schema analysis is to identify evolutionary pathways to solutions. Observational emergence identifies emergent behaviors of the evolved controller. To show the usefulness of the framework, this dissertation applies it to two agents: software and hardware agents, which are the two main fields of agent research.
xiii
In the first case study, the framework is applied to the construction of a behavior controller for a hardware agent. The hardware agent called Khepera is a kind of behavior-based mobile robot. Eight proximity sensors are used as input to the behavior controller and two control values are produced from the controller to drive the two motors of the robot. Fuzzy logic, a kind of rule-based systems, is used for the behavior controller, and a genetic algorithm tunes the internal parameters of the fuzzy behavior controller. Analysis is performed on the evolved controller and the evolutionary process using the framework. In the second case study, the framework is applied to the construction of a behavior controller for a software agent. The software agent is a conversation agent that can act as a virtual representative of a web site interacting with visitors using natural language. The agent consists of three main components: dialogue act categorization, structured pattern matching, and knowledge construction and representation. Dialogue acts (DAs) are classified by automata that accept a sequence of keywords to identify user’s intentions. In addition, subsumption architecture controls the interactions among DA analysis modules to make the classification of DA more effective. We utilize pattern matching techniques rather than conventional natural language processing techniques to match users’ queries with a knowledge base. The two case studies show that the framework can be effectively applied to the construction of controllers for intelligent agents. In addition, the analysis on the evolutionary process presents that the solutions have resulted from the adaptability of each component that constitutes the solution rather than from chances or other evolutionary phenomena. The analysis
xiv
on the behaviors of the evolved controllers shows that the controller has obtained emergent behaviors through the evolutionary process. Especially, the emergent behaviors have resulted from the interactions of lower level components of behaviors. Consequently, we have confirmed that the framework can be effectively applied to the construction of controllers for intelligent agents. Particularly, it has advantages in analyzing the behaviors of an evolved controller and in analyzing the evolution.
Key Words: Adaptability, Conversational Agent, Evolutionary Activity Statistics, Evolutionary Algorithm, Fuzzy Logic, Intelligent Agent, Mobile Robot, Observational Emergence, Schema, Soft Computing Framework.
xv
CHAPTER 1 INTRODUCTION The “information highway” is presenting us with computer-based works of increased complexity, requiring new ways of interactions between human and computer. In existing ways, computer-based systems are guided by the direct manipulation of users. However, the increase of the amount of information requires more time and effort of users and present users with more difficulties. Recent research on agents has opened a new way for solving those more complex works. Though there is no commonly accepted answer for what an agent is, it can be generally regarded as a computer-based system that can reason and act in order to achieve users’ goals while interacting with not only users but also environments. For the fulfillment of users’ goals, an agent needs accurate recognition of current situation and reasoning. Because of the complexity of interactions with users or environments, they are typically ill-defined, difficult to model, and with large-scale solution spaces. In these cases, precise models are impractical, too expensive, or non-existent,
1
1. INTRODUCTION because the relevant available information is usually the form of empirical prior knowledge and input-output data representing instances of the system’s behavior. Fortunately, soft computing technologies provide us with a set of flexible computing tools to perform these approximate reasoning and search tasks with such imperfect information. This dissertation presents a soft computing framework for intelligent agents by augmenting conventional soft computing frameworks and applies it to two case studies, software agent and hardware agent, to show its usefulness. The organization of this chapter is as follows: In Section 1.1, we introduce fundamental concepts of intelligent agents, describe soft computing techniques briefly, and motivate the need for a new framework by describing the problems and limitations of the previous soft computing frameworks. In Section 1.2, we discuss the research objectives of this study. Finally, Section 1.3 outlines the organization of the rest of this dissertation.
1.1 Background and Motivation In this section, we first describe on the definition of agent and what properties an agent should have. Second, we describe what soft computing is and why it is necessary for intelligent agents. Finally, we mention the merits and demerits of conventional soft computing frameworks, and discuss the need for a new soft computing framework to overcome the demerits of the conventional frameworks.
2
1. INTRODUCTION
1.1.1 Fundamental Concepts of Intelligent Agents Before proceeding further on the concepts of intelligent agents, we first present a category of agents, which almost agent researchers can accept. Zarnekow categorized agents at a highest level as in Fig. 1.1 [16]. Intelligent Agents
Human Agents (e.g. travel agent)
Hardware Agents (e.g. robot)
Software Agents
Information Agents
Cooperation Agents
Transaction Agents
Fig. 1.1: Categories of intelligent agents [16]. All the categories have the common feature that they independently perform tasks, which require specialized knowledge or consists of many time intensive steps, on behalf of their contracting party or user. Definitions of Intelligent Agents Though it is commonly accepted that hardware and software agents should perform certain tasks for their users that they cannot undertake themselves because of insufficient time or lack of knowledge, it has not yet been possible to agree on a generally accepted, comprehensive, definition of an intelligent agent. However, many researchers have tried to define what an agent is [16, 29, 72]. Brenner et al. [16] referred to an agent as “an agent always re-
3
1. INTRODUCTION quires a certain amount of intelligence to perform its tasks. Consequently, one refers to intelligent agents. A non-intelligent agent can be considered as any traditional software program, because even traditional programs perform a specific task and provide their users with a direct time saving. Only the intelligence permits an agent to perform its task largely autonomously and to require intervention from the user only for important decisions.” An agent that cannot process tasks independently is not much useful because its user should intervene in the process and, thus, gain only a minimal time saving. Consequently, autonomous processing is one of both important criteria for intelligent agents and main differences between intelligent agents and traditional software programs. An agent must interact with environments to achieve its goals and be capable of gathering information on its environment. Making decisions based on this information, it then initiates specific actions. Taking these considerations into account, Brenner et al. [16] concluded that “intelligent software agents are defined as being a software program that can perform specific tasks for a user and possesses a degree of intelligence that permits it to perform parts of its tasks autonomously and interact with its environment in a useful manner.” Shoharn [72] defined a software agent as “a software entity which functions continuously and autonomously in a particular environment, often inhabited by other agents and processes.” The continuity and autonomy ensures that an agent is capable of carrying out activities in a flexible and intelligent manner that is responsive to changes in the environment without human intervention. Franklin and Graesser [29] defined an autonomous agent as “a system
4
1. INTRODUCTION situated within and part of an environment that senses the environment and acts on it, over time, in pursuit of its own agenda and so as to effect what it senses in the future” Although the above definitions are mainly concerned with software agents, many concepts apply equally to hardware agents. As there is no commonly accepted definition of an intelligent agent, we refer to an intelligent agent as having autonomy and interaction with its environment, mainly based on Brenner’s definition. In addition, we use the two terms, intelligent agent and agent, interchangeably unless we refer to one of them specifically. Properties of Intelligent Agents This section describes in detail some of important characteristics of intelligent agents to obtain an understanding of the tasks and functions of them. One thing to note here is that not every agent must have all the listed characteristics. Actually, there are many systems with different complexities: A simple agent with very low complexity may have one or two of the characteristics whereas agents with high complexity can have all the characteristics in the ideal situation.
0
Autonomy: One of important differences between agents and other non-agent entities is the capability of an agent to follow its goals autonomously. It refers to the principle that agents can operate on their own without the need for human guidance, even though this would sometimes be invaluable. Hence, agents have individual internal states and goals, and they act in such a manner as to meet its goals
5
1. INTRODUCTION
0
on behalf of its user [62]. Reactivity: Reactivity designates that an agent must be capable of reacting appropriately to influences or information from its environment. This environment can consist of other agents, human users, external information sources, or physical objects. Reactivity forms one of the fundamental requirements for an intelligent agent and, therefore, all agents should support it to some extent. The agent must have suitable sensors or possess its own internal model of its environment
0
in order to be able to react to changes in the environment [16]. Reasoning and learning: Reasoning capability puts an agent in the position of being able to observe its environment and to make specific decisions when changes occur in this environment. AI techniques, such as rule-based systems, knowledge-based systems, or neural networks are well suited for reasoning. These include an artificial evolution model in which agents create new generations of agents that
0
have increasingly complex reasoning mechanisms [9]. Proactivity: Proactivity designates that agents do not simply act in response to their environment, they are able to exhibit goal-directed behavior by taking the initiative [88]. It is a level above the reactivity. The capability for an agent itself to take the initiative requires that the agent have well-defined goals or even a complex goal system. However, true proactive behavior occurs only to a very limited extent
0
with complex goal systems. Social ability: An agent often interacts with its environment, espe-
6
1. INTRODUCTION cially humans or other agents via some kind of communication languages. This characteristic designates two kinds of properties. One is the communications ability with humans or other agents. The other is the cooperation ability with other agents. However, the latter does not always require certain kind of communication languages [16, 62, 88]. If the listed characteristics are compared with any existing agent, it is apparent that few of them, currently available or in development, have all or most of the properties. In addition, traditional algorithmic approaches cannot easily solve the listed properties, because the properties are difficult to model, ill defined, and require large-scale solution space. Here comes the need for soft computing techniques in order for software and hardware agents to overcome the complexity.
1.1.2 What is Soft Computing? Soft computing is a recently coined term describing the symbiotic use of many emerging computing disciplines [18, 32, 45, 47, 79]. Zadeh defined it as “... in contrast to traditional, hard computing, soft computing is tolerant of imprecision, uncertainty, and partial truth.” [93] Soft computing encompasses computational techniques such as neural nets, fuzzy systems, probabilistic reasoning (e.g., Bayesian networks), and evolutionary algorithms (e.g., genetic algorithms). The common denominator of these technologies is their departure from classical reasoning and modeling approaches that are usually based on Boolean logic, analytical models, crisp classifications, and deterministic search. These classical approaches describe the system to be modeled by complete and precise infor-
7
1. INTRODUCTION mation. The state of the system can be stated to be binary truth-values by formal reasoning process. Real world problems, however, are typically ill defined, difficult to model, and with large-scale solution spaces, and therefore precise models are impractical, too expensive, or non-existent. The relevant available information is usually limited to empirical knowledge and input-output data representing the system’s behaviors. Consequently, approximate reasoning systems capable of handling such imperfect information are required. Soft computing techniques can handle imperfect information of computationally hard problems that do not seem to be solved by classical algorithmic approaches. Soft computing is expected to show good performance in the processing of linguistic information, processing of pictorial information such as classifying characters, filtering information from textual and pictorial databases, analogical reasoning, and so on.
1.1.3 Need for New Soft Computing Framework There are two main frameworks in applying soft computing techniques to the construction of agents. One is to apply each soft computing technique independently and the other is to combine two or more techniques. The former framework has advantages in its methodological simplicity and facilities in construction if applied to a relatively simple and tiny problem. However, it has some drawbacks in determining the internal parameters of the techniques because it requires expert-level knowledge and needs more time and effort depending on the problems and the technique used. For example, it is well known fact that to determine the optimal topology
8
1. INTRODUCTION and weights of an artificial neural network is hard, time-consuming work. The latter framework is proposed to solve the problems of the former framework. This framework combines two or more different soft computing techniques to relieve the difficulties of determining internal parameters. Especially, the combination of an evolutionary algorithm and other soft computing technique has been focused very much, where the evolutionary algorithm tunes the internal parameters of its partner through evolution. This framework has advantages in that it requires relatively less expert-level knowledge and expertise and can show better performance because of the global searching capabilities of the evolutionary algorithm. Much of previous work has shown good results in finding good behavior controllers for agents by adopting this framework [11, 19, 31, 43, 54, 60, 69, 77]. Despite the success of the combination of soft computing techniques and evolutionary algorithms, it has at least three shortcomings. First, it cannot guarantee that evolved solutions are from adaptive evolution. In general, evolutionary phenomena contain not only adaptive evolution but also others such as chances, necessity, and random genetic drift [20, 58, 84]. Therefore, it cannot be said that the solutions have resulted from adaptive evolutionary process. In other words, the same good solutions as previous ones may not be obtained anymore in the follow-up experiments even if experimental conditions are the same. Therefore, to prove or show that the evolved solutions have resulted from adaptive evolution has great scientific importance on the justification of the experiments and evolution. Second, it cannot show evolutionary pathways to the solutions [19, 27, 37, 55]. Here, we mean by evolutionary pathways that what, when, how, and why some genes have successfully evolved into the solutions while the
9
1. INTRODUCTION others not. These questions are critical to show the causes of adaptation and evolution. Finally, it is not able to analyze the evolved solutions. Evolution necessarily causes the changes of simple structures into complex ones that may be viewed as emergent behaviors or emergent properties [41, 73]. However, the second framework cannot tell whether certain behaviors are emergent because it does not have any formal definition of emergence.
1.2 Research Objectives In the above discussion, we have identified some problems of the two conventional frameworks and motivated the need for a new soft computing framework to construct a behavior controller for an agent. This section summarizes the research objectives of this dissertation with the details of how these objectives are accomplished deferred to a later chapter:
0
New soft computing framework: The two conventional frameworks described above have some drawbacks, especially in the perspective of evolution. In this dissertation, a soft computing framework is proposed to show the roles and effects of evolutionary algorithms during
0
the evolution of intelligent agents. Quantification of adaptive evolution: In general, it is hard to say that evolution has been adaptive during the evolutionary process. However, without any confirmation that the evolution has been adaptive, there is no guarantee that follow-up experiments will yield the same or similar results as previous ones, even though conditions are the
10
1. INTRODUCTION same. The quantification of adaptability of evolution helps us show more scientifically and objectively that the evolution has been adap-
0
tive during the evolutionary process. Adaptive behaviors: If a certain level of adaptability is maintained, evolution can produce components that have adaptive behaviors. These adaptive components are expected to contribute to the formation of good solutions. Quantification of adaptability will help to show the
0
reasons that some components are more adaptive than others are. Evolutionary pathways to solutions: Illustrating evolutionary pathways to solutions is to show how the solutions are evolved. In other words, it is used to show that the solutions are not from chances or other genetic phenomena but from the adaptations to the environ-
0
ment. Illustration of emergence: Evolution has the capability of creating new complex structures from simple structures. This kind of concept, i.e., the creation of new structures or properties, is called emergence. In natural evolutionary systems, emergence has been continuously created as lives evolve, and no one denies this fact. In artificial evolutionary systems, much work has insisted that behaviors of the evolved systems are emergent without any formal definition of emergence. Consequently, the insistence cannot be commonly accepted. In this dissertation, we analyze the behaviors of evolved agents and show that their behaviors are emergent relying on a formal definition of emergence.
11
1. INTRODUCTION
1.3 Organization of the Dissertation Chapter 2 discusses the results and problems of previous works in the perspective of soft computing frameworks. In Chapter 3, a soft computing framework for agents is proposed and each component of the framework is described in detail. In Chapter 4, a hardware agent system is developed to show the usefulness of the proposed framework. In this chapter, the framework is applied to the construction of a behavior controller for a mobile robot. After evolving the behavior controller, we analyze the evolution and the evolved controller’s behaviors with the framework. In Chapter 5, a software agent system is developed to show the usefulness of the framework in the paradigm of software agents. The software agent, called conversation agent, is to answer to questions from users with natural language interface. It can process both Korean and English. Its role is to introduce a web site to users as a virtual representative. The behavior controller of the conversational agent is evolved by an evolutionary algorithm and analyzed with the framework. We conclude in Chapter 6 by summarizing the contributions of this dissertation and discussing promising directions for further works.
12
CHAPTER 2 RELATED WORKS In this chapter, we discuss related works in topics close to this dissertation. Section 2.1 presents discussions about conventional approaches to the combination of evolutionary algorithms and other soft computing techniques. Here, we discuss mainly about the combination of fuzzy logic and evolutionary algorithms, and the combination of neural networks (NNs) and evolutionary algorithms, because they are the most commonly used types of combination of soft computing techniques. Section 2.2 discusses related work on emergence and measuring adaptive evolution.
2.1 Conventional Soft Computing Frameworks Over the past ten years, an increasing number of hybrid algorithms have been proposed, in which two or more soft computing technologies have been integrated to improve the overall algorithm performance. Many researchers have explored the use of genetic algorithms (GAs)
13
2. RELATED WORKS to tune fuzzy logic controllers (FLC). These methods differ mostly in the order or the selection of the various FLC components that are tuned, i.e., term sets, rules, and scaling factors [11, 31, 37, 38, 49, 53, 54, 55, 60, 69]. Bonarini [11] presented some approaches based on evolutionary reinforcement learning algorithms, which were able to evolve in real-time fuzzy models that control behaviors. He discussed an application where an agent learns how to adapt its behavior to the different behaviors of the other agents it is interacting with, and another application where a group of agents coevolve cooperative behaviors by using explicit communication to propose the cooperation and to distribute reinforcement to the others. Fukuda and Kobota [31] proposed a robotic system with structured intelligence. They focused on a mobile robotic system with a FLC and proposed a sensory network to allow the robot to perceive its environment. An evolutionary algorithm was used to improve the robot’s performance. They showed the effectiveness of the proposed method through computer simulations of collision avoidance and path-planning problems. Herrera et al. [38] directly tuned each rule used by the FLC. They used a real encoding for a four-parameter characterization of a trapezoidal membership value used in the rule antecedent and consequent. The population was the concatenation of all rules. A customized crossover operator was also proposed. The fitness function was a sum of quadratic errors. Kinzel et al. [49] tuned both rules and term sets. They departed from the string representation and used a matrix to encode the rule set. They also proposed customized crossover operators that were similar to the two-point crossover for string encoding. They first initialized the rule base according to intuitive heuristics, used GAs to generate better rule base, and finally
14
2. RELATED WORKS tuned the membership functions of the best rule base. Lee and Takagi [53] tuned the rule base and the term sets. They used a binary encoding for each three-tuple characterizing a triangular membership distribution. Each chromosome represented a Takagi-Sugeno rule, concatenating the membership distributions in the rule antecedent with the polynomial coefficients of the consequent. Liu et al [55] proposed a novel FLC, called linguistic hedge FLC, to simplify the membership function constructions and the rule developments. The design methodology of linguistic hedge FLC was a hybrid model based on the concepts of the linguistic hedges and the GAS. The linguistic hedge operators were used to adjust the shape of the system membership functions dynamically, and ran speed up the control result to fit the system demand. Genetic algorithms were adopted to search the optimal linguistic hedge combination in the linguistic hedge module. Melin and Castillo [60] applied different hybrid approaches for controlling the battery charging process. The hybrid approaches combined soft computing techniques to achieve the goal of controlling the temperature of the battery during the electrochemical charging process. They reduced the time required for charging a battery with the use of fuzzy logic, NNs, and GAs. In the neuro-fuzzy-genetic approach, NNs were used for modeling the electrochemical process, fuzzy logic was used for controlling the process, and GAs were used to optimize the fuzzy system. Seng [69] proposed a neuro-fuzzy logic controller where all of its parameters could be tuned simultaneously by GA. The structure of the controller was based on the radial basis function NN with Gaussian membership functions. They used a flexible position coding strategy of the neuro-
15
2. RELATED WORKS fuzzy logic controller parameters to obtain near optimal solutions. As explained above, many researchers have shown that GAs can tune the FLC parameters to produce appropriate FLC rules, shapes, and centers of fuzzy membership functions. However, they did not show whether their evolutionary processes were adaptive. They also did not show evolutionary pathways to the obtained solutions. There are many forms in which GAs can be used to synthesize or tune neural network (NN): to evolve the network structure, to evolve a set of weights for a given structure, and to evolve the reward function. Encoding both NN structure and parameters requires too large GA chromosome such that the GA cannot search the global space efficiently. Therefore, the above approaches are usually mutually exclusive in most cases. Chen and Miikkulainen [19] applied a NN to music composition to generate melodies. An evolutionary algorithm was used to find a NN that maximizes the chance of generating good melodies. Composition rules on tonality and rhythm were used as a fitness function for the evolution. They observed that the model learns to generate melodies according to these rules with interesting variations. Eccles et al. [25] developed a virtual ecosystem using GAs, artificial NNs, and fuzzy systems. The ecosystem simulated and regulated the motion and interactions of computer animated agents in a virtual environment. Within the ecosystem, each agent had its own NNs that govern its motion strategy so that the creature could learn to perform different behaviors, such as searching for food and evading predators. The inputs of the NNs were connected with creatures’ eye-like sensors and the outputs of the networks were attached to creatures’ legs to drive the creatures. The NNs
16
2. RELATED WORKS were evolved using GAs, including single tournament selection, two-point crossover, and one point mutation. Augmented fuzzy cognitive maps regulated the overall operation of the environment and allowed each creature to switch between different NNs for finding food and avoiding predators. Floreano and Mondada [27] evolved a real, neural-network driven mobile robot. The whole evolutionary process took places entirely on a real robot without human intervention. Although the experiments tackled a simple task of navigation and obstacle avoidance, They showed a number of emergent phenomena that are characteristic of autonomous agents. Hunter and Chiu [43] discussed the design of NN and FLCs using GAs, for real-time control of flows in sewerage networks. The soft controllers operated in a critical control range, with a simple set-point strategy governing ‘easy’ cases. A GA designed controllers and set-points by repeated application of a simulator. They also provided a comparison between NN, fuzzy logic and benchmark controller performance. They concluded that NN and FLCs had comparable performance, though NNs could be successfully optimized more consistently. Tan et al. [77] applied genetic programming to the construction of NNs. They described a genetic programming approach that constructs neural logic networks from training examples. Neural logic network, unlike ordinary networks, allows representation of a great multitude of logic operations used in human decision-making. Their method enabled the construction of the network from scratch, but also preserved the semantic contents of the network after training. The previous researchers discussed so far showed that soft computing techniques can be tuned to produce appropriate solutions for the tasks.
17
2. RELATED WORKS However, they did not show that their evolved solutions had resulted from the adaptabilities to the problems. Furthermore, it is still necessary to reveal the control mechanisms of the behavior controller obtained by evolution to understand how the controller works.
2.2 Analysis of Evolution Recently, the concept of emergence [21, 23, 61] has been focused as a result of a lot of research in non-linear dynamics, artificial life, complex systems, and behavior-based robotics. Emergence of a system, in a broad way, is the properties or behaviors that cannot easily be predicted from its internal properties. Examples of these phenomena are the flocking behavior in simulated birds from a set of three simple steering behaviors [67], some patterns in the game of life [10], and the “highway” pattern by the artificial Langton ant [75]. Despite the broadness of emergence phenomena, there is no unified agreement on what emergence is. One of the first definition on emergence was made in 1923 by Morgan who defined: “emergence is the denomination of something new which could not be predicted from the elements constituting the preceding condition,” “the product is not a mere sum of the separate elements,” or “the creation of new properties” [26, 61]. Bedau proposed weak emergence in contrast to strong emergence [5]. According to his definition, properties or behaviors of a system are weakly emergent if and only if they can be derived from the dynamics that governs the time evolution of the system’s microstates, only by simulation. Ronald proposed a three-step emergence test [68]. The three steps are design, observation, and surprise. He drew an
18
2. RELATED WORKS analogy with the concept of intelligence and the Turing test. Baas [3] defined two emergence: deducible emergence if there is deductional or computational process or theory, and observational emergence if it cannot be deduced. A lot of literature in evolutionary biology, artificial life, and evolutionary computation have revealed that good solutions for a given problem are not automatically obtained just because evolutionary algorithms are used, as there are many evolutionary phenomena such as random genetic drift, necessity, and adaptation that can affect the formation of the solutions [20, 84]. The good solutions obtained might have not resulted from adaptive evolution but from other unexpected evolutionary phenomena. In this case, the same results would not be achieved by follow-up evolutionary experiments if some set-ups such as mapping from genotype to phenotype [34, 78] are not proper. The capacity to produce good solutions via evolution, evolvability, can be defined in various ways as “genome’s ability to produce adaptive variants when acted on by the genetic system” by Wagner and Altenberg [82], “capacity to generate heritable phenotypic variation” by Kirshner and Gerhart [50], or “capacity to create new adaptations, and especially new kinds of adaptations, through the evolutionary process” by Bedau [6]. To obtain good solutions robustly, the evolvability should be maintained during the evolutionary process. Under this condition, good candidate solutions can contribute to the formation of good solutions. A promising way to measure evolvability was proposed by Bedau and Packard [6]. This method measures evolvability by quantifying adaptive phenomena in artificial and natural systems using evolutionary activity statis-
19
2. RELATED WORKS tics. The evolutionary activity statistics and its related diagrams are useful to show evolutionary dynamics and, therefore, to assess the level of evolvability in a system, which is a key step toward making evolutionary systems that are highly evolvable. The statistics has been applied to a variety of evolving systems for different purposes. They applied it to visualizing adaptive evolutionary phenomena [6, 7, 8], studying punctuated equilibrium dynamics in evolution [71], identifying long-term evolutionary trends [56], and classifying evolutionary dynamics [7].
20
CHAPTER 3 SOFT COMPUTING FRAMEWORK This chapter presents a soft computing framework for intelligent agents and describes each component of the framework in detail. We start by describing conventional soft computing techniques and frameworks in section 3.1. Section 3.2 presents the proposed framework and explains it focusing on the augmented analysis part. In subsequent sections, we describe each soft computing technique that is a component of the proposed framework. In section 3.3, we describe rulebased systems briefly and compare the merits and demerits of the system with other soft computing techniques. Next, we describe fuzzy logic as a kind of rule-based system. In section 3.4, we overview evolutionary algorithms and describe genetic algorithms in detail, which is one of the components of our framework. Section 3.5 describes three methods used to analyze the evolution
21
3. SOFT COMPUTING FRAMEWORK and emergent behaviors of intelligent agents: evolutionary activity statistics, observational emergence, and schema analysis.
3.1 Soft Computing Techniques and Frameworks Soft computing encompasses several computational techniques such as neural networks, fuzzy systems, probabilistic reasoning, and evolutionary algorithms. All the techniques have in common departure from classical reasoning and modeling approaches that are usually based on Boolean logic, crisp classifications, and deterministic search. In this section, we describe soft computing techniques briefly and discuss their merits and demerits. We also describe conventional soft computing frameworks and their shortcomings.
3.1.1 Comparison of Soft Computing Techniques Soft computing techniques are particularly effective in search and optimization when the underlying search space is large, multimodal, and when the characteristics of the search space are not well understood. It is under these conditions that classical reasoning and modeling approaches become impractical [12]. Fuzzy logic approaches are based on a fuzzy-valued representation of uncertainty and imprecision. Typically, they use linguistic variables to represent different information granularities, and triangular-norms to propagate the fuzzy boundaries of such granules. The basic mechanism used in fuzzy reasoning systems for inference is the generalized modus-ponens [92].
22
3. SOFT COMPUTING FRAMEWORK Fuzzy logic translates and embeds empirical, qualitative knowledge about the problem to be solved into reasoning systems that can perform approximate pattern matching and interpolation. Though fuzzy logic has advantages in that expert knowledge can be easily represented and, therefore, understandable, it does not have adaptation or learning features, since it lacks the mechanism to extract knowledge from existing data. Therefore, fuzzy logic is difficult to apply when expert knowledge of a give problem is insufficient. On the other hand, neural network has learning capability as one of its typical characteristics. With enough training data, the weights inside the neural network are adjusted by comparing the output of the network to the desired answer. Because of this training capability, neural networks have been applied to many problems such as pattern classification, pattern categorization, and control of vehicles. However, neural networks also have drawbacks. First, training neural networks is not straightforward and can be tricky. It requires much time and effort because no optimal solution exists in determining the internal parameters of the neural network. Second, the trained neural network is usually a kind of black-box. In other words, it is hard to understand just why the trained neural network works. Finally, neural network must have training data. In addition, test data are also required if the performance of the trained neural networks are to be measured. Another soft computing technique is probabilistic reasoning. Probabilistic reasoning such as Bayesian networks [4, 64, 87] and DempsterShafer’s theory of belief [24, 70] gives us the mechanism to evaluate the outcome of systems affected by randomness or other types of probabilistic uncertainty. Its main characteristic is its ability to update previous outcome
23
3. SOFT COMPUTING FRAMEWORK estimates by conditioning them with newly available evidence. The probabilistic reasoning methods can work in an unsupervised mode in contrast to neural networks that should be trained before it is used. Therefore, there is no need to have a set of data to train on. The methods, along with techniques like clustering that reduce the data to essentials for easier classification, identify patterns in raw data. That is particularly important when attempting to root out something like Medicare fraud and abuse from a stack of claims. However, probabilistic reasoning methods have also drawbacks in that the probabilistic relationships among variables should be extracted from data set. This might require too much time if the data set is too large. The last soft computing technique is evolutionary algorithms, especially genetic algorithms in this dissertation. GAs, proposed by Holland [41], give us a way to perform randomized global search in a solution space. In this sense, a population of candidate solutions, encoded as chromosomes, is evaluated by a fitness function in terms of its performance. The best candidates evolve and pass some of their characteristics to their offsprings. Soft computing, of course, has disadvantages. A soft computing technique is not appropriate for some problems. For all problems, there is the issue that soft computing techniques find solutions that are good enough but perhaps not the best. There is always a chance that a better solution exists elsewhere, but it is also always true that if it exists, a good enough solution eventually can be found. Furthermore, soft computing techniques tend to be nonlinear ones. If the size of a problem doubles, then the size of the solution and the amount of computation needed to find it become more than doubles. Frequently the size of the solution and the amount of computation required to ferret it out would quadruple or increase even more.
24
3. SOFT COMPUTING FRAMEWORK Behavior Controller
Evolutionary Algorithm
Fig. 3.1: Conventional soft computing framework.
3.1.2 Conventional Soft Computing Frameworks As described earlier, two main frameworks are most commonly used in applying soft computing techniques to the construction of intelligent agents. The first framework is to apply each soft computing technique to the construction of intelligent agents. This approach has advantages in its methodological simplicity and facilities in the construction if each technique is applied to a relatively simple and tiny problem. However, it has some drawbacks in determining the internal parameters of the techniques because they often require expert-level knowledge and need more time and effort. The second framework is to combine two or more soft computing techniques to relieve the difficulties of determining the internal parameters. Especially, the combination of an evolutionary algorithm and other soft computing technique has been focused very much as in Fig. 3.1. The behavior controller in Fig. 3.1 is usually implemented using soft computing techniques, e.g., neural networks, fuzzy logic, probabilistic reasoning, etc. Several evolutionary algorithms are available for the evolution of the behavior controller. The evolutionary algorithms tune the internal parameters of the behavior controller through evolution. This framework has advantages in that it
25
3. SOFT COMPUTING FRAMEWORK requires relatively less expert-level knowledge and expertise and it can find behavior controllers that show better performance because of the global searching capabilities of the evolutionary algorithms. Despite the good results of the combination of soft computing techniques and evolutionary algorithm, this framework has at least three shortcomings. First, the second framework cannot guarantee that the solutions have resulted from adaptive evolution. Because, in most artificial evolutions, adaptive evolution is the key to produce successful results, the inability to guarantee adaptive evolution might cause serious problem. For example, suppose you just have succeeded in getting scientifically very important data from the last experiment of evolution, and somebody just happen to delete your data by mistake when you are away just a few seconds. When you realize this situation, you might think that that is ok and just one more experiment will suffice to get the same results. However, such a happy situation (successful evolution) would not occur that often unless evolution is adaptive. High probability Desirable Evolutionary Causes and Effects
High Evolvability
Low probability
Good Solution
NonAdaptive Evolution
Adaptive Evolution
Bad Solution
Fig. 3.2: Possible evolutionary pathways.
26
Low Evolvability
3. SOFT COMPUTING FRAMEWORK In general, evolutionary phenomena contain not only adaptive evolution but also others such as chances and necessity. Therefore, good solutions do not necessarily mean that the evolution has been adaptive (see Fig. 3.2). In other words, if the solutions are not from adaptive evolution but from other evolutionary phenomena, then no good solution from followup experiments may be obtained even in the same experimental conditions. Therefore, proving or showing that the solution is the result of adaptive evolution has great importance in the justification of the experiment and the evolution. Second, the second framework cannot show evolutionary pathways to the evolved solutions. Here, evolutionary pathways means that what, when, how, and why some genes have successfully evolved into the solutions while the others not. Lastly, the second framework lacks methods for analyzing the obtained solutions. Evolution necessarily causes the changes of simple structures into complex ones that may be viewed as emergent behaviors or emergent properties. However, the second framework cannot analyze these aspects.
3.2 Soft Computing Framework for Intelligent Agents In this section, a soft computing framework for intelligent agents is proposed. Detailed description of the components of the framework will be presented in the following sections. Though conventional frameworks can evolve intelligent agents of good performance, they lack some analytical
27
3. SOFT COMPUTING FRAMEWORK Research goals
How to achieve the goals
What we can get or show
Rule-Based System
Agent Construction
Hardware Agent
Software Agent
Evolutionary Algorithms
Evolution Analysis
Evolutionary Activity Statistics
Adaptive Evolution
Adaptive Behavior
Schema Anaysis
Behavior Analysis
Analysis of Evolution
Emergence
Evolutionary Pathways to Solutions
Observational Emergence
Fig. 3.3: Proposed soft computing framework for agents. points. This framework augments the conventional framework of the combination of soft computing techniques and genetic algorithms by rule-based systems and several analysis methods as in Fig. 3.3. Rule-based systems are used for behavior controllers because they are easy to analyze when combined with the augmented analysis methods, and understandable to humans. As a matter of course, evolutionary algorithms tune the rules in the behavior controllers. Especially, an emphasis is put on the evolution and be-
28
3. SOFT COMPUTING FRAMEWORK havior analysis to overcome the shortcomings of the previous framework. The analysis part in Fig. 3.3 is adopted to overcome the shortcomings of the conventional frameworks as in the followings: 1. Evolution Analysis
0
Quantification of Adaptive Evolution The conventional frameworks cannot convince that the evolution of intelligent agents has been adaptive because several evolutionary phenomena exist including adaptation. Therefore, there are needs for certain methods for measuring or observing adaptive evolution. Furthermore, they should be quantitative to be used as objective measures. In the proposed framework, evolutionary activity statistics measures the adaptability of evolution
0
objectively and quantitatively. Identification of Adaptive Genes Identifying adaptive genes is important in that those genes are clues to uncover the secret of the evolutionary construction path-
0
ways to solutions, and to understand what behavioral properties are adaptive in the problems. Evolutionary Pathways to Solutions In the conventional frameworks, serious analysis on the evolutionary pathways to solutions has not been done while much attention has been paid to the construction of behavior controllers of good performance. However, it is necessary to analyze the construction pathways to solutions in order to show that the so-
29
3. SOFT COMPUTING FRAMEWORK lutions are from adaptive evolution rather than from other evolutionary phenomena. 2. Behavior Analysis Evolution is said to generate complex structures from simple structures. A complex structure is emergent in that it cannot be expected with the simple structures that compose the complex structure. Decisions on whether certain behaviors are emergent should be made relying on formal definition of emergence. We adopt the notion of “observational emergence” [3] to judge whether an agent’s behavior is emergent.
3.3 Rule-based Systems Rule-based systems, especially fuzzy logic, are composed of rules based on which the systems make decisions on which behavior to take. The rules take the form of “IF condition THEN action.” Usually, the rules can be represented as linguistic expressions such that importing expert’s knowledge into the systems is very easy compared to other soft computing techniques. Although rule-based systems have advantages in that expert knowledge can be easily represented and, therefore, easily understood by humans, it does not have adaptation or learning features, since it lacks the mechanism to extract knowledge from existing data. We adopt rule-based systems as the controller for two reasons: First, expert knowledge can be easily incorporated into the systems and represented in verbal forms. This implies that it is pretty much easier to analyze
30
3. SOFT COMPUTING FRAMEWORK Rule 0 a is A 1
b is B 1
Fuzzy
b is B 2
Fuzzy
b is B n
Fuzzy
Rule 1 a is A 2
Aggregator
Fuzzy
Defuzzifier
Crisp
b
...
a
Rule n a is A n
Fig. 3.4: Block diagram of a fuzzy controller. the rule-based systems. Second, even when combined with evolutionary algorithms, the degree of facility in analytic perspective does not change so that the systems can be easily analyzed to reveal what, how, when, and why the rules have evolved successfully or not. These aspects of facility of analysis match best with the analysis methods adopted in the framework to remedy the drawbacks of the conventional frameworks.
3.3.1 Fuzzy Logic The basic structure of a fuzzy logic controller consists of three conceptual components: fuzzification of the input-output variables, a rule base that contains a set of fuzzy rules, and a reasoning mechanism that performs the inference procedure on the rules and given facts to derive a reasonable output. An example of a fuzzy inference system with crisp output is in Fig. 3.4.
31
3. SOFT COMPUTING FRAMEWORK
3.3.2 Fuzzy Sets and Rule Base For a clearer explanation, assume that the fuzzy logic controller uses
132# 465879;: be input linguistic variables defined in the universe of discourse @BA to represent the value of the 4 th variable. A fuzzy set C2 on 1D2 is defined as follows: the eight input variables and two output variables. Let
E2F5HGJILKMFNFO$1,2 QPR1,2#= The integration sign in (3.1) stands for the union of
(3.1)
132#MFNFO$1,2
pairs,
and they do not indicate summation or integration. Similarly, “/” is only
S &# | :l7 ×", respectively.
, we set hÚ5 ¢ 5 9 = | :l7
¢ 5 3=! | and 9 5
With these values, we can calculate new activity as in Fig. 4.11 (d). The
new activity values of the fuzzy model is higher than that of the neutral shadow, which means that new activity signaling the positive adaptability flows more frequently into the fuzzy model compared to that of the neutral shadow. Therefore, we can conclude from this figure that the fuzzy model continuously generates new adaptive fuzzy rules much more than the neutral shadow does.
4.5.2 Adaptive Behaviors Fig. 4.13 shows 29 salient waves that have pretty much higher activities than other waves do. The birth and death of a rule can be seen in this
figure. For example, S occurs before generation 100 and persists for 200 generations. To find out what properties of the salient rules allow them to
71
4. APPLICATION TO A HARDWARE AGENT 6000
S
1
5000
Activity
4000
3000 S
4
S
S3
29
2000
S
2
S
5
1000
0
0
100
200
300
400 500 Generation
600
700
800
900
Fig. 4.13: Evolutionary activities of salient waves. persist longer than other rules, some of them are decoded as in Table 4.2.
0
Analysis of Salient Rule S : In order to identify what behavioral
properties make this rule have high evolutionary activity value, an experi-
ment has been conducted. Fig. 4.14 (a) shows the behaviors of S , Fig. 4.14 (b) shows speed changes during the simulation, and Fig. 4.14 (c) shows the
activations of S . As can be seen, S drives the robot to move forward when there is no obstacle. As this behavior is important for the robot to move in
open space, S has been explored at the beginning stage of the evolution. However, it is interesting that it checks robot.
0
1/
sensor located in the back of the
Analysis of Salient Rule S : To test the behavioral properties of S , it
is appended to the previous rule set, which has just one rule, S . Therefore,
the robot now has two rules, S and S as in Table 4.3. Fig. 4.15 (a) shows
the behaviors of S combined with S , Fig. 4.15 (b) shows speed changes
72
4. APPLICATION TO A HARDWARE AGENT
(a) Behaviors of S ( . 0.06
10 Left Motor
0 0
5
10
15
20
25
30
10 Right Motor
5
0 0
5
10
15 Step
20
25
0.05 0.04 Activation
Speed
5
0.03 0.02 0.01 0 0
30
(b) Speed changes.
5
10
15 Step
20
(c) Rule activation.
Fig. 4.14: Behaviors of S and corresponding data.
73
25
30
4. APPLICATION TO A HARDWARE AGENT Table 4.2: Salient rules. Rule S S
S S S
} %
klklk S* )
Gene Code Meaning 0001010001030411441 ( F) ( FF) and ( 0400010114000110211 ( VC) and ( VF) ( S) and ( B) 0300000410130104031 ( VF) and ( C) ( BF) and ( F) 0003120304010102401 ( M) ( FF) and ( 0411001112140400311 ( F) and ( F) and ( and ( VC) ( F) and ( B)
klklk
1311000003040411021
1 / 5 SRht5 S Ó ½ ~ O Zr : r É º Á Á % s, j ], Q #G b >, ª Õ o ¦ = Q # " » Ä H ~ % ar É KÐ o
¦ r É כ t þ [r É ª ÕQ tw l 3
H t¦ \Ð s tw l 3Ç ô . s 9 |H ët þ [r É h &x £ 6 õ o _¶ é " ¦ `Ð H h & \f "æ ×¹ כ
. t }¼ ÜÐ lr > _Ó ½ ~ O Zr : r É o a ) K \/ @Ç ôr ì 3 $
128
¦ `º à 'É + ½º à O \ . 9 h &¼ ÜÐ o H ß éH íÇ ô¨ ½¸ \f "Ó ½ Ï 1 µ h & 'x l 1 s ¤ £ :
í $¼ ÜÐ c ¨ |º à eH ¤ 4 ú ¸Ç ô¨ ½¸ Ð _ o ¦ \í AÇ ô . ª Õ Q , lr > _Ó ½ ~ O Zr : r É Ó ½ Ï 1 µ
í $ \/ @Ç ô 3 % Ç ô
ñ & _ O \ 3 % l: MH ë \ Q # " 'x l 1 sÓ ½ Ï 1 µ h & t #Ò Â¦ \¨ ½r ìÉ + ½º à O \ 3 % . sH 7 H ë \f "H lr > Ó ½ ~ O Zr : _ ß é h &¦ `G F¤ 4
lA 0
# tx p 0A þ + \ s Ô à¦ \A 0Ç ôè Ô áÔ à (Ó ÉA hÓ ½ ~ O Zr : ¦ `j ] ß îÇ ô . sÓ ½ ~ O Zr : r É ¨ ½¤ ¡ » õ r ì 3 $ s H º ¿Ò Âr ì¼ ÜÐ sÒ ÀQ #R 4 e . ¨ ½¤ ¡ »Ò Âr ì \f "H © ½: Ë g lÍ ø ì r ¼ Û 7 % õ o ú ·¦ o§ £ 7¦ ` Ë + ½
# tx p 0A þ + \ s Ô à¦ \¨ ½¤ ¡ »Ç ô . r ì 3 $ Ò Âr ì \f "H o ¸ Ö x l 1
í $, ¼ Û v r ì 3 $, ª Õ o¦ a ' Ï 1 ¹ h &Ó ½ Ï 1 µ¦ ` h &x 6
# o < ü 'x l 1 h &a ' h & \f " o a ) \ s Ô à¦ \r ì 3 $Ç ô . o ¸ Ö x l 1
í $x : >H h &x £ 6 h & o ¦ \º Ã u h &¼ ÜÐ ¤ £ 8
ñ &
lA 0Ç ôÓ ½ ~ O Z¼ ÜÐ f " o h &x £ 6 h & t t¦ \o Ìa ' h & s¦ º à u h &¼ ÜÐ Í ø ó > Z É + ½º à e . ¼ Û v r ì 3 $r É j þx á 7 KÐ H o õ
ñ & _r ì 3 $ \ sx 6& ÷ 9a ' Ï 1 ¹ h &Ó ½ Ï 1 µr É o a ) j ]Q # l _Ó ½ Ï 1 µ h & 'x l 1t þ [¦ `ß 1 µ yH < X h &x 6a ) . r : H 7 H ë \f "H j ] ß îa ) Ô áU Y e> 0¼ ß _» Äx 6
í $¦ ` ú · Ð lA 0
# \ s Ô à ¨ ½ _º ¿ t Ò Å¹ כì ⧠£ 2 è Ô áÔ à? JQ # \ s Ô à< ü
¼ ×? JQ # \ s Ô à _º ¿ t V Y \ h &x 6Ç ô . Í ' : P V Y ¨ ½ \f "H j ] ß îÇ ôÔ áU Y e> 0¼ ߦ \
¼ ×? JQ # \ s Ô à ¦ \A 0Ç ô 'x l 1j ]Q # l _¨ ½¤ ¡ » \ h &x 6Ç ô . s V Y ¨ ½ \f " x 6Ç ô
¼ ×? JQ # \ s Ô àÐ H HÔ á ¦
H 9 {x á 7 _ 'x l 1 lÍ ø ì sx l 1Ð © 4¦ ` sx 6
i % . HÔ á _ # = Wh > o y tp ' Gf "כ ú ° s 'x l 1j ]Q # l _ 9 { 4 §¼ ÜÐ sx 6& ÷¦ sÐ Ò Â ' r : º ¿ t j ]Q #כ ú ° sÐ © 4 _º ¿¸ '¦ \ ¨ ½x l 1
lA 0
# sx 6a ) . 'x l 1j ]Q # lÐ H © ½: Ë g lÍ ø ì r¼ Û 7 % _ 9 {x á 7 ( tj ]Q # l¦ \ sx 6
¦ » Ä ú ·¦ o§ £ 7¦ ` sx 6
# ( t 'x l 1j ] Q # l _/ ?Ò Â p '¦ \¸
ñ &Ç ô . j ] ß îÇ ôÔ áU Y e> 0¼ ߦ \ sx 6
#
129
o a ) j ]Q # l \/ @Ç ô o 9 x 'x l 1r ì 3 $¦ `º à 'Ç ô . º ¿ : P V Y ¨ ½ \f "H j ] ß îÇ ôÔ áU Y e> 0¼ ߦ \è Ô áÔ à? JQ # \ s Ô à¦ \A 0Ç ô 'x l 1j ]Q # l _¨ ½¤ ¡ » \ h &x 6Ç ô . s V Y ¨ ½ \f " sx 6& ÷ H è Ô áÔ à? JQ # \ s Ô àH Û / R sÔ à _ ©/ @ o Ð f " Q #¦ \ s x 6
#Ó ½ ~H ë < ü © ñ x 6¦ `º Ã '
H / @o A þ + \ s Ô à s . s/ @o A þ + \ s Ô àH o 'r ìÓ À, ¨ ½¸ h & J BA, g ª Õ o¦ t d¨ ½¤ ¡ » õð ³³ & _j [t Ò Å¹ ¨כ ½
í $¹ כè Ð sÒ ÀQ #R 4 e¼ Ü9 x 6 _ _¸ ¦ \
lA 0
#¸ Ð ¦ \ sx 6
# 9 { º _ v> 0¼ ×t þ [¦ ` dÊ < ¼ ÜÐ +o '¦ `r ìÓ ÀÇ ô . ¸ ¢Ç ôÐ ò ´¦ Ö h & o 'r ìÓ À¦ \A 0
#o 'r ìÓ À¸ t ý Ñ s _ © ñ x 6¦ `í O [¨ ½¸ ¦ \ sx 6
#j ]Q #Ç ô . x 6 _ 9 | _ ¦ \ t d< X s '
Z s¼ Û \ eH ú ² õ BA g lA 0 Kf " sH 7 H ë \f " H x : h & Q # % o l O ZÐ H J BA g l O Z¦ ` h &x 6
i % . º ¿ t V Y ¨ ½¦ \x :
#j ] ß îÇ ôÔ áU Y e> 0¼ ß tx p 0A þ + \ s Ô à¦ \A 0Ç ôj ]Q # l _¨ ½¤ ¡ » \ò ´¦ Ö h &¼ ÜÐ h &x 6c ¨ |º à e§ £ 6¦ ` X S
i % . ¸ ¢Ç ô o r ì 3 $¦ `x :
#j þx á 7 K º Ä s r É o ³ & © _ ß í t ü Ó s m y¨ ½
í $¹ כè _ h &x £ 6
í $ \f " q© 2a ) ß ít ü Ó e¦ ` X S É + ½º à e 3 % . o a ) j ]Q # l \/ @Ç ô 'x l 1r ì 3 $¦ `x :
#j ]Q # l o õ
ñ & \f "Ó ½ Ï 1 µ h & 'x l 1¦ ` \ Sq p 1
i %¼ Ü9, ¤ £ : y sQ Ç ôÓ ½ Ï 1 µ
í $ sÐ ú ± r É ß é > _¨ ½
í $¹ כè t þ [ _ © ñ x 6 \ _Ç ô כ e¦ ` ú ·º à e 3 % . r : h &¼ ÜÐ , j ] ß îÇ ôÔ áU Y e> 0¼ ß \ s Ô à _j ]Q # l¦ \¨ ½¤ ¡ »
H < Xò ´ õ h &¼ ÜÐ h &x 6c ¨ |º Ã e§ £ 6¦ ` ú ·º Ã e 3 %¼ Ü9, ¤ £ : y o õ¨ ½
í $ a ) j ]Q # l _ 'x l 1r ì 3 $ õ o õ
ñ & \/ @Ç ôr ì 3 $ \ © h &¦ ` t m¦ e§ £ 6 ¦ ` ú ·º Ã e 3 % .
þ Ù d& ÷H ú ´: h &x £ 6
í $, / @o A þ + \ s Ô à, o ¸ Ö x l 1
í $x : >, o ú ·¦
130
o§ £ 7, ( tH 7 o, tx p 0A þ + \ s Ô à, sx l 1Ð © 4, a ' Ï 1 ¹ h &Ó ½ Ï 1 µ, ¼ Û v , è Ô áÔ à (Ó ÉA hÔ áU Y e> 0¼ ß.
131
