for a moment and attempt to describe the same scene without using the word âavoidâ ..... minimal understanding of such a metaphor and will be able to add ...
METAPHOR BASED REASONING FOR SOFTWARE AGENTS AND HETEROGENEOUS ROBOTS USING BODY BASED FEELING SEQUENCES
BY ERIC BERKOWITZ
Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate College of the Illinois Institute of Technology
Approved Advisor
Chicago, Illinois December 2000
© Copyright by Eric Berkowitz 2000
ii
ACKNOWLEDGMENTS
iii
TABLE OF CONTENTS
Page ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix CHAPTER I. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 A Small Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 II. A DEEPER LOOK AT METAPHOR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.1 Another Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Overview of MIND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.3 Metaphors in Perception and Reasoning . . . . . . . . . . . . . . . . . . . 10 2.3.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.3.2 The Human Implementation . . . . . . . . . . . . . . . . . . . . . . . . 13 2.3.3 The Computer Implementation . . . . . . . . . . . . . . . . . . . . . . 15 2.4 How This Model Can Helps Us Communicate with a Computer or Software Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.5 Metaphor and Detail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.5.1 The Motivation for Replacing Actual Detail with Metaphorical Detail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.5.2 Shared Detail Required For Communication/ Specific Detail not Useful . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 III. A MODEL OF COGNITION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.1 Reasoning with Metaphor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.2 Using Feeling Sequences as a Mapping Mechanism in the Categorization of Natural Language Discourse . . . . . . . . . . . . . . 24 3.3 Our Ability to Process Spatial Information at the Instinctive, Biological Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
iv
CHAPTER
Page
IV. OTHER RELEVANT WORKS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.1 Other Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.2 Limitations of These Approaches . . . . . . . . . . . . . . . . . . . . . . . . 35 4.2.1 Isolation Language from Reasoning . . . . . . . . . . . . . . . . . . 35 4.2.2 The Advantages of Hierarchical Structures . . . . . . . . . . . . 35 4.2.3 The Shortcomings of a Lexical Approach . . . . . . . . . . . . . 36 V. MORE EXAMPLES OF MIND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 5.1 Another Look at the “Assertion out of the Argument” . . . . . . . . 37 5.2 Finding Expected Continuations . . . . . . . . . . . . . . . . . . . . . . . . . 45 VI. THE MODEL OF MIND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 6.1 System Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 6.2 The Semantic Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 6.3 The Feeling Based Lexicon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 6.4 The Memory Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 6.5 The Metaphor Hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 6.6 The Lexical Sequence Network . . . . . . . . . . . . . . . . . . . . . . . . . . 56 6.7 The Feeling Sequence Network . . . . . . . . . . . . . . . . . . . . . . . . . . 59 6.8 The Memory Sequence Network . . . . . . . . . . . . . . . . . . . . . . . . . 61
v
CHAPTER
Page
VII. MIND’S INTERNALS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 7.1 Mental Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 7.2 Objects and Parts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 7.3 Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 7.4 Placing an Image in a Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 7.5 Finding Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 7.6 Moving Objects in a Mental Space . . . . . . . . . . . . . . . . . . . . . . . 70 7.6.1 Mass, Counterforce and the Laws of Physics . . . . . . . . . . . 70 7.6.2 Friction and Pressure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 7.6.3 Can Bend? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 7.6.4 Moving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 7.6.5 Transposing Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 7.7 Propagating Force . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 7.7.1 Forces and Obstacles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 7.7.2 Forces and Containers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 7.8 Feeling Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 7.9 Feeling Collection and Filtering . . . . . . . . . . . . . . . . . . . . . . . . . 77 7.10 The Focal Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 7.11 Refining Metaphorical Abstractions . . . . . . . . . . . . . . . . . . . . . 78 7.12 Tiered Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 VIII. THE METAPHORS MIND USES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 8.1 Metaphor Classifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 8.2 Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 8.3 Container . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 8.4 Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 8.5 Force . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 8.6 Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 8.7 Change and Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 8.8 Hierarchical Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 8.9 The Metaphor Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 8.10 Inclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 8.11 Feelings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 IX. BUILDING HIGHER LEVEL CONSTRUCTS . . . . . . . . . . . . . . . . . . . . . . 91
CHAPTER
9.1 Higher Level Constructs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 9.2 Perceptual and Lexical Metaphors . . . . . . . . . . . . . . . . . . . . . . . . 92 Page
X. A FEELING BASED LEXICON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 vi
10.1 The Lexicon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 10.2 Using The Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 10.3 The Semantic Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 10.4 The Entry and Its Direct Links . . . . . . . . . . . . . . . . . . . . . . . . . . 97 10.5 The Indirect Links and Propagating Activation of Feelings . . . 99 10.7 Comparing the Activation of In to Out . . . . . . . . . . . . . . . . . . 101 10.8 Resultant Mental Imagery and Filtering . . . . . . . . . . . . . . . . . . 102 XI. THE METAPHOR NETWORK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 11.1 Overview of Representing Metaphors . . . . . . . . . . . . . . . . . . . 106 11.2 Noun (Object) Metaphors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 11.3 Real Nouns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 11.4 Preposition (Relationship) Metaphors . . . . . . . . . . . . . . . . . . . 110 11.5 Real Prepositions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 11.6 Verb Metaphors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 11.7 Real Verbs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 11.8 Third Hierarchies for the Nouns and Verbs . . . . . . . . . . . . . . . 114 XII. MOP MEMORY AND THE BEAUTY OF A CLOSED SYSTEM . . . . . . 116 12.1 Memory Organization Packets . . . . . . . . . . . . . . . . . . . . . . . . . 116 12.2 Adaptations and Changes for MIND . . . . . . . . . . . . . . . . . . . . 119 12.3 Two Tiered Cluster Mops . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 XIII. ON THE NATURE OF INHERITANCE . . . . . . . . . . . . . . . . . . . . . . . . . . 126 13.1 The Standard View of Inheritance . . . . . . . . . . . . . . . . . . . . . . 126 13.2 The Inverted View of Inheritance . . . . . . . . . . . . . . . . . . . . . . 126 13.3 Defining Generality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 13.4 Implications for Inheritance Design . . . . . . . . . . . . . . . . . . . . . 127 13.5 MOPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 13.6 Inheritance and MOP memory . . . . . . . . . . . . . . . . . . . . . . . . . 129 13.7 The Definition of Specialized . . . . . . . . . . . . . . . . . . . . . . . . . 131 13.8 Placement Significance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
vii
CHAPTER
Page
XIV. CONCLUSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 14.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 14.2 Significance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 14.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 APPENDIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
viii
LIST OF FIGURES
Figure
Page
1.
Processing “I will leave it out of the argument” . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.
Basic Diagram of Mind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.
“I will leave it out of the argument” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.
Memory of Outside . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.
High Level Memory of Outside . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
6.
Memory of Argument Implication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
7.
High Level Memory of Container with Content . . . . . . . . . . . . . . . . . . . . . . . . . 42
8.
Segment of Mental Imagery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
9.
Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
10. Memory of Noun with Force . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 11. Expected Continuation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 12. Full Diagram of MIND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 13. The Semantic Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 14. The Lexical Sequence Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 15. The Feeling Sequence Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 16. Memory Sequence Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 17. Container With Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 18. Container with Content After Placement in a Space . . . . . . . . . . . . . . . . . . . . . . . . 67 19. Can Bend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Figure
Page
ix
20. Obstructed Bending . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 21. Forces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 22. Inclusion in the Metaphor Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 23. MIND’s Metaphor Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 24. Feelings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 25. Region of Accessibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 26. Feeling Based Definition of blocked . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 27. Network Entry Points for In . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 28. Result of Activating In . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 29. Result of Activating Out . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 30. Entry Points for Argument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 31. The Intersection of the Result of Activating in with the Result of Activating Container . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 32. Forces In and Out of a Container . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 33. Portrayal of Consumption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 34. Examples of Using the Image Hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
x
1 CHAPTER I INTRODUCTION 1.1 Problem Statement Driving down State Street toward the Illinois Institute of Tech nol ogy, a student swerves abruptly to avoid a pothole in the recently dug-up pavement. At the same time, hundreds of miles away, a candidate for the United States Senate is attempting to dodge a question to avoid offending part of her consti tuency. These two scenarios appear to have nothing at all in common. What is therefore of great interest is the fact that both descriptions use the word “avoid” to describe the subject’s action. As will be described in this work this use of an identical word is not a random occurrence, but is at the heart of how we communicate and reason. The first scenario in which the student is driving down the street is an obvious case of avoidance in which the student changes the trajectory of the automobile so as not to impact a threatening object. The scene with the political candidate however, is not such a clear cut case of “avoidance.” The would-be senator neither moves, nor has anything to physical avert. Still, the word “avoid” makes perfect sense in this context, and, in fact, conveys an accurate and complete understanding of the situation. The reason for this is that despite the apparent differences, we perceive both situations presented here as cases of avoidance. The first scenario is an obvious case of avoidance, and the second scenario being more abstract, is translated, in our minds, into one where a person needs to actually move to avoid a physical object. Our perception of abstract concepts in our world is based
2 on concepts of space and force that are grounded in our own bodies and sense of physical existence. The senatorial candidate can dodge embarrassing questions and deflect criticism in order to rise in the polls and stay on the track to the Senate. It is through these metaphors, grounded in our own sense of physical existence and our comprehension of the working of a physical universe, that we perceive and reason about even the most abstract concepts. It is also the only natural way to communicate a description of what took place to any other person. The use of the term “avoid” with regard to the action of the candidate is the natural means of perceiving and reasoning about the situation. One need only stop for a moment and attempt to describe the same scene without using the word “avoid” or a synonym. Any other method of description requires explicit, verbose mention of details and still never reaches the level of completeness achieved by using a word that on its face, has no relevance here. The discussion of the situation at hand is elevated to the level of an experience common to both the speaker and the listener. Since the speaker has no direct knowledge of the listener, the only background the speaker can confidently rely on having in common is that set which involves bodily experience. By describing the candidates actions in this way, the speaker conveys his or her understanding of what happened and gives the listener an ability to appreciate the events even if the listener has a limitted understanding of the world of politics. This elevation of the description of a scenario to a body-based, force-based description brings with it an advantage beyond the ability to communicate. It allows us to reason about the situation using the knowledge we posses about the body-based
3 description even with no knowledge about the actual situation. The candidate may want to steer the conversation away from the offending issue or bring the conversation to a halt. This sounds like the description of the student in the car and, indeed, relies on the same physical portrayal. 1.2 A Small Example Two people are debating. One makes a sweeping statement to which the other person responds that there is no way to test the validity of such a statement. “Fine,” the first one begins, conceding his partner’s point with “I will leave the assertion out of the argument.” This perfectly acceptable response should allow the partners to the debate to continue without further discussion of the offending statement. The response, however, fails an attempt at literal analysis. The argument is an abstract concept, not a container allowing objects to be inside or outside. The assertion, for its part, is also an abstract concept, and has no physical properties that would allow it to be in or out of anything. Last ly, even if there is some meaning to an assertion being out of an argument, so what? Why does this statement allow the debate to continue as if the offending assertion had never been uttered? Answers to these questions obviously exist since the conversation, as presented, is a completely normal and acceptable segment of human discourse. The answers reveal themselves in the manner in which this statement is handled by the system presented in this dissertation. Figure one is a segment of output showing, in a verbose manner, how the statement “I will leave the assertion out of the argument” is handled.
4 Lines one through nine show the system reading the sentence. Line 13 contains information grouped in parentheses. These groupings demonstrate that the system indeed understands that the argument is being referred to as a container and the assertion referred to as an object. Focusing now on lines 20 through 36, attention should be paid to the terms “propagating-force” and “impacted” used in the analysis of the sentence. The analysis culminates with lines 37 and 43 that state “expectation affected focal not met” and the fact that the conclusion is the focal object. Leaving the assertion out of the argument, is understood in terms of forces, objects and impact. The answer to the question “so what?” posed earlier is found in lines 37, 42, and 43. Leaving the assertion out of the argument contradicts an expectation that the focal object will be affected and the focal object is the conclusion. Since the conclusion is not affected when the assertion is left out of the argument, the debate may continue without focusing on this particular assertion. As will be described in the following sections, people indeed reason in this way. This work will not attempt to discern the many possible evolutionary reasons the human mind works in this way. What will be presented are some of the benefits of this form of reasoning both for people and for machine intelligence designed to work in this manner. This dissertation presents my research in the development of a computer reasoning system exploiting body-based reasoning. Such a system can allow a computer to communicate with people using their own natural, body-based description of actions and events. It can also allow a software agent or robot to perceive the world in terms of its own body, to mimic our own form of ad hoc reasoning about new situations using appropriately selected body-based knowledge,
5 and to communicate with other agents, robots and people about which it knows nothing, relying only on the implicit assumption, built into its form of reasoning, that we all share common physical experiences.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43
Reading Reading Reading Reading Reading Reading Reading Reading Reading
I WILL LEAVE THE ASSERTION OUT OF THE ARGUMENT
("Pred" (NIL NIL M-OUTSIDE 2) "Base" M-OUTSIDE) Activating M-OUTSIDE.42 (M-OUTSIDE.42 ((CONTAINER ARGUMENT) (OBJECT ASSERTION)) ((OBJECT ASSERTION) (CONTAINER ARGUMENT) (FEELINGS M-FEELINGGROUP.31))) ("FRD->" 0 A FOUND NIL) ((A . FOUND)) ("FRD->" 0 D FOUND NIL) ((D . FOUND) (A . FOUND)) ("FRD->" 1 A3 IMPACTED (PROPOGATING-FORCE)) ((A3 . IMPACTED) (D . FOUND) (A . FOUND)) ("FRD->" 1 A2 IMPACTED (PROPOGATING-FORCE)) ((A2 . IMPACTED) (A3 . IMPACTED) (D . FOUND) (A . FOUND)) ("FRD->" 1 A2 IMPACTED (PROPOGATING-FORCE)) ((A2 . IMPACTED) (A2 . IMPACTED) (A3 . IMPACTED) (D . FOUND) (A . FOUND)) ("FRD->" 2 A2 IMPACTED (PROPOGATING-FORCE)) ((A2 . IMPACTED) (A2 . IMPACTED) (A2 . IMPACTED) (A3 . IMPACTED) (D . FOUND) (A . FOUND)) ("FRD->" 2 D1 IMPACTED (PROPOGATING-FORCE)) ((D1 . IMPACTED) (A2 . IMPACTED) (A2 . IMPACTED) (A2 . IMPACTED) (A3 . IMPACTED) (D . FOUND) (A . FOUND)) ("FRD->" 2 A3 IMPACTED (PROPOGATING-FORCE)) ((A3 . IMPACTED) (D1 . IMPACTED) (A2 . IMPACTED) (A2 . IMPACTED) (A2 . IMPACTED) (A3 . IMPACTED) (D . FOUND) (A . FOUND)) ("Expectation" AFFECTEDFOCAL "not met.") Reading PERIOD M-OUTSIDE.42 (focalobject?) CONCLUSION
Figure 1. Processing “I will leave it out of the argument”
6 CHAPTER II A DEEPER LOOK AT METAPHOR
2.1 Another Example One needs only glance at a newspaper to see the plethora of metaphors that augment our everyday language. The New York Times on November 6, 1998 had, among its other front page offerings, articles whose titles included the phrases “Gingrich is Under Siege” and “Democrats in Political Debt.” It is my belief, along with Lakoff, Johnson, and Greene, that metaphors not only augment our language and make colorful headlines, they also form the very foundation of human communication and reasoning;
human thought would be impossible without metaphor.
Communication is indeed based on the way we reason and the way we reason is based on metaphor is true, then the creation of computer systems that communicate the way we do requires the creation of a system that reasons with metaphors the way we do. In this paper I will present a description of such a reasoning system. 2.2 Overview of MIND Imagine two people, one attempting to solve a differential equation and one attempting to obtain information from his boss. Imagine that the first person finds the equation hard to solve. He must now decide what to do. He may try to push harder by applying the Laplace transform or he might try to avoid the issue entirely by finding a way around solving the equation, perhaps obtaining the solution from a friend or a book. He might also give up. Returning to the second person, imagine that he finds his boss unwilling to release the information. He might push harder and use a more
7 compelling argument, perhaps threatening to walk off the job. He may try to avoid the issue by doing his work without the information or he may just give up and quit. These examples demonstrate, in a small way, the power of metaphor. Had I phrased the first problem in terms of whether to use the Laplace transform and the second problem in terms of when one might threaten one’s boss, no reader would discern any sim ila rit y. At the metaphorical level however, they are the identical problem of whether to push harder, go around, or give up. Both people are engaged in performing a task which they instinctively view as a path. Both are being blocked by a perceived obstacle and both perceive themselves in an imaginary space, applying a force against an obstacle that is insufficient to overcome it. Via this perception of the problem, both now view the same set of options as stated before: push harder, go around or give up. This metaphorical view of their situation lets both people reason about their respective situations using the same mechanism built into both of their minds—the ability to reason about space and objects in it. How to go around, push harder and give up are domain specific issues dependant on the details of the current situation and have no common features but, at the metaphorical level, are indistinguishable. The fact that these three options exist is the contribution of this metaphorical perception. The details of how each person might implement these options differ but, at the metaphorical level, their two problems are indistinguishable. In fact, only at this level do we perceive the three options instead of a myriad of possible approaches to a problem with no overt organization or way to select a particular one. The human mind reasons about a situation, using its capacity to process spatial information, at the metaphorical level.
8 Both people, from the preceding example, could meet and discuss their situation with each other and offer advice to each other at the metaphorical level despite the fact that the first person never held a job in his entire life, and the second never passed eighth grade algebra. Should a computer program be able to exploit metaphor to reason in this way, the possibility would then exist to communicate with them int elligen tly, at the metaphorical level, about tasks being performed by an agent or robot, while being completely unaware of, and uninterested in, the details of performing that task. To further demonstrate that both people perceive their situations as paths in space, note that each could naturally describe how far along he is. One might ask them if either would like to leave what he is doing and return to it later. Attempts to reason about either person’s situation in our own minds instinctively invoke an image of steps sequentially placed in space, forming a contiguous link between the starting state and the goal making the above metaphors seem entirely natural. If I were to send out fifty different agents to perform fifty different tasks, I would expect to be able to tell all of them at once that anyone who is blocked or who is not more than half way to its goal should return to doing what it did yesterda y. While at the metaphorical level, each agent received the same instructions, each one will fill in a different set of details. This proposal will present a model which will permit this kind of communication. 2.3 Metaphors in Perception and Reasoning
9 2.3.1 Background. Human communication is so rife with metaphor that much of it goes unnoticed. In their works on metaphor Lakoff [1987, 1989] and Johnson [1987] present numerous examples of spatial metaphors from everyday life. Greene [1995, p. 3] presents the following paragraph containing many examples from their work, demonstrating clearly how we use metaphors of space and force to communicate ideas that, a priori, have no relation to either space or force.
For example, getting through college is a long, bumpy road; after you get underway during orientation week, there are many steps to take and rules to follow. You may at first make headway, but then run into an obstacle or seem to have reached a dead end. Though an uphill struggle, there's no turning back, so you push hard (perhaps under pressure from your parents, who are fed up with your lack of progress), at last grasp the ideas just in time to pass the exams coming up soon (without leaving out any questions), and while sometimes coming to feel you are only spinning your wheels, you hope, if you don't get sidetracked, to reach your goal in the future, down the road.
No one would attempt to convey these concepts without using the metaphors, and if one did, it is doubtful one would find a way to do it. This, again, is because we perceive the world in terms of force and space, along with a small number of other concepts that I will introduce later in this paper. Any task with an objective is immediately perceived as a path through space linking the current situation on one end to the objective on the other. We want to avoid getting off-track when returning to what we were doing because we fear we might get stuck going down the wrong path. We immediately understand what is meant by these phrases since this is how we conceptualize a task.
Further, our minds are designed to process spatial
10 information [Kosslyn, 1980; Trehub, 1991] 1 with no need for conscious acts of deduction. Thus, by using a metaphor of a path through space, one can instantly convey to another person all the information required to envision the situation and mentally analyze it. Flow charts, organizational charts, Venn diagrams and other spatial representations all exploit our innate ability to reason about space, and allow us to perceive what we might otherwise try to understand but fail to grasp. The last sentence of the previous paragraph is an important metaphor about our bodies’ ability to grasp. The metaphor of grasping is actually a complex combination of other metaphors, that refers to acquiring and maintaining an idea. First, we perceive the idea as a physical entity. Then we perceive the acts involved in learning, understand ing, and remembering the information in terms of the bodily act involved in grasping such as reaching, clutching, and stabilizing our control over this physical object. Sim ila rly, the important concept of try/fail/succeed is a metaphor for applying force with our bodies against an object in space resisting that force. The metaphor try harder invokes bodily sensations of additional effort and more pressure against our body from the resisting entity whether we are talking about trying to make a point in an argument, solving a mathematical equation, or pushing a car up a hill. Johnson [1987] describes how we perceive abstract concepts as containers. I add the concept of an object. An object is distinguished from a container by the fact
1
Kosslyn presents a theory of a sophisticated visual buffer as will be discussed later in the text. The existence of this buffer necessitates the ability to process spatial information.
11 that a container defines an interior and exterior space with a boundary between them while an object is solid and simply occupies an area of space. We want to stay out of trouble and stay in a good mood so we do not go out of our minds. Trouble, moods, and minds become containers in space. We want to avoid bad thoughts and find some good ideas so this proposal will be a superior piece of work. Thoughts and ideas become objects. By referring to trouble as a container, one can instantly convey the idea that what may happen when one is in trouble, will not happen when one is out of trouble on an instinctive level without explicitly stating it. We understand in and out of trouble, just as we understand that water spilled on a plastic bag will not affect what is in the bag but may affect what is outside the bag. I will leave it out of the argument uses the concept of a container to convey the fact that anything out of the argument will not affect or contaminate anything that remains in the argument, especially the argument’s point. 2.3.2 The Human Implementation.
The model described above is of a
system where situations in the real world trigger an appropriate metaphor in the mind that is then used for reasoning about that situation. At appropriate times, needed details that pertain to the situation at hand are used to interact with the external world such as choosing the Laplace transform, or using threats against one’s boss after deciding to push harder in the first set of examples. The triggering of the metaphor, which includes reasoning about an obstacle, is based on a combination of default perceptions and bodily feelings. Both people in the examples described are engaged in a task which they naturally perceive as a path through space. Thus they are both reasoning about their
12 given tasks in this manner. At some point, each person ceases to progress toward his goal despite the continued application of force intended to cause forward motion. This incongruence causes a buildup of frustration as progress slowly grinds to a halt or causes shock if forward momentum suddenly drops to zero. Either way, the person feels blocked—that force intended to cause momentum is not doing so. The mind already perceiving a path now perceives an obstacle, the physical (spatial) entity that causes the same bodily feelings of frustration. This natural process immediately leads to the reasoning process described above as those options known to pertain to a physical obstacle in space are reviewed. So we need a set of feelings, but what does that actually mean? It appears this really means two things. The first one is the feelings themselves, although we cannot identi fy, quanti fy, or name them. But perhaps these feelings derive from the second thing, a set of physical phenomenon that actually generates the aforementioned feelings. In other words, those things about which we have feelings. We cannot identify the feeling but there is a tangible prototypical object, event, etc. that we can identify as making us have that feeling. For example, there is a feeling of blockage – but blocked is not actually the underlying feeling – blocked is the physical situation we can describe that causes the feeling we associate with that situation. We describe metaphorical situations as being blocked for several reasons. The most basic is just to convey information about what we are feeling. We cannot name the feeling so we use the common situation in which that feeling is generated in order to talk about it. If I say I am blocked with no other information, one actually learns nothing of my situation other than the fact that I have a particular feeling. Now we come to the fact
13 that the metaphor does more than just convey information about a feeling we are otherwise unable to convey. The metaphor also serves the reasoning process. There are similarities between all situations that generate the blocked feeling that allow for a common reasoning process. The question that might be asked now is something like this. So we respond to balance in the world (again perhaps an evolutionary ingrained aversion to falling or having things fall on us) but how then do we notice this balance in its myriad of forms in our universe? A processor that feeds back into itself, when processing data about a real situation of balance, will create and notice internally a prototypical balance scenario. These questions arise because a scene that includes balance must include other elements. Here is a case where the internal prototype that is recreated may not look like the external stereotypical balance situation. Externally we may think of balance as the physicist's balance beam with two weights on it. Inte rnal ly, balance may have more to do with our own bodies and falling. This would give imbalance its inherent bad nature as opposed to some arbitrary dislike for a balance beam that does not form two ninety-degree angles. 2.3.3 The Computer Implementation. The implementation of MIND shown in Figure two, includes five components: 1.
A Semantic Network
2.
A Feeling Based Lexicon
3.
A Memory Network
4.
A Spatial Processor
14 5.
A Memory Matcher Figure two portrays schematically the layout of the components in MIND.
Decomposing MIND into a set of discrete components, although beneficial for an overall understanding of the system, is a difficult task. The separate parts of MIND’s memory network have, in many respects, grown to be indivisible. The set of integrated networks is one of MIND’s greatest strengths but also creates difficulty in creating a visual portrayal of the system. The one network that has been singled out is the semantic network. This part of the system uses natural language clues to determine the feelings generated by a situation described in text. MIND works by using the feelings generated from its input to determine appropriate high level metaphors for reasoning along with mappings between elements of the metaphor and elements of the real-world situation. Using the abstraction generated, MIND can manipulate a visual representation or rely on memories to perform reasoning. The result of MIND’s analysis can be returned as output, then fed back for further reasoning or first combined with additional input and then fed back for further processing.
15 The hierarchical architecture of the metaphorical prototype network within the memory network allows MIND to use very general metaphors when limited
Figure 2. Basic Diagram of MIND
16 information is available for use in determining an appropriate abstraction. MIND can then refine the metaphor by using a similar but more detailed metaphor in traversing the hierarchy with the benefit of any new information when it is available. Feelings are represented in MIND with tokens so it is important to realize that the feelings are real while the tokens chosen for them are arbitrary. If all English speakers suddenly decided to change the word for frustration to tiskul, the feeling would still feel like frustration. Only the token used to represent it will have changed. The tokens are matched to metaphors stored in the memory network. No reference to domain-specific knowledge is made and the metaphor is matched directly via the feelings. 2.4 How This Model Can Helps Us Communicate with a Computer or Software Agent As an agent goes about performing its task in the real world, its “bo dy” is constantly sending feelings to the triggering system just as our bodies are constantly sending feelings to our minds. In any given situation, if an appropriate metaphor exists in the system, the agent can use it to reason about the situation and determine a course of action to be implemented using domain specific intelligence. Even in a case where specific implementation is unavailable and perhaps especially in such a case, the agent will be able to understand its situation at the metaphorical level and request assistance from an external source using such metaphorical discourse as “I am stuck and need a stronger method.” In using the metaphors which are the same ones we use in our own minds, the agent has gained the ability to communicate about the situation at that level. If the agent can not proceed in its task and it is using the
17 obstacle metaphor, it can tell someone or another agent that it encountered an obstacle but is now going around it. Thus an agent moving from network to network on the Internet will, upon encountering difficulties, refrain from sending information about improper handshaking and its attempts to use more robust protocols (assuming this is the nature of the difficulty), and would communicate with us about being stuck and trying harder. A person communicating with the agent described above may never have used a computer before and may very well be a “technophobes.” The information at the metaphorical level would allow him to completely understand what the agent was saying while the more detailed description would send him screaming. It would also make him more comfortable with a computer-based agent to know that at some level it thinks the way he does and he can understand what it is doing and why. 2.5 Metaphor and Detail A picture is worth a thousand words. Discourse that draws a picture in the minds of the partners to the exchange is more useful to them than attempting to convey each and every element of the situation or event being described. This is true despite the fact that by using some form of imagery, that on its face, is totally unrelated to the actual domain, one immediately loses much of the details. A police sketch artist is trained to extract from another individual exactly those details which we normally do not convey and find very difficult to convey. In fact, discourse involving these details is often awkward and imprecise, as the holder of the original mental image attempts to convey a concept and then often initiates an
18 exchange attempting to verify through query and response whether it appears, through the recipient’s ability to provide an articulation of the image being drawn in his mind, that the image being formed in the listener’s mind is the one the speaker intends. Since the difficulty of expression on the part of the original speaker exists also with the listener, the test of similarity between the two mental images is crude and inexact at best, and completely susceptible to subjective interpretation. 2.5.1 The Motivation for Replacing Actual Detail with Metaphorical Detail.
Given the fact that we still choose to communicate with metaphorical
representation, it must be true that either: 1.
The lost details are of such little value in the discourse that they can be omitted; or
2.
The difficulty in conveying the original image compels us to use an alternative representation despite the cost; or
3.
Conveying the actual image is impossible and the metaphorical representation is the only possible choice; or
4.
There is some advantage to the metaphorical representation that actually outweighs the value of the lost details.
I contend that the latter is always true - there is value to the metaphorical representation. It is not second fiddle or a fall back measure; rather, it is actually the representation of choice. In discourse, the situation is not as it might first appear – there is an attempt to convey general information without detail and any detail is clutter. Although detail in discourse is distracti ng, frustrati ng, and often even impossible to inject as
19 described above and, perhaps of little value when interpreted by the listener, it is actually an essential element of mental imagery and part of the natural flow of human communication. In fact, mental imagery and the flow of mental images as a stream are required for reasoning, reminding and memorizing events and situations. 2.5.2 Shared Detail Required For Communication/ Specific Detail not Useful. We use metaphors because they are encapsulations of complete detailed scenery and feelings that can be conveyed without the interruption of discourse. Additi onally, we use metaphor because we expect the image conveyed to be identical to our own. We expect the associations in the listener’s mind to be the same or at least to have a nonempty intersection with our own, and we expect the feelings generated to be the same. In other words, we expect to create the same foundation for perception and reasoning in our listener that we have in our own minds. Perception and reasoning can not exist without this ability since to reason is to augment comprehension with classification and analysis which can not take place without some form of completeness of thought. Without this, we might need classification for every possible magnitude of available information. Worse, to elevate reasoning to a conscious level without metaphor would require the existence of semantics for each and every problem domain. Infinite sets of semantics would need to be shared by all parties to the discourse, requiring each party to have not only an intimate knowledge of both the domain and the set of semantics specific to that domain. Our ability to exploit information from one domain in another would be hindered by a need to create translations and mappings at the conscious level of
20 semantics between all of the domains we are trying to correlate. Similarities that seem obvious would be completely obscured behind semantics and our ability to generalize would be lost, as would be the ability to classify and correlate. Complete domain specific reasoning systems would need to exist and it would be impossible to communicate about anything with anyone who is unfamiliar with either the domain itself or the domain semantics.
21 CHAPTER III A MODEL OF COGNITION 3.1 Reasoning with Metaphor The theory that metaphor has a central role in reasoning and not just in language has been well established in the work of Lakoff and Johnson [Lakoff and Johnson, 1980; Johnson, 1987; Lakoff, 1987]. They demonstrate in countless examples, some of which were presented at the beginning of this paper, how we perceive the abstract in terms of the physical. Metaphors are experientially grounded; there is a direct correlation between the physical experience used as a basic level image schemata and the abstract experience leading to a natural mapping between them. This is because the experience of the physical and the abstract are both perceived as experiences of our bodies in space. Examples such as “life has cheated me” and “cancer finally caught up with him” [Lakoff and Johnson, 1980] demonstrate that we personify inanimate objects and abstract concepts in order to understand them in terms of our own bodies. Langacker [1987] supports this view of metaphor stating that the metaphors Lakoff and Johnson claim form a foundation for cognition also form the foundation for the semantics of language. Frijda [1986] shows a list of prototypical responses to basic human emotions which could be abstracted metaphorically to pertain to any situation evoking emotion. “Disgust” leads to “rejecting” whether the rejection is spitting out the first taste of a new soft drink or refusing to listen to a political speech at conflict with one’s own views. In a similar manner, “desire” leads to “approach” whether it is purchasing the
22 new drink or attending a speech. Frijda and Swagerman [1987] present a computer program, ACRES, which operates based on what they define as concerns. Concerns are those things important to ACRES, such as its needs and wants. ACRES accepts user input and attempts to relate this input with its list of concerns and determine a goal if possible. The authors point out several shortcomi ngs in ACRES such as the fact that the “emotions” in ACRES are disembodied since ACRES has no body to generate these emotions. The authors also note that ACRES has no body with which to perform instinctive actions based on emotion such as fleeing and approachin g. Still, it demonstrates the use of feelings to stimulate relevant reasoning processes in a computer program. 3.2 Using Feeling Sequences as a Mapping Mechanism in the Categorization of Natural Language Discourse An interesting outcome of combining these approaches to the human use of metaphor with the assertion that much of the richness of metaphors is in the feelings they create, is the possibility of combining reasoning and language using feelings as a common form of data. If it is true as Langacker [1987] states that metaphor also constitutes the basis of the semantics of language and if metaphors can be described by the feelings they create, then it should be possible to define language, to some degree, in terms of feeling based definitions. As will be explained in detail later in this work, I have successfully combined natural language parsing and reasoning relying on overlapping data networks. Thus, although there are language constructs that may not be directly used in reasoning and although not all reasoning mechanisms can be stated in language, in those areas where the two networks overlap, natural
23 language constructs can be directly reasoned about and the reasoning process can be directly described in language. Once a concept can be articulated in language, that is, once it can be described as a combination of elements from the natural language network, it can be reasoned about without the need for further mapping in reasoning constructs. This exemplifies one of the basic premises of education; if a student can describe a concept to the instructor then the student has achieved understand ing. While it is true that basing the understanding of natural language on feelings creates only a basic metaphorical understanding of the text, I am asserting that it is this understanding that is required for a significant amount of reasoning. Such an understanding of natural language is not a replacement for a complete parsing of natural language. Rather, it serves to augment any such natural language processing with a mechanism suitable for high level reasoning that is also usable when a complete understanding of the text may not be possible. Moreover, it supplants domain specific reasoning requiring reasoning mechanisms for the infinite number of possible domains with a generalized reasoning method. Even when a complete literal parsing of natural language is possible, it may not facilitate reasoning. If reasoning takes place as manipulation of mental imagery, then even a completely understood communication may be inadequate for reasoning if it does not convey a complete mental image. If the one attempting to understand the communication is not sufficiently familiar with topic of the discourse, creating an image for reasoning in that person’s mind can prove difficult as that individual is incapable of perceiving any details not directly contained in the communication. An individual who is only partially familiar with the topic may perceive incorrect details
24 if complete details are not provided.
In the case of a metaphorical spatial
representation, it can more readily be assumed that the appropriate detail will be added by the listener once an appropriate metaphor has been conveyed.
Since we all
exist as embodied beings in a physical world, the listener will by default possess some minimal understanding of such a metaphor and will be able to add appropriate details even if not contained in the communication. 3.3 Our Ability to Process Spatial Information at the Instinctive, Biological Level The ability of the human mind to manipulate spatial images based on its biology is a basic foundation of our model. Kosslyn [1980], in his research on the mind’s ability to process visual information, demonstrates that the human mind has a visual buffer for manipulation of data in its visual format. This buffer is an array of limited size in which each cell can be filled in to form an image in the mind. Using test subjects to support his theory, Kosslyn demonstrated that people use identical processes to answer questions about an object whether a person is actually viewing the object or recalling it from memory. Some of the operations supported by Kosslyn’s visual buffer are the ability to shift an image in the buffer in order to focus on different parts of it and to zoom the image in or out in order to perceive less of the image with more detail or more of the image with less detail. Different parts of the image contain pointers to additional details and information, allowing the zoom operation to fill in details as it expands a particular segment of the image. Using these and other manipulations of the image in the buffer, we can inspect the image
25 and draw inferences and conclusions as if the entity in the image actually existed in the physical world. Trehub [1991] presents another model for an image processor in the mind based on the physiology of the eye. He developed a shift register which can mimic the human eye as a moving image passes in front of it. The mechanisms in the simulation notice pattern and movement and can infer expected continuations.2 Mueller [1990] presents a computer program call DAYDREAMER that explores various sequences of events and experiences based on current situations. It does this by daydreaming as the human mind does. The program daydreams possible flows of events based on some real, imagined or remembered scenarios. The model of cognition proposed in Greene and Chien [1993] is a cognitive system grounded in feelings-based schemas consisting of a hierarchical structure of body-based feelings organized into clusters and sequences. Each sequence models a characteristic scenario, a prototypical description of a set of related concepts. In their model, feelings are sampled at discrete intervals and each sample is deemed a feeling cluster. The clusters are then bound. In their example of a sequence for purposeful action, they describe a feeling of need followed by a ‘GO!’ This is in turn followed by a sense of effortless progress and declining discomfort. Finally, the sequence terminates with a stabilization upon satisfaction of the need. Each feeling in the model may actually be a whole hierarchy of feelings. The feeling of a need in
2
The ability to develop expected continuations is an important facet of our model and one that will be addressed later in this paper.
26 the model is actually made up of a need, a more general feeling of discomfort, and finally the specific discomfort. The exact nature of this hierarchy is not described. Exploitation of these sequences of feeling clusters requires that some mechanism exists for perceiving them in their spatial context. Greene [1996], relying on the work of Kosslyn [1980] proposes the addition of visual information to the feeling cluster and expands on his earlier concept of a hierarchical structure by proposing an object oriented inheritance system for metaphorical concepts that allows higher lever constructs to inherit methods from both their lower level ancestors and their metaphorical foundations. Greene suggests that such a buffer may allow the exploitation of feeling sequences for reasoning once a suitable set of such sequences is developed. Kozak [1994] presents a model that uses a rendition of Kosslyn’s visual buffer to analyze, in a very primitive manner, spatial information. Kozak’s buffer is a threedimensional space in which vectors can be projected out from a given point to determine the existence of some entity along their path. By projecting these vectors, the system can determine if its simulated self is in a closed container or if there is an opening. Kozak models the feeling of needs as a container to be filled and her buffer can determine how full a container is. The model presented here, MIND, uses a spatial processor capable of shifting images in a fashion similar to Trehub’s shift register and of analyzing images in a way similar to Kosslyn’s visual buffer. Unlike the visual buffer, the spatial processor is designed solely to manipulate metaphorical information and not arbitrarily complex visual images. As such it is far simpler in its design. The spatial processor is
27 designed to support Kozak-style analyses of the entities within it and also the manipulation of entities appearing in it by an external process simulating a script such as those proposed in Greene and Chien [1993]. It can then notice patterns of movement and determine expected continuations. The spatial processor in MIND, while simpler in design than that of Kosslyn or Trehub, is not a passive mechanism as such Kozak’s processor. It is an active simulator of spatial abstractions. Utilizing the processor generates feelings that are fed back into MIND’s reasoning system. While it is possible to scan the image in the processor to determine factually whether an entity is contained within another, MIND does not depend upon such active attempts to seek information. Instead, the feelings being generated in the processor themselves activate concepts in MIND’s reasoning engine. MIND knows it is in when it feels in.
28 CHAPTER IV OTHER RELEVANT WORKS 4.1 Other Works Martin [1990] describes a system called MIDAS that attempts to understand the meaning of a metaphor by expanding its collection of lexical entries. This approach equates metaphors with regular lexical entries. Martin stores his lexicon in a network which allows his program to impute sensible meaning to metaphorical usage of termin olo gy. His goal is to produce a system that will respond correctly to naturally worded human input that may contain metaphors. He presents the example of his Unix assistant being asked “how do I kill EMACS?” In attempting to parse this input, MIDAS does not find a definition of kill suitable for use in relation to the text editor EMACS. It then attempts to expand its definition of kill to include one suitable to EMACS by determining whether any definition it has for kill might apply to a parent concept of EMACS in the lexical network. MIDAS determines that killing includes terminating and since EMACS is a computer process, a computer process is a process, and processes can be terminated, the input must actually mean “How do I terminate EMACS.” MIDAS then responds with an appropriate repl y. In this process, metaphors become a set of synonyms for existing lexical entries. The questions can then be raised: “why use a metaphor?” and “why not just say terminate if we mean terminate?” Lakoff and Johnson [1980, p. 3] state that “[m]etaphor is for most people a device of the poetic imagination and the rhetorical flourish-a matter of extraordinary rather than ordinary language.” In other words, metaphor conveys
29 much more than lexical meaning, and understanding the meaning of language using metaphor is much more than finding a suitable lexical replacement for the offending phrase. To kill a process is much more than to terminate a process. Martin’s system responds with an appropriate instruction, telling the user to press control–c on the keyboard; this response, however, would be the same had the user asked “how do I terminate EMACS?” Killing does involve terminati ng, but it is also much more than that. This nuance is lost by attempting to impute only lexical meaning to a metaphor. CopyCat [Mitchell, 1993] is a system which uses conceptual slippage to allow the use of analogy (metaphor) in responding to input.
CopyCat attempts to
understand the meaning of its input by allowing the definition of known concepts to slip or to expand to include elements from the input if it can find a suitable relationship between the known concept and the input. CopyCat is not intended to demonstrate a comprehensive approach to metaphor. It operates on patterns of letters and not human language, avoiding issues raised here of metaphor’s richness and embodied meaning. It does, however, exploit the richness inherent in the use of metaphor itself. Unlike MIDAS [Martin, 1990] CopyCat examines the “pressures” exerted on the system to create an analogy and the “stress” created by doing so. “Pressures” refers to those facts which compelled the creation of one anal ogy, such as relating orange juice to milk because it is a drink over another analogy such as relating it to a basketball because they are both orange. “Stress” refers to how much of a stretch it is to make the analogy work based on how closely related the two concepts are.
30 An attempt to actually use a metaphor for reasoning is presented in Kozak [1994]. Kozak prepared a Prolog program called FESINA which demonstrated that several abstract scenarios could be interpreted by a computer system using a small group of fundamental schemas which constitute bodily metaphors. FESINA uses a simple version of a visual buffer, as described earlier in this paper, that could determine the location of entities within it. Based on these determinations, the system generates feelings and attempts to use them to match one of the fundamental schemas. If, in attempting to discern the nature of its situation utilizing projected vectors in its buffer, FESINA determined it is bound on all sides, it would create a feeling of containment and invoke the fundamental schema relating to containment. FESINA’s mechanisms are crude and it is not a complete system because it requires operator intervention at several points. It does, however, serve to demonstrate that such systems are possible. A far more robust system is described in Narayanan [Narayanan, 1997]. Narayanan presents a system called KARMA that uses connectionist simulators called x-schemas to capture the richness of embodied metaphors and a belief network used to draw inferences. KARMA attempts to understand metaphors by simulating the base concept in the corresponding connectionist network and mapping elements of the simulation to the target domain, the actual situation. By examining the simulation and its mapping back to the target domain, the system can exploit domain specific knowledge in its belief network to draw inferences. Unlike Martin’s system described above, KARMA can draw inferences based on the richness of the metaphor.
31 KARMA, when presented with a metaphor such as “Brazil fell into recession” [Narayanan, 1997, p. 223] used an x-schema to simulate the act of falling. Using specially designed mapping mechanisms, elements of the source domain are mapped onto the target domain. As such the system would know that Brazil uncontrollably descended into recession. Additional knowledge in a belief-network allows KARMA to make domain specific inferences such as how Brazil might try to rise out of the recession. KARMA is not a general system and is designed to handle “active representations of motion and manipulation words” [Narayanan, 1997, p. 20]. As such, its abilities are in some ways analogous to MIND’s perceptual path metaphor, though it lacks the ability to handle a whole class of metaphors such as those based on the container concept and those based on embodied feelings and not embodied action. An additional shortcoming of these systems is that they are unable to participate in metaphor based discourse. For example, Martin [1990] responds to a query that uses metaphor after attempting to extract from the metaphor its lexical meaning. It draws no further inference from the metaphor nor can it use the metaphor on its own since, through making the metaphor a lexical entry, it can not determine when it would be correct to use the metaphor. Narayanan [1997] uses simulators to maintain the richness of the metaphors in text, as described above, but does not do the reverse. Given a complex situation, it can not determine a metaphor appropriate to describe it.
32
4.2 Limitations of These Approaches 4.2.1 Isolation Language from Reasoning.
The work presented here
demonstrates a system not subject to many of the limitations inherent in the approach described here. One of the main advantages of MIND is the set of integrated networks utilized by all of the constituent elements of the system. This combination of language, reasoning and remembering/reminding avoids the need for an unbounded amount of domain specific mapping mechanism required to convert data utilized by one element of the system to the format required by another. Once the system has begun to process either feeling based or text based information, the internal data structures created at all steps of processing are usable by any element of the system. Further, the system’s own output, as a reflection of internal data structures, is appropriate to be directed back into the system and can be processed in the same manner as the original input. Thus, it is possible that two complete systems of this nature, with slightly different networks, could converse with each system reasoning about the output of the other. This ability can not exist in systems that change the format of the data as it is moved through the system. 4.2.2 The Advantages of Hierarchical Structures. The integrated networks, along with the hierarchical structure of the data stored in the network, also provide a significant portion of the mapping ability of the system that obviates external mapping mechanisms. Entities can be perceived as being represented metaphorically by very generalized constructs but can be refined to be represented by more detailed
33 metaphors when possible by attempting to move the known characteristics of that entity down the metaphor hierarc hy. The hierarchical approach also allows the system to draw its processing power from a very narrow base. At its most basic level, the system knows how to process a small set of metaphorical primitives. Since these primitives form the origin of the entire network, and other higher level constructs are abstractions and combinations of the primitives, the entire system is as capable as the basic processor. Systems that lack this hierarchical approach to metaphor storage cannot take advantage of this abilit y. 4.2.3 The Shortcomings of a Lexical Approach. One of the other pitfalls MIND avoids is that of attempting to expand, or stretch definitions in order to allow them to lexically include metaphorical interpretations. Such approaches suffer from several drawbacks. One is the need to develop a form of expandable lexicon and then to develop restrictions to avoid uncontrolled expansion. Another issue is the loss of detail inherent in metaphor which can not be encapsulated in a lexical description. MIND does not suffer from this weakness since metaphors are interpreted in a spatial processor and, as such, the amount of detail is only limited by the sophistication of the processor.
The lexical approach to metaphor also suffers many of the
shortcomi ngs listed above such as the inability to generalize mapping mechanisms.
34 CHAPTER V MORE EXAMPLES OF MIND 5.1 Another Look at the “Assertion out of the Argument” The example in section 1.2 showed how MIND perceives the statement “I will leave that assertion out of the argument.” We will now look at how MIND actually processes the information that it is being provided and how it goes from hearing about an argument to thinking about objects and forces and how they affect each other. For convenience, the listing from Figure one is shown here again in Figure three. In lines one through ten MIND is reading the sentence. In line 11 MIND shows that it has determined that the base of its interpretation (“Pred Base”) is MOUTSIDE. That is, its top level memory of an outside situation. The content of MOUTSIDE, shown in Figure four, is a list of entries enclosed in parentheses. The first entry in M-INSIDE is inside designating that this is a specific memory inheriting attributes from the general categorical memory inside. Following that are the elements of the memory: the existence of an object that inherits attributes from M-TOP-NOUN, that is anything that can have physical properties imposed on it, a container that also inherits from M-TOP-NOUN, and a set of feelings. The feelings serve a dual purpose. They serve to let MIND know when it encounters an outside situation by being used to match against a stream of input feelings. They also tell MIND what a speaker is trying to convey when the speaker uses the word inside. The feelings are stored as a group. The integers stored with each feeling name do not designate an ordering and are for MIND’s internal identification purposes onl y. In
35 addition to the feelings themselves each feeling has an associated level used to distinguish relative levels such as fear and terror.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43
Reading Reading Reading Reading Reading Reading Reading Reading Reading
I WILL LEAVE THE ASSERTION OUT OF THE ARGUMENT
("Pred" (NIL NIL M-OUTSIDE 2) "Base" M-OUTSIDE) Activating M-OUTSIDE.42 (M-OUTSIDE.42 ((CONTAINER ARGUMENT) (OBJECT ASSERTION)) ((OBJECT ASSERTION) (CONTAINER ARGUMENT) (FEELINGS M-FEELINGGROUP.31))) ("FRD->" 0 A FOUND NIL) ((A . FOUND)) ("FRD->" 0 D FOUND NIL) ((D . FOUND) (A . FOUND)) ("FRD->" 1 A3 IMPACTED (PROPOGATING-FORCE)) ((A3 . IMPACTED) (D . FOUND) (A . FOUND)) ("FRD->" 1 A2 IMPACTED (PROPOGATING-FORCE)) ((A2 . IMPACTED) (A3 . IMPACTED) (D . FOUND) (A . FOUND)) ("FRD->" 1 A2 IMPACTED (PROPOGATING-FORCE)) ((A2 . IMPACTED) (A2 . IMPACTED) (A3 . IMPACTED) (D . FOUND) (A . FOUND)) ("FRD->" 2 A2 IMPACTED (PROPOGATING-FORCE)) ((A2 . IMPACTED) (A2 . IMPACTED) (A2 . IMPACTED) (A3 . IMPACTED) (D . FOUND) (A . FOUND)) ("FRD->" 2 D1 IMPACTED (PROPOGATING-FORCE)) ((D1 . IMPACTED) (A2 . IMPACTED) (A2 . IMPACTED) (A2 . IMPACTED) (A3 . IMPACTED) (D . FOUND) (A . FOUND)) ("FRD->" 2 A3 IMPACTED (PROPOGATING-FORCE)) ((A3 . IMPACTED) (D1 . IMPACTED) (A2 . IMPACTED) (A2 . IMPACTED) (A2 . IMPACTED) (A3 . IMPACTED) (D . FOUND) (A . FOUND)) ("Expectation" AFFECTEDFOCAL "not met.") Reading PERIOD M-OUTSIDE.42 (focalobject?) CONCLUSION
Figure 3. “I will leave it out of the argument”
36
(DEFMOP M-OUTSIDE (OUTSIDE) (OBJECT M-TOP-NOUN) (CONTAINER M-TOP-NOUN) (FEELINGS M-FEELING-GROUP (1 container) (CONTAINER 320) (2 surface) (SURFACE 200) (3 object) (OBJECT 20) (4 full-empty) (FULL-EMPTY 200) (5 balance) (BALANCE 46.5) (6 obstruction) (obstruction 319) (7 path) (PATH 283) (8 process) (PROCESS 236.81) (9 enablement) (ENABLEMENT 682.88) (10 force) (FORCE 6.0000005) (11 counterforce) (COUNTERFORCE 6.0000005) (12 iteration) (ITERATION 17.9) (13 part-whole) (PART-WHOLE 11.25) (14 restraint-removal) (RESTRAINT-REMOVAL 35.88) ))
Figure 4. Memory of Outside
The parent of M-OUTSIDE is shown in Figure five. It defines only the physical attributes of an outside situation. The first entry states that outside inherits from the more general categorical memory CONTAINER-RELATION. Following this, we see that the relationship between the physical entities is an outside relationship and the actual physical entities are an object, a container and an outside – the outside of the container. The container of course, also has an inside but like the infinite list of other entities that might exist in an outside situation, noticing the existence of outside causes MIND not to focus in it and to focus on the entities listed here. After recognizing that the reference is to an outside situation MIND, maps the entities in the input to the entities in the remembered situation. As shown in lines three and 13 of Figure three, MIND matches up the physical entities and then the
37
(DEFMOP OUTSIDE (M-CONTAINER-RELATION) (RELATION OUTSIDE) (IMAGENAME OUTSIDE) (ENTITIES M-ENTITY-GROUP (1 CONTAINER) (2 OUTSIDE) (3 OBJECT)))
Figure 5. High Level Memory of Outside feeling set that is appropriate for what is being described. At this point, it is important to mention that only two of the three objects in the memory are mapped to entities in the memory. In fact, no mappings are required at all. Mappings merely give the ability to draw more domain specific inferences; they are not required for MIND to process the input. MIND can reason using only the high level metaphorical forms stored in the memory. In this case MIND, knowing nothing about the situation being described except what it is told and what it knows via the fact that it is an outside situation proceeds to form a spatial perception by creating the appropriate mental ima gery. MIND uses its prototypical image of a container, and starts to use it to represent the argument that it mapped to a container, as described earlier. The issue that one might have raised at some point in this discussion is that MIND somehow deduces that the focal object is not affected and that focal object is the conclusion. Yet MIND is not creating mental imagery and there is no focal object and no mention anywhere of a conclusion. The key to determining the existence of an unmentioned conclusion is in how MIND creates its imagery. Before settling on using as a representation an image mentioned in a categorical memory, MIND constantly attempts to find more specific image classifications by examining new information
38 provided in the input and specific knowledge about objects mapped to general categories. When MIND reads “...the assertion out of the argument,” its first choice of an image for argument is that of a container. Before settling on this image, MIND checks whether it knows anything about arguments that will allow it to use a more detailed image. MIND’s memorized definition of an argument is shown in Figure 6. MIND knows that an argument has implications. If it knows nothing else (and
(defmop ARGUMENT-IMPLIES (M-IMPLICATION) (FOCALOBJECT CONCLUSION)) (DEFMOP ARGUMENT (M-TOP-NOUN) (IMPLIES ARGUMENT-IMPLIES))
Figure 6. Memory of Argument Implication
at this point it in fact knows little else about arguments except a few inherited attributes), MIND knows that the existence of an argument implies the existence of a conclusion that is the focus of the argument. After processing this information, MIND has knowledge of the existence of an entity not mentioned in the input. MIND then checks its memory of the current mapping of argument, a container to see if a more detailed type of container can incorporate this new information. In fact, a more detailed type of container is the container with contents as shown in Figure 7. This entry in the memory contains very little information. All it states is that a container with content has a focal object that must be able to be imparted physical characteristics and that there is a remembered image called container-with-content.
39
(DEFMOP CONTAINER-WITH-CONTENT (CONTAINER) (IMAGENAME CONTAINER-WITH-CONTENT) (FOCALOBJECT M-TOP-NOUN) )
Figure 7. High Level Memory of Container with Content
The actual physical relationship of the content to the container is implicitly found in the image. It is important to remember that although the imagery MIND creates will now have another object, that will, in fact, become the focal object, there is no mention of this object in the text, and MIND has no situational entity to which it can map this object. MIND will also refine the image for the assertion. Before describing how that happens, I will describe how MIND creates its mental imagery. A segment of MIND’s mental space after insertion of the imagery is shown in Figure 8. MIND’s internal list of mapping for the imagery is shown in Figure 9. The portrayal of the imagery shows two planes of MIND’s mental space. Each plane represents the x and y axis. The planes themselves constitute segments of the z axis with each plane representing what is behind (z + 1) the plane to its left. In the list of mappings the groupings represent the name of the entity and then a pair representing which of MIND’s mental spaces holds the image, a dot, and then the symbol used to represent the entity in the image. With Figure 8 showing mental space zero, the assertion is represented by ‘D,’ the argument by ‘A,’ and the conclusion by ‘B.’ Each part of the representation in the mental space contains a symbol described above and a digit. The digits are used to distinguish constituent parts of a single entity. The container has two sides shown, and a bottom represented
40 by A1, A2 and A3 respect ively. The conclusion has one part represented by B1 and the assertion, also having one part, is represented by D1.
There is a major difference between the manner in which MIND determines where to draw the conclusion in the image and how it determines where to draw the assertion. The location of the conclusion is predetermined by a remembered image of a container with content. Thus MIND places the conclusion in the location of the content in the memorized image. It is important to remember that its existence and
Figure 9. Segment of Mental Imagery location in the memorized images inherently represent all that MIND needs to know and their presence in the mental imagery will affect MIND’s reasoning process. The location of the spatial representation of the conclusion is determined from the input. The assertion is said to be “outside of the argument.” MIND knows where to place the assertion because it knows the spatial interpretation of “outside.” As we have seen, MIND knows what “outside” feels like, and what are the elements required to have “outside.” MIND also knows what “outside” looks like, and how to create an
((ASSERTION (0 . D)) (ARGUMENT (0 . A)) (CONCLUSION (0 . B)))
Figure 8. Mappings
41 image of “outside” using objects it can portray in a mental space. Now that I have shown how MIND creates its mental imagery, I will return to refining the imagery for the assertion. MIND also has some knowledge about the assertion that is not mentioned in the input. MIND knows that assertions are considered to be able to exert a force in contexts in which they are mentioned. MIND therefore refines the image used to represent the assertion from an image of an object to an image of an object with a force. The memory of noun with force is
(DEFMOP NOUN-WITH-FORCE (M-TOP-NOUN) (IMAGENAME OBJECT ) (FORCE 3D) (EXPECTATION AFFECTEDFOCAL ) )
Figure 10. Memory of Noun with Force shown in Figure 10. A noun with force exerts, by default, a three-dimensional force (one that expands in all directions) and causes an expectation that this force will, in some way, affect the focal object of the context in which it is mentioned. MIND, therefore, not only places a representation of the assertion in space, but also places a representation of a force. As will be described in detail in later sections, MIND has a sufficient understanding of the laws of physics to know how a force propagates and what happens when it encounters another entity. Lines 16 through 36 of Figure 3 show very verbose feedback from MIND’s placement of the force in the mental space and what MIND notices in the space as the force spreads. After MIND gathers all this
42 information, it then checks any expectations it has. In our case there is one expectation, that the focal object will be affected and MIND notices that it is not. 5.2 Finding Expected Continuations Once MIND determines a spatial understanding of a situation, it can also find expected and/or possible continuations. It does this by using remembered sequences of feelings and events at the level of the categorical memories it is using to portray the current situation. A small example will demonstrate this point. In Figure 11, MIND is reading the simple input “I am stuck.” What is important to see in this example is that MIND infers from the input a presumed antecedent and then, using this sequence of events, determines expected and possible continuations. On line seven, MIND states that it has found a probable antecedent (presequence) for an actor being stuck. In lines 14 and 15, MIND is actually treating the antecedent as input causing it to generate the same results as if it were actually fed into the system as input. As will be demonstrated later, one of MIND’s strengths is its ability to reinterpret its own output as input and feed back into itself. Using this new “input,” MIND notes in line 20 that what is being described is actually a failed attempt. Line 22 shows a list of possible continuations found by determining what memorized sequences actually begin with internal-force, counterforuce, stuck. The sequence that is found has, as its next act, try harder and give-up.
43
1
Reading I
2
Reading AM
3 4 5 6 7 8 9 10 11 12
Reading STUCK ("Pred" (NIL NIL M-STUCK 1) "Base" M-STUCK) Activating M-STUCK.49 (M-STUCK.49 ((ACTOR I)) ((SCENE M-DO-OBJECTIVE) (FEELINGS S-FEELING-SEQUENCE.46) (PRESEQUENCE S-PRESEQUENCE.48))) ("FRD->" 0 A FOUND NIL) ((A . FOUND)) ("FRD->" 0 C FOUND NIL) ((C . FOUND) (A . FOUND))
13
M-STUCK.49
14 15
Reading INTERNAL-FORCE Reading COUNTERFORCE
16 17 18 19
Reading STUCK ("Pred" (NIL NIL FAILED-ATTEMPT 1) "Base" FAILED-ATTEMPT) Activating FAILED-ATTEMPT (FAILED-ATTEMPT NIL ((RESPONSES M-RESPONSE-GROUP.41)))
20 21 22
FAILED-ATTEMPT (TRY-HARDER GIVE-UP)
Figure 11. Expected Continuation
44 CHAPTER VI THE MODEL OF MIND 6.1 System Components The first models of MIND consisted of five components. These were designed to isolate the components of the reasoning system into self contained, independent modules. The five components were: 1. A minimal body as required to generate the feelings such as frustration and shock. 2. A metaphor reasoning system that processes spatial information and feelings. 3. A hierarchical memory system. 4. A feeling tokenizing system. 5. A metaphor triggering system These components existed as five distinct subsystems. This design, although allowing a maximal amount of freedom in the design of each subsystem, required each component to have mechanisms for interpreting input coming from other components and for preparing output in the correct format to be transferred to other components. Even more critical, was the need for each component to maintain independent data structures. As the complete system developed, it became apparent that the various subsystems should be integrated to work with a common data structure store in a common data network. Many of the capabilities of MIND emanate from this shared network design.
45 MIND is now built around the following components using the structure shown in Figure 12. Although the underlying concept remains the same, the new component structure permits sharing the data network. 1. A Semantic Network 2. A Feeling Based Lexicon 3. A Memory Network 4. A Spatial Processor 5. A Memory Matcher 6.2 The Semantic Network The semantic network will be described in detail later in this work. The following will not repeat that description but will describe how the semantic network fits into the overall component structure of MIND. The semantic network is portrayed in Figure 13. It consists of links between the nodes in the memory. Each link has an activation coefficient designating the relative strength of the relationship between the two connected nodes. Nodes in the semantic network can be activated, causing the activation to propagate. The resultant set of activated nodes and their corresponding activation levels can then be used as an input set to other components in the system.
46
Figure 12. Full Diagram of MIND In Figure 13, we see that on, outside, inside, stumble, and fall are lexemes. They are linked into the network to nodes of the semantic net via links with varying
47 activation levels. When a lexeme is activated, its activation is multiplied by the activation levels of the links along which it travels. This will cause slow dissipation of the activation as it spreads throughout the network until it finally drops below a threshold value and propagation stops. Only a subset of the nodes in the memory network are actually linked into the semantic network. Each of the connected nodes represents either, one of the basic feelings that are used to create the entries in the feeling based lexicon, or a named aggregate feeling representing a combination of basic feelings. 6.3 The Feeling Based Lexicon Like the semantic network, the feeling based lexicon has been described earlier in this work in Chapter IV. In the overall system, the lexicon serves to provide direct links between lexemes and nodes in the semantic network. It is via the lexicon that complete feeling sets can be created from
external textual
information. Each entry in the lexicon is linked to its direct entry points in the semantic network. These links make up its basic feeling based definition. When a lexical entry is activated, it causes activation of its direct entry points that in turn causes propagation of the activation throughout the network, yielding a complete feeling set. Although it may appear as though the lexicon exists as a separate entity, and indeed in its earliest incarnations it was one, it exists as another subset of the overall memory network. The lexicon represents those nodes in the network to which a natural language word or phrase can be associated. The nodes form the basic
48 building block of recognizable natural language sequences that will be discussed in the section on the sequence recognizer.
49
6.4 The Memory Network
Figure 13. The Semantic Network
The memory network forms the linkage between the components of the system.
It is actually not a single network.
Instead, it is a collection of
50 superimposed networks sharing a common node structure but having different sets of links. The component networks are not subnetworks; they are not severable, and any attempt to separate them will necessarily result in duplicated data. They represent different perspectives on the same data, represented by the different sets of links forming different organizational structures. The metaphor network views the nodes as a hierarchical structure representing levels of abstractions, while the semantic network has no form of hierarchical structure at all. This network structure allows the various components of MIND to look at exactly the same data but to interpret it in different ways. The node for in can be part of the feeling based lexicon representing the lexeme in linked to its direct entry points into the semantic network. To the semantic network, the same node is linked via activation coefficients to the rest of the semantic network, allowing propagated activations. To the spatial processor, the same node represents the prototypical spatial representation of in. The complete memory network consists of the following component networks. 1. Feeling Based Lexicon 2. Semantic Network 3. Metaphor Hierarchy 4. The Lexical Sequence Network 5. Feeling Sequence Network 6. Image Sequence Network 6.5 The Metaphor Hierarchy
51 The metaphor hierarchy is based the levels of abstraction that can be used in perceiving situations. The most generalized perceptions form the top of the metaphor hierarc hy. Lower nodes in the tree represent metaphors with added detail. This added detail is not domain detail but spatial detail that can be exploited should it be alluded to or should there be domain specific detail to which it can be mapped. The top level abstraction for all verbs in the hierarchy is the metaphor named “Do,” representing movement of an actor along a path. The basic abstraction “Do” would be the representation for “I think.” Although represented by a path, it lacks landmarks required for measurement of distance and therefore proportions. Without any additional information, one can not measure how much one has “thought” or divide thinking into units - how many thinks in a thought?
Thus, reasoning is
limited to a sense of progress or forward motion but is prevented from quantifying that motion. The first level of specialization is “Do Quantifiable.” This represents acts in which the amount of the act performed can be measured and divided into discrete countable units. For example in the statement “I run,” one can quantify how far one runs. In this case, the only available landmarks are current and previous positions. One can measure in terms of units, the distance that has been traveled. The next level of specialization is “Do with Objective.” Here, there is a goal or objective to be reached; not only can the amount of the act performed be quantified, but so can the distance to the goal. Given a current position on a path and a goal, one can measure the distance between them determining how much of the act
52 needs to be performed and measuring progress in units not just based on a feeling of unquantifiable forward motion. A sibling specialization to “Do with Objective” is “Do with Origin.” While in “Do with Objective” the distance to the end of the path can be measured, in “Do with Origin,” the distance from the beginning of the path can be measured since there is a known origin but no known destination. In this case, the amount of the task performed is quantifiable but there is no known end to the path so the amount to be performed, if there is such a set amount, is unknown. A specialization of “Do with Objective” and “Do with Origin” is “Do with Complete Path,” combining the attributes of both higher level abstractions. In this perception of a task, the amount performed can be compared to the amount remaining, allowing relative levels of completion to be used. One can be thirty percent finished or half way done. It is important, at this point, to mention the manner in which MOP memory works. For example, the metaphorical representation of “Do with Objective” is a composite of metaphorical concepts themselves represented as part of the metaphor hierarc hy. MOP memory allows an abstraction to be replaced by one of its specializations in a processes referred to as refinement. This allows the goal, which is represented as a metaphorical object, to be replaced by any of the metaphorical specializations of object. It can, for example, be an object with force such that approaching it causes some affect to the actor. The force can be a destructive force, and the actor may incur damage as it approaches. This ability to perceive additional details exists without detracting from the value of using the higher level abstraction.
53 The actor can be a high level object or it can be a prototypical person. As a person, new inferences can be drawn about acts such as tripping, falling over and getting up. The person itself is represented not as a new entity, but as a body built up as a composite of objects attached and jointed in the form of a body. As will be explained in detail later, the placement of a person as the object moving along a path can be caused by directly supplied information or by inferred mapping. The statement “I think,” overtly places a person on the path since “I” is a personal pronoun. The statement “India stumbled on its way to economic reform” implies the mapping to a body since, although India under normal circumstances is not thought of as a person, the act “stumbled” is linked, via its feeling based definitions, to bodily feelings of imbalance, pain, and uncontrolled motion, all of which imply the existence of a body. The metaphor hierarchy exists as its own network within the memory network. It links those nodes in the memory network that represent metaphorical prototypes. These prototypes contain imagery that allows them to be recreated as mental images in the spatial processor leading to the ability to perform spatial reasoning. The links that make up this part of the memory form a structure representing levels of abstraction in metaphorical detail. 6.6 The Lexical Sequence Network MIND’s ability to derive spatial abstractions and mappings from natural language derives in part from a rudimentary front-end parser. While this is not a full featured parser, it does demonstrate many of MIND’s natural language abilities,
54 particularly the advantages of using its sequence matcher and memory network to parse text in a manner that supports applying spatial reasoning to the text. The parser is built upon Micro-DMAP [Riesbeck and Schank, 1989]. As it reads texts, it attempts to turn the text into a specific instance of a known generalized memory. Micro-DMAP is a parser that relies on information stored in the MOP network to match patterns in its input. Micro-DMAP moves through its input maintaining a record of partially recognized patterns and predicted continuations. To serve as part of the MIND system, Micro-DMAP was augmented with several significant new capabilities. It utilizes the semantic network to derive feelings and feeling sequences. Utilizing these feelings, it determines an appropriate metaphorical prototype to be used to create a feeling based abstraction of the text. Based on the metaphorical abstraction, it determines which mappings can be made for elements of the abstraction and which elements of the abstraction will remain in their generalized form. It constantly attempts to create mappings for generalized forms utilizing new information from the text and searching the memory for implied mappings based on elements of the text. It also maintains a record of the activated memories. The lexical sequence network is made up of ordered sequences of high level lexical classifications linked to memories of generalized situations to which the sequences can be mapped, as portrayed in Figure 14. When a match between a segment of text and a sequence is made, the feelings activated in the semantic network, the number of mappings that can be made, and the inferences that can be drawn, all serve to determine the correct metaphorical prototype, and the level of
55 metaphorical abstraction from the metaphor tree that will be used as the spatial representation for reasoning. In Figure 14, the sequence named M-DO-OBJECTIVE is linked to a sequence actor, on, path, to, goal. On and to are required lexemes, and appear as part of any sequence matching M-DO-OBJECTIVE.
Actor, path, and goal are lexical
classifications. They will match, as part of the sequence, any lexeme that appears beneath them in the hierarchy of the memory network. The sequences are made up of the highest possible lexical classifications, in order to allow maximum freedom in matching actual sequences to those in the network. Where possible, general categories such as verb or noun are used to designate what form of lexical entry should appear at a given point in a sequence. At other times, a specific lexeme may be the clue to matching the rest of the sequence and the mapping of entities. In this case, the sequence will contain the needed lexeme at the correct position. Completely unrecognized elements of an actual sequence that can not be mapped into the network are ignored, in the hope that utilizing those elements that are recognized, will yield a correct interpretation.
56
Lexical sequences can be as short as noun, verb matching any noun followed by a verb. This basic type of sequence will be associated with the highest level abstractions since, without additional information, no mappings can be created beyond the existence of a noun mapped to an object and a verb mapped to Do. More complex sequences such as noun, verb toward noun would match an actor performing an act toward a goal or objective. Although currently not exploited, all components of MIND support much more complex lexical sequences. Sentences such as “I ran toward Sara inside the room under the broken ceiling” can be
Figure 14. The Lexical Sequence Network matched by creating spatial representation for all of the clauses, and combining them as subspaces represented as entities in a metarepresentation of the entire sequence. Such subspace/metaspace representation is supported, although the current
57 implementation does not exploit it. The sequence recognizer is also designed to recognize subsequences and combine them into larger sequences of sequences. 6.7 The Feeling Sequence Network As a feeling based reasoning system, MIND needs to be able to match concepts not only to textual clues from discourse, but also to feelings generated either by the system’s interaction with its environment or as a result of manipulations in its spatial processor. If MIND is trapped in a container, it must realize this via a sequence of feelings common to “trapped” situations. It does not determine trapped via a set of rules defining tests for trapped. It does not even actively test to see if it is trapped. The sequence recognizer is watching the generated feelings and when that sequence indicative of trapped is noticed, the appropriate nodes in the memory network are triggered, causing MIND to know it is trapped. The feeling sequence network is made up of ordered collections of feelings linked to either an aggregate feeling or concept as portrayed in Figure six.
In
MIND, the aggregate feeling stuck can be triggered by matching the following sequence of feelings: Internal Force, Counterforce, Disappointment, Null Movement. The feeling of stuck represents a situation in which the application of force does not yield change. This is more primitive than the feeling of blocked, as blocked implies the existence of some obstacle or impediment. One can be stuck for any number of reasons, including being held back or a lack of adequate friction. In Figure 15, failed attempt, blocked, and attempt success are generalized situations linked to sequences of feelings and possible continuations in the form of
58 situational responses. The scenario blocked is typified by the sequence internal force, counterforce, obstructed, stuck, pressed. If such a sequence is recognized, it can be called blocked and the word blocked indicates the existence of these feelings. The possible responses, in general categorical form, for blocked are try harder, go around, and give up. The feeling sequence network utilizes the same sequence recognizer as the lexical sequence network allowing for equally complex sequences.
As MIND
matures and more complex spatial reasoning abilities are added, the ability to recognize more complex feeling patterns will also be required. This ability is also essential for MIND’s ability to become part of an artificial entity with a body that may create a constant flow of complex feelings about which MIND will need to interpret and reason.
59
6.8 The Memory Sequence Network
60 The memory sequence network combines sets of memory nodes into a sequence constituting a complete mem ory. The sequences constitute a script or movie representing a remembered flow of events. These sequences allow MIND to have presuppositions about what preceded a recognized scenario and what might
Figure 15. The Feeling Sequence Network follow it. These sequences also allow MIND to conceive rational responses to situations. MIND’s first response, based on a memory sequence, to feeling stuck is to try harder, by applying increased force. This response is not an arbitrary phrase produced by rules. It is a representation of memories in the memory network that
61 allow MIND not only to know that trying harder is an appropriate response, but also to reason about trying harder. It can remember through added sequences what might happen when it tries harder. It can model trying harder in its spatial processor and come up with possible outcomes without reliance on remembered events but only on its spatial abstraction of the current scenario. Figure 16 shows a segment of the memory sequence network. Parts of a sequence are named such as a presequence which is an assumed antecedent. The action part of the presequence for failed attempt is move and actor. The remembered scene is do objective. The possible continuations, try harder, go around, and give up shown here can also be considered part of the feeling sequence network (see Figure 6) but when considering the memory sequence network, are remembered continuations of a situation.
62
Figure 16. Memory Sequence Network
63 CHAPTER VII MIND’S INTERNALS
7.1 Mental Spaces The spatial processor is written as a combination of Lisp and C++, with Lisp performing most of the logic and C++ performing most of the manipulations. This arrangement appears to achieve a reasonable compromise between clarity and speed. MIND creates its mental images in one of ten available three-dimensional matrices. The size of the matrix is currently 20 x 15 x 15. Larger matrices allow more detailed images and more entities to be portrayed in the space. Entities in the matrix are represented as Lisp symbols. The first part of the symbol represents the object and the second part represents the part of the object. Each space can contain up to 26 entities represented by the letters of the alphabet and each object can have up to ten parts. 7.2 Objects and Parts The parts of a given object can be either attached or jointed. Attached parts are connected with a rigid joint that will not give under pressure; these parts must move in unison. Jointed parts are connected with flexible joints. Each such joint has a resistance factor which represents its resistance to bending. If a force less than the resistance factor is applied to a jointed part in an object, the object will either move without bending or stand still, depending on the magnitude of other counterforces. If the force applied to a jointed part is greater than its resistance
64 factor, then the joint will first bend and then the object as a whole will move if the force is greater than the sum of all counterforces. Mental spaces are built by placing images into the space. Images can be placed in the space at either an absolute location or at a location that satisfies some relationship with an object already in the space. The relationships that can be used when placing a new image in a space are: on, in front, outside, behind, and under. It is important to remember that these relationships are more than just keys to locating the placement of an image. They are part of the memory network with the full meaning of any node in the network. Thus MIND can do more than place an image on an object; it understands what it means to be on. 7.3 Images Images themselves can be arbitrarily complex and can contain numerous entities. One of the entities in the image can be designated as a focal object in that image. When an image is placed into a mental space, each object from the image placed into the space is given an identifier which is unique in that space. Once an image is copied into a space, the entities in that image are completely independent. The fact that they were part of a single image is not recorded or exploited in any way. For example, container with content is a remembered scene with two objects in it, the container and the content. Its representation as an image would appear as shown in Figure 17. Each of the elements of the array represents part of an object. The parts of each object are represented by integers that are equal after integer division by ten.
65 Numbers one through nine represent one object. Numbers ten though 19 represent another object. The first set, one through nine is smaller by one than the other sets since zero represents an empty location. In Figure 17, Container with Content, the container is represented by parts number one, two and three. There is no significance to the actual value of the numbers; they do not impose or imply any form of ordering. '(0 1 1 1 3
0 0 0 0 3
0 0 0 0 3
0 0 0 0 3
0 2 2 2 3
0 1 1 1 3
0 0 0 0 3
0 0 0 0 3
0 0 0 91 3
0 2 2 2 3)
Figure 17. Container With Content The content is represented by part 91. An object represented by parts 90-99 is automatically interpreted as the focal object in the image. Here the focus will be on the content of the container.
7.4 Placing an Image in a Space When placed in a space, each object will be given an alphabetic prefix unique in that space. Thus after being placed in the space the image may look like Figure 18:
'( 0 0 0 0 0 0 0 0 0 0 D1 0 0 0 D2 D1 0 0 0 D2 D1 0 0 0 D2 D1 0 0 0 D2 D1 0 0 0 D2 D1 0 0 E1 D2 D3 D3 D3 D3 D3 D3 D3 D3 D3 D3) Figure 18. Container with Content After Placement in a Space
66 As part of a full mental image of the two objects, the container and the content, are unique. After being given a prefix, there is no record that they were placed in the space together. When done in conjunction with the creation of a metaphorical abstraction for a perceived situation, the prefix given to an object in space may also be mapped to some element of the actual scene if such a mapping is possible. At the same time that images are placed in a space, any properties of the objects in the image are associated with their portrayal in the space. In the case of the container above, assuming that the sides are attached rigidly to the bottom, this information is stored with object in the space. This creates the ability to make instance-specific modifications to an object in a particular mental space. In a given situation, a container can be modified to have jointed sides without modifying the memory of the container with content. 7.5 Finding Objects In cases where MIND wants to actively determine the location of an object in a space, it can perform an active search. It is important to remember that MIND does not normally rely on initiating such searches but it does have this abilit y. As will be described later, MIND’s filtering mechanism is designed to reduce the overwhelming set of feelings generated to a subset of those that pertain to the focal object in a mental space. While greatly accelerating reasoning about the focal object this filtering may require a more active seeking of information regarding nonfocal objects in the space.
67 Seeking the location of an object in a mental space is implemented as an active operation on the space. This means that as the search takes place, feelings are generated even as objects other than the one being sought are found. This is consistent with the theory that if one is passively relying on active feelings, one only notices what those feelings trigger, but the moment one takes the effort to concentrate on the mental imagery in an active way, one notices more of the relationships among the objects in that space. The nature of the generated feelings will be discussed in a later section but, at this point, it is important to understand that seeking information in a mental space may reveal, through generated feelings, other information unrelated to what is being sought. Since there is no real conscious manifestation of absolute locations in the mental space, MIND’s abilities center around determining the relationship between objects in the space. MIND can determine the following spatial relationships: 1. On (higher and touching) 2. Above (higher not touching) 3. Under (lower and touching) 4. Beneath (lower not touching) 5. In 6. Out 7. In-front 9. Behind The symmetry between MIND’s ability to enact the relationships as described above, and its ability to determine these relationships is important to its ability to
68 perform spatial reasoning. MIND needs to be able to reinterpret the images it creates. MIND can determine the spatial relationship between objects in two different ways. First, it can determine the relationship in a general way, finding how one object is situated in relation to the other. MIND can also begin with a presupposed relationship and test whether or not this relationship actually exists between the objects in question.
Given the more limited scope of the test with the
presupposition, this is a much quicker determination to make but it is also less likely to generate feelings based on unrelated phenomena that exist in the space. MIND’s spatial processor can find the location of either an entire object or a part of an object. Although not currently utilized, this feature will facilitate the mapping of metaphorical object parts such as the sides, top, and bottom of an object in a mental space. 7.6 Moving Objects in a Mental Space Moving objects in a mental space is a complex and very involved task, particularly when a space is somewhat filled with compound objects in close proximity or in contact with one another. The act of moving one object can cause it to apply a force to another object and so forth, such that the original movement propagates throughout the space. Objects can even be arranged in the space in such a way that the propagating force returns to an object via some loop, causing them to temporarily form a sort of single entity for the purposes of the movement.
69 Objects are moved in MIND’s spatial processor in a two-step process. The first step is to determine whether the object to be moved can be moved. Depending on the complexity of the image currently in the processor, this assessment can be quite involved. If the object can be moved, it is moved along with any other objects that need moving. Any joints they may be bent as a result of this move are also bent at this time. 7.6.1 Mass, Counterforce and the Laws of Physics. The propagation of the force of a movement in a space necessitates that the resistance or counterforce propagate back to the original object. The spatial processor must therefore determine the total resistance with which an attempt at movement will be met. Each object has an inherent inertial resistance to movement that is a function of its total mass. Additionally objects can have a resistance factor greater than or equal to zero, allowing a given object to be practically immovable. A mountain is a real world object that could be represented in a mental space as an object with an almost infinite inertial resistance factor. Determining the total resistive force against an attempt at movement is made more complex by the existence of jointed parts. When a force is applied to a part of an object that is connected to the rest of the object via a bendable joint, the total resistance depends on whether the joint bends or not and, if the joint bends, whether or not it is then completely out of the way of other moving objects. Whether or not a joint will bend is itself a function of the magnitude of the force being applied to it and the total inertial resistance posed by the part to which it is jointed, including any objects it might be pressed up against.
70 In Figure 19, object A is up against object B which is jointed to object C. In both parts of the figure, if the resistance of the joint between B and C is less than the inertial resistance of B plus the inertial resistance of C, then the force applied by A will cause the joint to bend. However in the first part of the figure, this bending
Figure 19. Can Bend
frees object A to move without pushing B and C. In the second part of the figure, B still obstructs A. 7.6.2 Friction and Pressure. The spatial processor must also account for friction between an object and any object on which it is resting. The friction is a function of the mass of the upper object and a friction factor which may be associated with the lower object. This friction factor can range from zero (ice) to an effectively infinite number (glue). If the object being moved is under another object, then the mass of the upper object and any object it may be pressed up against must be taken into account when determining the total resistance.
Because of the complexity of performing the determination and the fact that the existence of joints allows object parts to bend, the spatial processor determines
71 the total resistance for each part of each object. A depth first analysis is performed in the following sequence: 1. Each object part it is pressed up against is analyzed. 2. Each object part it is under is tested. 3. Each object part it is on is tested. 4. Each object part to which it is attached is tested. 5. Each object part to which it is jointed is tested. During the analysis, the processor builds sets of encountered object parts, object parts that will move if the move is to be performed, and object parts that will bend if the move is to be performed. 7.6.3 Can Bend? If a jointed object is being, pressured into bending it must be determined whether or not the object can actually bend. The space it will traverse during the bending must be determined and then checked for other objects. In part one of Figure 20, object B will not be able to bend because of object E. In part 2 object B will be able to bend if it is not so tall as to hit object D.
72
7.6.4 Moving. If a move is to be performed, usually because the applied force is greater than the total resistance, then the sets created as described above are used to perform all of the actions implied by the move. This is performed as a sequence of actions starting from the outermost moved object working back to the
Figure 20. Obstructed Bending original moving object. During the actual move, a set of objects that have impacted other objects is created. Current ly, the spatial processor does not exploit this. 7.6.5 Transposing Spaces. The spatial processor is optimized for objects being moved in a positive direction along the x-axis. In order to facilitate a full range of motion, the processor transposes a space placing the direction of a desired movement along the x-axis.
This ability to manipulate mental spaces in their
entirety should also form the foundation for adding the ability for complex imagination involving mental images to MIND’s reasoning engine. If the ability to zoom in or out of detail levels in a mental space is added to MIND’s current manipulation abilities, MIND will be able to perform detailed and complex analyses of imagery in the processor.
73
7.7 Propagating Force MIND’s spatial processor can determine the effect of forces in a mental space. Forces are divided into three types, shown in Figure 21, with each type being built up as a complex combination of simpler forces. The most basic force is a linear force. A linear force travels through a space in a single direction without any outward propagation. It can only affect objects that exist along its direct line of travel. The second type of force is a two-dimensional propagating force. This force also travels in a single direction through the space, spreading as it travels. As it moves through space, it forms a conical expansion pattern and it may affect any object that exists within this triangular subsection of the mental space. The third type of force is the three-dimensional propagating force. This force starts from a single point of origin and spreads outward in all directions forming a spherical propagation pattern. As it spreads, this type of force may eventually affect every object in the space. 7.7.1 Forces and Obstacles. A force’s dispersal pattern may be disrupted by the existence of objects in its path. An object in a space may protect other objects from being affected by a propagating force if it blocks line-of-sight between the other object and the origin of the force.
74
Figure 21. Forces
7.7.2 Forces and Containers. The ability of solid objects to block the spread of a force forms the foundation for the high-level container abstraction. The many facets of containment: common fate, blockage, association, etc. emanate from the force blocking nature of the container’s boundaries. Without forces and the relative ability of solid objects to resist them, there would be no significance to the formation of a container and no concept of containment. 7.8 Feeling Generation Almost everything that takes place in a mental space generates feelings related to one or more of the objects in that space. Feelings are generated when an object is: 1. Moved
75 2. Detected as needing to be moved due to the movement of another object. 3. Touched by a propagating force. 4. Detected while searching for it or any other object in the space. 5. Impacted by the movement of another object. 6. Collided with another object during a move. 7. Placed in a space. 9. Found as needing to push another object in order to move. 9. Found as needing to bend in order to move. 10. Found as having pressure insufficient to cause bending during a move. 11. Bent. 12. Exerting a force in an attempt to move. 13. Met with resistance to a force it is applying. 14. Prevented from moving by a counterforce. All of the feelings generated are sent to a centralized location where they are collected and categorized. 7.9 Feeling Collection and Filtering The spatial processor generates an enormous number of feelings. If all of the generated feelings were allowed to be fed to the reasoning engine, it would quickly be overwhelmed and unable to maintain adequate response times. The feelings generated in the system are therefore filtered according to the currently existing focal object. Only those feelings that pertain to the focal object are actually sent on directly to the reasoning engine. This way, although MIND is “feeling” everything
76 that is going on, it is only actively paying attention to what is happening to that object in the space on which it is focusing. Thus, the feeling sequences being detected will be sequences of feelings related to the focal object. The feelings relating to non-focal objects are not lost or discarded, even though they are not automatically passed onto MIND’s reasoning facility. MIND can therefore, at any time, determine current feelings related to any object in the space or it may determine everything it is currently feeling. Feelings for objects other than the focal object are not accumulated into sequences. It is therefore not possible to trigger concepts based on feeling sequences for objects other than the focal object. 7.10 The Focal Object The focal object in a space can be set in one of two ways. If the space is being created as a representation of a natural language communication, the focal object can be determined from the natural language text. When creating the mental imagery and determining the mapping of entities in the real-world domain to spatial abstractions, MIND will set the object serving as an abstraction of the focus of the real-world domain as the focal object in the space. If there is no known focal object in the realworld domain, or if the metaphor was triggered by a matched feeling sequence, MIND can use a predetermined focal object for the metaphorical abstraction. For example, the metaphor container with content has the content set as its focal object. When a real-world situation is mapped to container with content, the content will be the focal object in the space. This will be true even if the content is not mapped to any real world entity.
77
7.11 Refining Metaphorical Abstractions When determining a metaphorical abstraction to use to represent a situation described in text, MIND first uses the information provided directly by the text. Using this information, MIND finds a match from the metaphorical abstractions it knows. After a match is made however, MIND can use implied information in order to find a better match and perhaps a more detailed metaphorical abstraction, and to create additional bindings between real world entities and metaphorical objects in a process called refining the abstraction. For example, the statement “I will not put that claim in my argument” overtly creates a container, the argument, and an object, the claim. Thus MIND’s first reaction is to represent this situation using only a container and an object. However, use of the word argument also creates implied information. Arguments have conclusions that are the focus of the argument and are metaphorically represented as an object inside the container representing the argument. This is a situation in which an object not even mentioned in the text and, with no current real world binding, can be added to a mental image and used as the focal object in that image. It is important to remember that although this example exploits a small amount of real world knowledge about the nature of arguments, this knowledge is not required. Without the benefit of any knowledge about arguments, MIND would make use of the overt container metaphor. It would still recognize and understand the significance of being in or out of the argument as being able or unable to affect other objects inside the container, respect ively. Thus MIND would still be able to impart meaning to that statement and use spatial reasoning. What it would,
78 of course, be unable to do is to notice the ability to affect the conclusion existing inside the metaphorical container since it would have no knowledge of the existence of the conclusion and, therefore, no knowledge of any special significance it might have. This demonstrates one of the strengths of MIND’s hierarchical evaluation system. MIND can use only the information overtly provided, to reason about a situation, though it has the ability to exploit specific knowledge when possible. 7.12 Tiered Spaces MIND’s pattern recognition works by first matching small recognizable patterns and then combining these patterns to match higher level patterns. These new combinations can be repeatedly combined to match even higher level patterns. By analyzing its input in this way, MIND can create a set of tiered representations of a situation with each separate tier represented in a separate mental space. For example, the statement I am on my way to a trophy involves two tiered spaces. The first space would be created by matching the phrase I am on my way. Although way usually implies a path, that information is not used here; instead, we have a situation represented by one object being on another. The speaker is represented by an object physically located on an object representing the way. This abstraction is useful for understanding the meaning of the speaker going off his way. The completion of the original statement, to a trophy, does indeed exploit a path metaphor. This more complex pattern involves an actor on a path with a goal.
While the first
representation can be used to rapidly determine the meaning of off, only the second representation includes any representation of distance from the goal. Path related
79 concepts such as sidetracked and work-around can only be represented in the second space.
80 CHAPTER VIII THE METAPHORS MIND USES
8.1 Metaphor Classifications To understand MIND’s exploitation of metaphor, it is important to understand the way metaphors are classified for this research. The theory behind MIND divides metaphors into three groups, perceptual, experiential and lexical. The perceptual metaphors might also be called meta-metaphors since they transcend experience and are derived directly from our biology and physiology. They are common to all humans and form the foundation of our perceptions. Experiential metaphors are those that are formed in our mind based on accumulated life experience. These are not static and can be invoked by the use of a common experience such as a metaphor during a conversation. An example of one of these metaphors is “it’s as easy as falling off a log.” This draws on the experience of falling, and requires such experience in order to be understood. Lexical metaphors are similar in nature to experiential metaphors except for the fact that they do not draw directly on experience. A statement such as “the witness clammed up” uses a common metaphor that is understood as a lexical entry or vicarious experience. The knowledge of what a clam is, is incomplete and may often be nonexistent. A person who grew up on a farm in Iowa and never went to the ocean, may have never encountered an actual clam. Still, this does not prevent him from understanding this lexical metaphor any more than it prevents him from understanding ordinary words.
81 The speaker using the metaphor does not actually expect the listener to have any experience with a clam. MIND currently supports four perceptual metaphors: object, container, path and force.
These four concepts, in various combinations, constitute our
understanding of our world. 8.2 Object The object metaphor refers to any physical mass that occupies space and shares our world with us. Objects have a location that cannot be the same as the location of any other object. The object has size and impedance to applied forces. A window, for example, is an object that impedes normal winds but not the winds of a tornado or lines of sight. In human thought and language we invariably turn anything being discussed into an object imparting to it attributes of physical existence as necessary to convey meaning. An example of this is the statement, carrying such a secret is a heavy burden, turns the abstract concept of a secret and the act of carrying into objects. 8.3 Container Our second perceptual metaphor is the container. Unlike the object which is an indivisible whole, the container defines a group of objects, called a collection. Each element of the collection has its own characteristics as an object. It is the container which defines the collection. In the physical world we perceive containers all of the time from our bodies which enclose our “selves” to the homes we live in. It is the boundary (an object or collection of objects) which determines whether some
82 other object is inside or outside of the collection. In the prototypical, spatial metaphor, the container provides a physical separation between what is in, and therefore an element of the collection, and what is out, and therefore not an element of the collection. In abstractions of this metaphor the container may be a Boolean function such as color(T)==green where T is any object. This would define the boundary or metaphorical physical position of the container which contains all green objects. 8.4 Path The third perceptual metaphor is a path. A path is an ordered collection of objects forming a contiguous physical link between other objects. It is this metaphor that is used in describing actions and processes. It is this metaphor which allows us to use terms such as half way finished and stuck when referring to any task. 8.5 Force The fourth perceptual metaphor is the force (not to be confused with a fictional omnipotent power). A force lacks physical substance, but has a perceptible effect. This metaphor is used to describe entities that can cause change in the physical world. In the phys ical world we all encounter the force of grav ity. In life we may complain that a company’s policies are holding us down and preventing us from rising in the ranks.
As described below, change in the real world is
metaphorically represented by movement. Forces therefore are metaphorically represented by pressure upon objects in space, possibly compelling them to move.
83 This spatial representation is a generalized representation used even for abstract forces such as hunger, greed and heat. 8.6 Relationships As entities in the world are mapped to spatial entities, or combinations of spatial entities, the complex relationships between real world entities are themselves mapped to spatial relationships. Once trouble becomes a container, one can be in trouble and out of trouble. One can be on time because one arrived in time to sit on the committee that has power over budgeting. It is the combination of objects and relationships that creates imagery in the human mind. 8.7 Change and Motion Given a complete representation of a situation via mappings to spatial entities and relationships, change in the real world is represented by change in the spatial representation or motion. This leads to the representation of actions as movements along a path in space. As the action can be broken down into a sequence of changes in relationships, this sequence can itself be represented as a contiguous, ordered set of objects in space forming steps in a path.
This gives a physical, spatial
representation to any form of change, no matter how abstract. 8.8 Hierarchical Complexity While presented as forming the underlying foundation of spatial metaphors, these basic metaphors are completely dependent on each other, and can not be placed into a traditional inheritance based hierarchy without forming a circuitous network
84 or making difficult and perhaps limiting decisions about which metaphor should inherit attributes of which other metaphors.
These top level metaphors are
represented in MIND in a network that does not require direct inheritance of attributes. Rather, it designates possible forms for perception. Path is a top level metaphor but a path may also be perceived as a single object. It may also be seen as a container which has as its content the steps along the path. It may also be perceived as a collection of objects representing the steps along the path. Thus, in MIND’s metaphor network, path is linked to object. This, however, does not designate that path inherits from object. Instead, it allows any concept which can be perceived by MIND as a path to also be perceived as an object. Since not all objects can be perceived as a path there is no route through the network from object back to path. 8.9 The Metaphor Network Figure 22 portrays a portion of the top of MIND’s metaphor network with its hierarchical structure. The four basic metaphors are shown with a bold outline. The basic metaphors do not form the root of the hierarc hy. In fact, the basic metaphor for action, the path, is several levels down from the top. The determining factor in the importance of a metaphor is not its level of simplicity but its centrality to the human experience and its prevalence in human reasoning. The relations in the network are classified by the type of metaphorical imagery they require. Outside and inside are both container relations, since they have no meaning without the existence of a container. On is an object relation, since it only requires and implies the existence of objects.
85
8.10 Inclusions Action metaphors and relation metaphors require the existence of one or more entities to give them meaning.
These required entities are stored within the
metaphor nodes as a set of included entities as portrayed in Figure 23. In addition, they also exist as elements in the prototypical image for the metaphor. The top most action metaphor, action, is actual a mapping of the metaphorical prototype path to a real world entity. Below action we find do. To have doing one must have a task and some entity performing the task; its metaphorical representation requires the existence of a path and an actor to move along that path. With the path representing the task and the actor representing the performer of that task, the actor must be on the path. Action can therefore use the spatial relation on to depict its prototypical image. Action must include the information that it is the actor who is on the path. The spatial processor knows in its own right how to place one object on another and also understands on when it exists such that it can interpret the image it creates as on. The images it will actually draw in its mental imagery will be the images stored with actor and path. 8.11 Feelings As described throughout this work, it is not enough for MIND to know what metaphorical prototypes look like in order to draw them or even to recognize them when they exist. MIND must know what they feel like. The ability to create an image of an actor on a path is only of value for feeling-based reasoning if MIND knows what on feels like.
86 The node for on includes feelings and activation levels for those feeling that in the aggregate, define what on feels like. These feelings can be activated by the concept on or on can be activated by the coappearance of these feelings. Figure 24 shows feelings linked into the network and included in the concept on.
87
Figure 22. Inclusion in the Metaphor Network
88
Figure 23. MIND’s Metaphor Network
89
Figure 24. Feelings
90 CHAPTER IX BUILDING HIGHER LEVEL CONSTRUCTS 9.1 Higher Level Constructs Higher level constructs can be built out of the perceptual metaphors. When working on agents, one of the higher level constructs has to be that of the agent. This concept can be built by combining an object, a container and a 3-D propagating force. The object is surrounded by a container known as its region of accessibility as shown in Figure 25. The object may exert a force which will affect other objects within its region of acce ssi bil ity. The exact nature of the region of accessibility depends on the type of force being applied. If one is yelling, the region of accessibility encloses all entities within ear-shot. If one wants to push something, it
Figure 25. Region of Accessibility includes only entities less than an arm’s length away. It can also be a totally abstract concept. If I only know algebra but not calculus, and I want to find the area under
91 curves on a graph, my region of accessibility only includes those curves that are actually straight lines. 9.2 Perceptual and Lexical Metaphors So far MIND has been explained in terms of its ability to handle concepts that are perceived in terms of its four perceptual metaphors. This ability to reason with more complex metaphors can be built upon this foundation, exploiting MIND’s existing abilities. In addition to the four perceptual metaphors discussed so far in this paper, other metaphors, from the experiential and lexical class, also pervade our language and reasoning. The concept of a home is a common one. We feel stronger and more confident on our home turf than we do in unfamiliar surroundings. A student at IIT will feel more at home at the Galvin Library than he would seeking a book at one of the libraries at Northwestern. A sports team feels better about home games than it does when it is on the road. MIND has the ability to create this higher level metaphorical perception by building on its existing base of perceptual metaphors. Since MIND views the ability to cause change in terms of forces, being at home and feeling able to do more can be modeled as increasing the magnitude of the forces MIND can apply. MIND’s use of comparisons when determining whether one force will overwhelm another, combined with an increase in what it thinks is the magnitude of its own forces, causes the MIND’s reasoner to assume it can do more than it otherwise would.
92 The lexical metaphor clammed up can also be modeled in terms of the four perceptual metaphors. When we say someone clammed up we mean that someone suddenly ceased to provide information, as in “the witness clammed up and refused to testify any more.” Described in terms of the four perceptual metaphors, we have an actor applying a force to move information from another actor’s container to it own container via its ears. This force suddenly ceases to cause the information to move despite the fact that more information exists in the other agent’s container. MIND would also stop feeling the flow of information. A feeling of shock will be generated, as MIND suddenly ceases to progress toward its goal of acquiring information. Since MIND perceives its world in terms of the four perceptual metaphors, this description portrays the way MIND will view what is happening when clamming up takes place. Thus by having a name assigned to this sequence of events and feelings, MIND can both recall the phrase clam up when it is happening, and understand the term when it is spoken.
93 CHAPTER X A F E E L I N G B A S E D L E X I C ON 10.1 The Lexicon The feeling-based lexicon is not intended as a replacement for standard lexical definitions. The lexical entry for blocked, for example, includes direct activation of the feeling nodes force, counterforce, obstruction, and balance, the nature of this entry is explained below. As I will describe, although much information is included in these feeling nodes and their links, there is no attempt to define what is and is not blocked in absolute terms. 10.2 Using The Definition As shown in Figure 26, the feeling-based definition of blocked tells us how it feels to be blocked [Lakoff 1987]. The figure shows that, when blocked, one is focusing on an obstruction that is an object along a path. This constitutes the current imagery. In addition to the imagery, there is a feeling sequence that determines blocked. The seqence of feelings is that of applying a force, encountering an object, feeling pressure from that object, and not moving, is portrayed in the diagram as internal force, obstructed, pressed, counterforce, null motion. Further, while this set
94 of feelings leads up to a blocked situation, the methods that constitute a continuation of events are, trying harder, going around, and giving up shown in the figure as add
F e e l i n g
S e q u e n c e f o r B l o c k e d
Figure 26. Feeling Based Definition of blocked
95
96
97
98 effort, circumvent, and submit. These feelings, when coupled with mental imagery [Ko ssl yn 1980], allow reasoning about situations in which one feels blocked without the dictionary definition of blocked. The feeling-based definition also allows an entity such as an agent to construct the lexeme blocked from an aggregate of feelings created either by feedback from its environment or by processing mental imagery in an imaginary space. In either case, we are again at the situation where there is no dictionary definition of blocked yet the system will activate the lexeme blocked, by virtue of feeling the way blocked feels. 10.3 The Semantic Network The semantic network itself is based upon two key elements, feeling and phrasing. The lexical entry for a given item is not intended to be an encompassing definition of necessary and sufficient conditions for an entity to be called by that lexical entry, nor is it intended to be a list of possible variants on those conditions as one usually finds in a lexicon. Rather, the lexical entry is a set of links into the metaphorical hierarchy and feeling nodes [Greene, 1995] that define in what form we perceive and how we feel about those entities that can be categorized under the given lexical entry. The entry for a single lexeme includes a combination of feelings and associations that trigger other feelings and mental imagery. This linkage between lexical entries and elements of reasoning yields the ability to analyze situations at the highest level of abstraction and to express responses at that level. It is at this level that natural discourse takes place allowing the system to be either an entity capable of expressing situations, or a full partner to natural discourse. Figure 26 portrays a segment of the lexicon related to blocked. Blocked is directly linked to force, counterforce, obstruction and balance. Obstruction is
99 designated as the focal object in a blocked situation and as such may become the focus of reasoning processes. The concept of obstruction itself is linked to object and path thereby linking blocked to these concepts. Additi onally, blocked is linked to a feeling sequence that, protot ypically, precipitates a body-based feeling of blocked. Last ly, blocked is linked to a collection of generalized methods, each of which is a prototypical response to a body-based feeling of blocked. 10.4 The Entry and Its Direct Links For example, the entry for in [Jackendoff, 1992] shown in Figure 27, is
Figure 27. Network Entry Points for In represented in the semantic network by the entry points container, obstruction and safety. This may at first appear a rather sparse definition of in, but one must consider the following points. First, is the richness of the metaphorical entry container. The concept of container is itself linked to a wealth of feelings and mental imagery giving context and allowing meaning for the concept in. The definition of container
100 in the network consists of links to those relationships which imply the existence of containment such as in, out, inside, outside, with and without, those feelings associated with the existence of containment, imagery of containment, and feeling and image sequences [Greene and Chien, 1993] which involve containment. As this type of definition exists as active elements of the reasoning engine, perceived situations can be directly reasoned about, and the outcome of reasoning can be expressed as elements of the lexicon. Second, the concept of obstruction is also a link to a wealth of information in the network, which includes not only feelings and imagery, but also presuppositions and feeling and image sequences. While container can exist on its own, the concept of blockage requires several elements to exist. The most basic are the actor, the obstacle, a perceived path from the actor’s current position through the position of the obstacle, and a perceived attempt by the actor to move along the path in the direction of the obstacle without any advancement along the path as a result. It should be noted that many of the higher level concepts in the above explanation are themselves composite feelings and imagery elements. The attempt at movement consists of the application of a force resulting in a simultaneous counterforce which does not lead to movement towards a goal. The concept of safety is also a composite consisting of an actor, a perceived threat, and a boundary stronger that the threat. 10.5 The Indirect Links and Propagating Activation of Feelings
101 The three entry points, container, obstruction, and safety are connected into the rest of the network via links that are used to propagate activation levels. Each link has an activation coefficient c, where 0< |c|
