PROGRAMMING AGENTS WITH VISUAL RULES Richard Robinson, Devin Cook, and Steven Tanimoto
Department of Computer Science and Engineering Box 352350 University of Washington Seattle, Washington 98195-2350 USA Email:
[email protected]
ABSTRACT
A visual programming facility is described which is based on a visual rule formalism. It permits programmers to specify the behaviors of agents in the class of program objects in the Agentsheets system of Repenning. The resulting agents can be used in designing simulations for two-dimensional cellular worlds such as in automobile trac ow situations, video games, and cellular automata studies. The programming facility, called \Agent Builder," is integrated into the Agentsheets system (providing a substitute for the nonvisual AgenTalk editor) in such a way that agent programming becomes a process of arranging icons in a worksheet, using a special gallery of icons. Several applications of Agent Builder are presented, and its limitations and possible extensions are discussed.
1. INTRODUCTION For at least three decades, computer technologists have attempted to create programming environments that would enable novices to program computers. In some cases the goal has been to liberate professionals such as scientists from the need to hire programmers. In others, the objective has been to make programming child's play [Perrone and Smith 1995]. As argued by Papert, computer programming can be a learning activity that empowers children to create mental models for robot-turtle control systems and mental models for natural processes [Papert 1980]. More recently, Resnick has extended LOGO to allow children to study the phenomena that emerge from a multitude of concurrent processes [Resnick 1994]. Both the original LOGO and Resnick's Star LOGO, however, are textual languages, which require that children be able to read and be able to do keyboarding on the computer. Even if the children are old enough to read and type, these languages are unnatural if the children's native language is not English. Thus there has
been interest in the use of visual languages for children's programming. Visual languages for children's programming have usually employed either control- ow, data- ow, or rulebased approaches. One control ow approach is the use of linear scripts in PLAY [Tanimoto and Runyan 1986]. Such a script consists of a sequence of iconic sentences, each of which expresses a short action sequence by combining a play character with an action (e.g., the bear walks to the right). Another control- ow children's programming system was a commercial system called ChipWits. In this system, a robot-like agent would move through a two-dimensional maze under the control of a visual program created by the child. The program was a ow chart whose nodes contained iconic descriptions of conditions and actions. These condition/action pairs could be considered as visual rules. The use of visual rules for programming is a powerful approach, because the control semantics are easy to understand and the the conditions and actions themselves can be arbitrarily complicated. A visual rule describes what should happen in a particular situation during a simulation. The condition part of the rule speci es a particular class of states, typically characterized by certain components of the world being in a speci ed con guration. In the PRoLogicA system [Yeung 1989], a rule consisted of two diagrams: the \before" diagram and the \after" diagram. The before diagram speci ed a condition in a portion of an array or tree using patterns that could be processed by a two-dimensional uni cation algorithm analogous to that used in PROLOG. The after diagram speci ed the structure into which the matching part of the world is to be transformed. Those components of the world that matched variables in the pattern could be composed in the after diagram, the composition being speci ed by the reuse of the patterns from the before diagram in the after diagram. More recent systems using visual rules include BITPICT [Furnas 1991], which uses cellular logic-like trans-
formation rules (like those used in parallel image processing [Preston and Du 1984]), and ChemTrains [Bell and Lewis 1993], which uses labeled line drawings as the visual representations in rules. While PRoLogicA used topological properties of the drawings to distinguish invariants from structures to be changed in matrices and trees, ChemTrains permits a more general object, called the \simulation display," to be manipulated. Unlike BITPICT and the system Agent Builder that we describe here, but like PRoLogicA, ChemTrains does not make use of a cellular division of the workspace. On the one hand, avoidance of the grid permits more exible matching, but on the other hand, it makes it more dicult to specify rules whose conditions include empty space, distance-sensitive conditions, or size-sensitive conditions. This paper presents a visual-rule programming capability that combines the immediateness and obviousness of visual rules with an existing powerful simulation system (Agentsheets). Agentsheets is a simulation/programming environment built on Macintosh Common Lisp which provides drawing tools for icons ([Repenning 1994]), drawing tools for worksheets, a parallel simulation control structure, and Lisp-like primitives for de ning behaviors of agents ([Repenning 1993], [Repenning and Citrin 1993]). Our system illustrates how the same worksheet structures that are used to lay out example worlds for simulation can also be used for the programming that is necessary to de ne the behaviors of new agents. We thus attain a degree of homogeneity between programming at the agent-de nition level and simulation worksheet level.
2. APPROACH USING AGENTSHEETS Agentsheets provides an accessible environment for exploring interactions among software agents and their surroundings. However, Agentsheets does not provide a simple way to de ne the behavior of depicted agents. Specifying agent behavior requires a user to write programs in the textual \AgenTalk" language embedded in Lisp (Apple Computer's Macintosh Common Lisp) and using a unique object oriented extension called OPUS [Repenning 1991]. Our implementation of Agent Builder attempts to mitigate this problem by allowing a user to de ne the behavior of agents by arranging icons on an agentsheet. Agent Builder analyzes and parses the pictographic program representation and creates MCL de nition code for the agents depicted in the agentsheet. AgentSheets is illustrated schematically in Figure 1. The AgenTalk editor, highlighted in the diagram, is a non-visual programming component, and Agent Builder, our new module, provides a visual substitute for it. Our goal was to create an extension to Agentsheets that makes it easier to program agents. We hope that
End-user level
Designer level
Gallery
Worksheet
Expanded Gallery
Class Browser
Depiction Editor
AgenTalk Editor Tool Store
Figure 1: Agentsheets system components as seen by end users and by designers (after Repenning, 1994). Our visual Agent Builder module is a substitute for the non-visual AgenTalk editor.
non-expert users will be able to use Agent Builder to design and implement simple interacting agents without previous programming training or experience and with little instructional aid. This can be seen as an effort to narrow the gap between end-user and designer in permitting end-users to participate in agent design without having to learn AgenTalk. We considered two approaches to the problem of de ning actions and reactions of agents in Agentsheets without having to write code in Lisp. These approaches were (1) an interactive training system along the lines of KidSim's \programming by demonstration," and (2) an iconic rule de nition system. An interactive training system works by allowing the user to play the part of the agent. Programming is accomplished by manipulating agents (typically with a mouse) to demonstrate desired behavior in speci c situations. The KidSim paper ([Smith et al 1994]) explores an example in which a user trains a monkey to maneuver over a rock placed in its path. The user moves the monkey over the rock, and the rule associating the before and after situations is recorded so that it can be used by monkey agents whenever they encounter a similar condition. For purposes of our project, programmingby demonstration posed two problems. As Smith, Cypher and Spohrer point out, programmingby demonstration lacks explicit mechanisms for sequencing rules. It is dicult to specify or understand the overall behavior of an agent since only one situation can be viewed and edited at a time. We feel that when a user examines a potentially complex agent in a rich environment in such a
system, he or she should have visual access to the full range of the agent's capabilities. More practically, we viewed this approach as cumbersome or complex to implement, and ill-supported by the existing capabilities of Agentsheets. The approach we have chosen to pursue, then, is that of designing a rule authoring system in which icons are arranged to visually depict each action and reaction of the agent to its surroundings. We conceptualize the multiple depictions mapped to an agent subclass as being multiple states that the agent can be in, permitting the de nition of complex behavior for agents via a network of state-changing rules. The Agent Builder programming environment consists of a base gallery, into which the user or programmer adds his own agents' depictions, and one or more worksheets in which the user programs the rules governing agents' behavior. Our review of several working Agentsheets applications indicated that most behavioral de nition code (class code) is built upon one basic construct, which might be expressed in English as: If I'm a Depiction1 and to the Direction1 of me is a Depiction2 then I'll do something.
These are typically compounded into case statements with a meaning to the e ect: If I'm a Depiction1 and to the Direction1 of me is a Depiction2 then I'll do Something1, If I'm a Depiction1 and to the Direction1 of me is a Depiction3 then I'll do Something2, If I'm a Depiction1 and to the Direction1 of me is AnythingElse then I'll do Something3.
The car and road example in the Agentsheets user manual exploits this format, having class code characterized by forms such as: (case (effect (0 1) 'depiction) (road (self 'move 0 1)) (t (play-sound 'honk)))
To enable the speci cation of this construct in a visual context, we rst examined the possibility of creating a state/action array. A portion of such an array might look like that in Figure 2. When an agent needs to take action based on the object to its right, it would nd its own depiction at the start of a row in the right-neighbor table, and behave according to any actions found under the appropriate column heading. In this example, if an agent's depiction (using Repenning's example) is hCar-Red-Ei and to the right of it is
Figure 2: Portion of a state/action array for agent programming.
a hRoad-E-Si, then the agent should move over one to the right and change its own depiction to hCar-Red-Si. While the simplicity of this rule construction mechanism has de nite appeal, it's clear that the size of the rule de nition matrices will quickly get out of control, with the matrix growing at a rate O(n2 ) of the number of depictions involved in interactions. The workspace must be big enough to accommodate all the actions an agent performs for each potential neighbor, so the columns could easily end up spaced far enough apart that not all combinations of agents and actions would be visible on the screen at once. Our solution was to model a visual syntax upon our original English version of the case construct. This made the visual language easier to design, though it gives the language a syntax similar in some ways to that of traditional text-based programming languages. With this fundamental design feature established, most of the rest of the language fell into place. We realized early on that if Agent Builder supported only the classic If-Then semantics, it would be necessary to write individual If (or Question) statements for every new potential neighbor. Some sort of Else token could provide default handling, but this would overwhelm the class de nition workspace with visual noise. To provide maximum functionality with a minimum of repetition, Agent Builder uses the spatial relationship of icons to align the appropriate hIfi's with possible predicates, producing a Case statement, instead of a simple If-Then statement. The example in Figure 3 illustrates this. This fragment translates into English as roughly: \If my Depiction is Car-Red-E and to the Right of me is a Road-EW, MoveRight. If my Depiction is Car-Red-E and to the Right of me is a Road-E-S, MoveRight and TurnSouth. If my Depiction is Car-Red-E and to the Right of me is a Car-Red-W, Erase it and Explode. If my Depiction is Car-Red-E and to the Right of me is anything else, become a Car-Parked-E." Introduction of a compound hIfi statement, default handling with hOthersi, and visually cascaded inher-
Figure 3: Portion of a state/action array representing multiple cases.
itance in our syntax makes it possible to herd many cases into one statement. Without this ability, the code could grow in size O(n2) as new clauses are added to support each agent depiction's handling of new neighbor depictions, along with the primary statements for the new depiction itself. Finally, in order to make Agent Builder work without additional elements in the Agentsheets tool store, we created an active agent called Compile. The programmer positions this agent at the end of the class de nition workspace, delimiting the code area. When the programmer \whacks" the hCompilei, it generates Lisp class creation and method de nition code for the agent subclass described by the worksheet.
3. PROGRAMMING IN AGENT BUILDER The Agent Builder base gallery (not shown due to space restrictions) contains all the functionality directly available to the programmer in Agent Builder. The tokens in the base gallery are shown in the following table: If Depiction Others Operate-On Whack Erase
Create a case construct. Query depiction. Default behavior indicator. Method name. Method name. Method name. Default: (self 'erase). New-Depiction Method name. Defaults to (self 'depiction new-depiction). Foreground-Task Method name. Compile Code terminator and compiler agent. Left Direction modifier. Right Direction modifier. Up Direction modifier. Down Direction modifier. Move-Left Method name. Move-Right Method name. Move-Up Method name. Move-Down Method name.
This may seem a somewhat poverty-stricken semantic base, but the language is extensible in that the programmer can use custom depictions to \name" the methods being described. Programming in Agent Builder is accomplished by creating a class de nition worksheet. Agent subclass creation and behavioral speci cation occupy three distinct areas of this worksheet. The top row of the class de nition worksheet contains a list of all depictions for the agent subclass whose behavior is being de ned, with no spaces between icons. The names of the rst two icons in the row determine the name of the agent sub-class, though the programmer does not need to know or use this information. The second line in the class de nition worksheet is left blank. Our intention is that this area might be used in the future for variables that could carry agent state information or other data, as discussed in the Future Work section, above. Method de nitions for the agent subclass begin on the third line of the worksheet. The left-most column contains icons that are depictions for the methods being de ned. In the example in Figure 7, the OperateOn and ForeGround-Task methods are de ned for the agent sublass CAR-AGENT. Statements in the Agent Builder language are built from Actions and Questions. Actions are method names or message transmission calls, such as Whack, OperateOn, Move-Right, and Depiction. Actions can be preceded by Direction Modi ers. The Direction Modi ers (Right, Left, Up, Down) simply translate or send the message to a neighboring agent. If no Direction Modi er is used, the message is directed at Self. A Question introduces a statement which conditionally executes other statements, much like the if-thenelse, cond, or case statements found in character-based programming languages. A Question consists of a the Question icon, hIfi, an optional Direction Modi er (one of hLefti, hRighti, hUpi, or hDowni), a query type (currently only hDepictioni, is explicitly supported), a target depiction, and a statement to be executed if the queried depiction matches the target depiction. An example of the simplest possible Question statement is shown in Figure 4.
Figure 4: A very simple question.
The condition in this example is, \Am I a Car-RedE?" If so, send a \Whack" message to Self. Another example, this time using Direction modi ers is given in Figure 5. This example asks the question, \Is the Agent to my right a Car-Red-E?" If so, send the \Whack" message to the agent above.
Figure 5: Illustration of direction modi ers in the condition a rule.
Note that a Direction modi er modi es only the message immediately following it. To send multiple messages to an adjacent agent, each message must be preceded by a Direction Modi er. Agent Builder provides default case handling with the depiction token hOthersi. This provides a mechanism for letting an agent take action in situations where it doesn't have a speci c rule. For example, in CarWorld whenever a hCar-Red-Ei has something to the right of itself that it doesn't know how to handle, it changes its depiction to hCar-Parked-Ei. The use of hIfi with hOthersi to generate complex case constructions with a visually cascaded syntax is discussed above, in the Approach section. Finally, to put the class de nitions to work, one puts a hCompilei in the rst column of the row at the end of the class de nition worksheet, delimiting the class de nition code. When the hWhacki tool from the tool bar is used to send a \whack" to the hCompilei, it generates and executes Lisp class creation and method de nition code for the agent subclass as described by the worksheet. As an aid to debugging, it's possible to use hOperate-Oni instead of hWhacki on the hCompilei. This causes the generated Lisp to be prettyprinted for examination in the Listener window.
4. RESULTS Although we have not conducted a comprehensive survey to evaluate the usability of Agent Builder, the reactions of our friends and colleagues have been very encouraging. We are especially pleased that associates who do no programming seem to catch on quickly to what is going on in a class de nition worksheet and express con dence that they could modify the ones they've seen or create new ones themselves. Figure 7 illustrates the Agent Builder de nition for the simple car class in the Agentsheets manual. The AgenTalk code produced by Agent Builder from the visual representation is shown below. (IN-PACKAGE :AGENTS) (CREATE-CLASS CAR-AGENT "Active-agent subclass generated by AGENT-BUILDER parser." (SUB-CLASS-OF ACTIVE-AGENT)) (D->C CAR-RED-E CAR-AGENT) (D->C CAR-RED-E-3 CAR-AGENT) (D->C CAR-RED-E-4 CAR-AGENT) (D->C CAR-RED-E-2 CAR-AGENT)
(D->C PARKED-E CAR-AGENT) (D->C PARKED-E-3 CAR-AGENT) (D->C PARKED-E-4 CAR-AGENT) (D->C PARKED-E-2 CAR-AGENT) (DEFMETHOD FOREGROUND-TASKS () (INSTANCE-METHOD-OF CAR-AGENT) "Method generated by AGENT-BUILDER parser." (SELF 'OPERATE-ON)) (DEFMETHOD OPERATE-ON () (INSTANCE-METHOD-OF CAR-AGENT) "Method generated by AGENT-BUILDER parser." (CASE DEPICTION (CAR-RED-E (CASE (EFFECT (0 1) 'DEPICTION) (ROAD-E-W (SELF 'MOVE 0 1)) (T (SELF 'DEPICTION 'PARKED-E))))))
Although the user must still learn a syntactic system and the meanings of several icons, the visual representation is arguably less cryptic to non-programmers than the Lisp-based representation. An Agent Builder representation of an example from a paper on KidSim [Smith et al 1994] is shown in Figure 8.
5. FUTURE WORK We feel there is at least a nugget of usefulness in our approach to extending Agentsheets, though we must acknowledge that much remains to be done before our implementationwould provide signi cant utility. Three aspects in particular stand out as needing further re nement. First, there are several ways that Agentsheets could be modi ed to provide for better integration of Agent Builder. Worksheet tools for inserting and deleting rows and columns would make it easier to edit the class de nition worksheets. The ability to change the current gallery selection (and thus the drawing selection) by sampling an existing element in the worksheet would reduce the amount of switching necessary between the gallery window and the worksheet being constructed. A capability to pick agents from more than one gallery would obviate the need for starting from our Agent Builder base gallery, though that would permit name con icts that would need to be handled. We would prefer that depictions retain their relative locations in the gallery le when it is reopened, since we nd it useful to organize the agents there for quicker access. The second aspect which needs to be addressed is that of extending the Agent Builder language to increase the complexity of the tasks available to agents. Among the many possible extensions and additional capabilities one might want to include are: (1) Variables to allow for more information states than the depictions alone carry. We had reserved the second row of the class de nition workspace for this purpose.
(2) Several other types of programming operators such as booleans, counting, simple arithmetic, and randomization. (3) Additional Query types (only Depiction is currently supported). (4) Additional direction modi er capabilities for diagonal access and access beyond adjacent neighbors. (5) Looping constructs. (6) Exploiting the agent-linking facilty provided by Agentsheets. The third area in which we feel additional work is needed is simpli cation of the visual language syntax. It should be easier to explain and understand Agent Builder class de nition worksheets, and presently it takes too much repetitive work to build a class de nition worksheet. We consider three syntactical changes that might provide relief in these areas. An Agent Builder program is somewhat redundant. The two most obvious examples of this are hDepictioni and hNew-Depictioni, which respectively query and alter an agent's depiction. In the case of the Depiction token, we've already remarked that Agent Builder currently supports only one kind of Question, namely the hDepictioni query. Even if we were to add other query tokens, hDepictioni should be the default query since the depiction is considered to represent the agent's state. Just as a direction modi er is used when a message is directed at an agent other than Self, a Query modi er could be used to change query type. In the case of the New-Depiction operator, since any new depiction can be found in the list of a subclass's depictions, the use of hNew-Depictioni seems redundant. A depiction's most obvious Action value would be, \Change depiction to me." The Agent Builder visual language provides enough extensibility to allow the programmer to de ne methods for the depictions in just this way. See the KidSim monkey reconstruction (Figure 8) for an example of how this can be done. The Robo-World example (see Figure 9) demonstrates another opportunity for reducing redundancy in the behavioral speci cation code. By adding a NextDepiction and Previous-Depiction method automatically to every class, we could greatly simplify the creation of such sequential state depictions. As the robot's battery discharges, the robot would go to the next depiction (i.e. state). As it recharges, it would go to the previous depiction. The notion of default Next- and Previous- states could be supported with a syntax modi cation that would allow for the same Question to be processed for multiple depictions. An example is shown in Figure 6. In this example the Question is asking, \If I'm a Robot-R-5 or a Robot-R-4 or a Robot-R-3 or a RobotR-2 or a Robot-R-1 then if what's to my right is a
oor, then move right and go to the next state" In this case, Robot-R-5 would become a Robot-R-4, Robot-R4 would become Robot-R-3, and so on.
6. REFERENCES 1. Bell, B., and Lewis, C. 1993. ChemTrains: A langauge for creating behavior pictures. Proc. VL'93, Bergen, Norway, pp.188-195. 2. Furnas, G. 1991. New graphical reasoning models for understanding user interfaces. Proc. CHI'91. pp.71-78. 3. Papert, S. 1980. Mindstorms: Children, Computers, and Powerful Ideas. New York: Basic Bks. 4. Perrone, C., and Smith, J. (eds). 1995. Child's Play WWW Home Page, with links to abstracts of the talks presented at Child's Play: A HandsOn Workshop on End-User Programming, held at Boulder, CO, Feb. 23-24, 1995. http://www.cs.colorado.edu/~corrina/childs-play.html. 5. Preston, K., Jr., and Du , M. J. B. 1984. Modern Cellular Automata: Theory and Applications. New York: Plenum. 6. Repenning, A. 1991. The OPUS User Manual, Asea Brown Boveri Research Center Arti cial Intelligence Group, Baden, Switzerland. 7. Repenning, A. 1993. Agentsheets: A Tool for Building Domain-Oriented Dynamic, Visual Environments. Ph.D. diss., Dept. of Computer Sci-
ence, Univ. of Colorado, Boulder. 8. Repenning, A. 1994. Bending icons: Syntactic and semantic transformations of icons. Proc. VL'94, St. Louis, MO, pp.296-303. 9. Repenning, A., and Citrin, W. 1993. Agentsheets: Applying grid-based spatial reasoning to humancomputer interaction. Proc. VL'93, Bergen, Norway, pp.77-82. 10. Resnick, M. 1994. Turtles, Termites, and Traf c Jams: Explorations in Massively Parallel Microworlds. Cambridge, MA: MIT Press.
11. Smith, D. C., Cypher, A., and Spohrer, J. 1994. KidSim: Programming Agents Without a Programming Language. CACM, Vol.37, pp.54-68. 12. Tanimoto, S. L. and Runyan, M. S. 1986. PLAY: An iconic programming system for children. In Chang, S. K., Ichikawa, T., and Ligomenides, P. A. (eds.) Visual Languages, New York: Plenum, pp.191-205. 13. Yeung, C.-F. R. 1989. A Graphical Programming
Environment with Pictorial Representations of Logical Axioms. Ph.D. diss., Dept. Computer Sci.
Univ. of Washington, Seattle, WA. (Available from Ann Arbor, MI: University Micro lms).
Figure 6: Handling multiple alternative depictions in a subcondition of a rule.
Figure 7: Simple car class de nition.
Figure 8: Agent Builder program for KidSim example.
Figure 9: Portion of de nition of Robo-Class. Approximately the rst third of the de nition is presented.
