Enhancing 3D User Interfaces with 3D Animation Encapsulated in Agents Christian Geiger,
[email protected] Volker Paelke,
[email protected] C-LAB Fürstenallee 11 33102 Paderborn, Germany http://www.c-lab.de/vis (Visual Interactive Systems)
Abstract In.this paper we describe how techniques from traditional cartoon animation can be used to improve user orientation, interface presentation and interaction feedback in 3D environments. By encapsulating the principles of traditional cartoon animation in agents, that implement the interface components animation behaviour, their use in 3D user interfaces becomes simple and efficient for the interface designer.
feedback. Adding more detail and better animation seems like the obvious solution to these problem, however a number of reasons preclude this "easy" way:
• Graphic performance is still a severe limitation, and while high-end graphics supercomputers may be able to support the necessary level of detail, workstations and PC's - despite massive improvements in recent years will limit the complexity of scenes and thus the amount of detail in interfaces for the foreseeable future.
• Additional detail is not cost effective. The design of 3D 1 Introduction The design of usable 3D interfaces for desktop visualization and virtual reality applications is a challenging task. Probably the most promising aspect of user interaction in 3D environments is to make the interaction more "life like", mimicking realworld tasks as closely as possible and thus enabling users to control an application with their everyday skills. While the potential of 3D interfaces has been largely recognized, technical constraints and the lack of experience have limited their practical application so far. One central problem in 3D applications is the trade-of between detail and speed that is always involved in realtime 3D animation. To improve user understanding of the interaction possibilities offered by a 3D environment and to increase his perception of changes in the environment would require additional cues and more realism in the 3D scene. The problem becomes even more important in collaborative multi-user environments, where the user has no way of anticipating actions of other users, so that the cognitive effort to make sense of sudden changes in his environment becomes significant. Closely related is the problem of disorientation that quickly arises in virtual environments of non trivial size, since little feedback about movement is presented apart from the visual display of the current user perspective and places tend to look very similar. In the following sections we will refer to these problem areas as user orientation, interface presentation and interaction
environments is already a cost intensive task and the addition of detail increases this cost even further. Additional investment in detail is only warranted if usability increases at least proportional to the additional cost. The above suggests to look for alternative ways to provide additional cues to the user and to increase the "appearance" of life-likeness (as opposed to closer emulation of reality through more detail) in order to make 3D interfaces more understandable and thereby improve their usability. Cartoon animation provides interesting insights on how the appearance of "life-likeness" can be improved without a large amount of detail or actual realism. Cartoon animation uses exaggeration and decidedly "unrealistic" techniques or tricks to reinforce the illusion of reality. The user is not consciously aware of these tricks. They work in a subliminal way to make the animation more understandable and thereby seemingly more realistic or life-like. Techniques borrowed from cartoon animation have been successfully applied to 2D user interfaces and with state of the art 3D animation tools it is also possible to apply them to 3D environments. We are currently experimenting with an agent based approach that encapsulates knowledge on cartoon animation principles, so that they can be applied easily in 3D user interfaces. This paper describes our current effort to design a system for rapid prototyping of 3D user interfaces. Our goal is to combine sophisticated visual techniques (like interactive
real-time 3D animation) with knowledge based concepts (like multi agent systems) to simplify the design of 3D interfaces and to enhance their usability. In this paper we present our initial ideas concerning the combination of cartoon animation techniques with an agent based approach for use in 3D GUI design. Also, we identify possible application scenarios and describe the main components of our proposed system. Our work is still in a preliminary stage, so that this paper serves more as a research proposal than a description of a working system. The following sections are organised as follows: First we introduce some principles of cartoon animation and indicate how they can be used to enhance 3D interfaces. In the following section we motivate the use of a multi agent system to implement the interface components and describe the general structure of the resulting "widgents" (widgets + agents). The visual and abstract behaviour of the widgents could be arbitrarily defined, but we currently restrict our considerations to cartoon principles and describe three application scenarios which will serve as short term, mid term and long term goals. After a description of the planned software architecture we finish with a review of related work and perspectives for future development.
2 Animation Principles in Cartoon Animation Early cartoon animation was mainly concerned with the exploration of a new media - a situation similar to realtime 3D animation and 3D user interfaces today. It was the development of believable characters by the animators of Walt Disney's studio in the 1930's that enabled cartoon animation to evolve from a technical novelty into an art form. Apart from technical advances in cartoon production it was mainly the "discovery" of a number of animation principles that enabled Disney's artists to create more life-like characters. Thomas and Johnston [28] give a detailed account on the development of disney animation, the "discovery" and use of these animation principles. Even today these principles form the basic skill set of cartoon animators. In [17] Lasseter explains these principles and how they can be applied to (non realtime, keyframe) 3D computer animation:
• Squash and Stretch defines the rigidity and mass of objects by distorting their shape during an action. Only totally rigid objects remain rigid in motion. Soft objects like most living things - are deformed by external forces during movement. Exaggerating this deformation increases the perception of object "softness". Squash and stretch can also be used to avoid "strobing" of fast moving objects. If an object moves very fast, so that the images no longer overlap in successive frames, the human visual system no longer perceives a moving object, but
successive separate images. This "strobing" effect can be avoided by stretching the object to retain an overlap (similar results can be achieved through motion-blur).
• Timing refers to the speed at which actions in an animation occur. Timing of object animation is essential to convey properties like mass and size and also to create a sense of realism by emulating physical effects like inertia and friction. Timing between actions is essential to focus the viewer on important actions and to avoid loss of attention.
• Anticipation is used to prepare the viewer for an action by catching his eyes and directing his attention to the area where the action will take place. Typically an action occurs in three parts: the preparation for the action - this is anticipation -,the action and the termination of the action (see follow-through). A typical device for anticipation are small preparatory moves.
• Staging means to present an idea or action, so that it is unmistakably clear. While some aspects of staging are specific to drawn cartoon animation the general idea transfers to 3D animation: Typical aspects of staging in 3D animation include the selection of an appropriate viewpoint to ensure visibility of important objects, sufficient contrast between objects, appropriate timing of actions - so that only one important action occurs at a time - and the direction of viewer attention towards important ideas and actions.
• Follow through and Overlapping Action is used to terminate one action and establish its relationship to the next action. Follow through actions are small actions that follow the main action to avoid sudden stops and add a sense of naturalness by emulating inertia, flow and continuity. Overlapping means to start a second action before the first action has completely finished to keep the interest of the viewer.
• Straight Ahead Action and Pose-To-Pose Action are the two contrasting approaches that are used to the create movement in cartoon animation. They correspond roughly to simulation oriented realtime animation and keyframe animation, respectively.
• Slow In and Out refer to the timing of object motion. Objects don't move with a uniform velocity but accelerate and decelerate. In cartoons slow in and out timing is not only used to achieve a sense of physical realism, but also to spend more time and detail at important actions and to "fast-forward" through less interesting sequences.
• Arcs are important when the movement paths of objects are defined. Objects seldomly move on lines but mostly on curved paths - arcs. The use of arcs helps to create a sense of naturalness and smoothness in actions.
• Exaggeration means to accentuating the essence of an idea through design and action. Exaggeration is commonly used as an attention getter and to illustrate the central idea or action in a scene. Most animation principles discussed here can be thought of as some form of exaggeration of reality.
• Secondary Actions are actions of an object that result from other actions, for example through collisions or friction between objects. Secondary actions can be used to heighten interest and add realism and complexity.
• Appeal is probably the most elusive goal to achieve in animation. It means to create an animation that the audience will want to see. Careful visual design, interesting content and convincing animation are all key ingredients that work towards "appeal". Summarising, all these animation principles are used in concert to communicate an idea clearly and without unambiguity by making the content of the animation more apparent than the animation techniques themselves. From a more technical perspective the increased "realism" caused by these animation principles can be traced to a number of qualities. These qualities include solidity, consistency and constancy of objects, perceptibility, naturalness and smoothness of motion and the direction of viewer attention (e.g., indication of impending actions through anticipation). To implement these qualities requires certain artistic means (e.g., staging, appeal) and the application of a sort of "cartoon physics" that exaggerates the characteristic phenenomena of the actions involved. Similar to cartoons, virtual worlds must not necessarily obey the physical laws of the real world. Therefore "cartoon physics", based on the animation principles described above, can be used as a metaphor for object behaviour in virtual worlds since animation based on "cartoon physics" is familiar to users from TV and computer games, has great expressive power and leads to more understandable animation.
3 Uses of Animation Principles in 3D User Interfaces Typical user interfaces rely on two forms of interaction: navigation and manipulation. In conventional graphical user interfaces (GUI's) manipulation, based on the windows, icons, mouse, pointer (WIMPS) metaphor, is the central form of interaction. An application is controlled and data is manipulated through standard interface components called widgets. Widgets (e.g. buttons, scrollbars) have a uniform appearance, feature simple functionality and tend to be similar in appearance and behaviour across applications and platforms. The layout of the widgets that constitute a user interface is relatively fixed and can thus be "designed for usability" by the interface designer. Navigation in con-
ventional GUI's is often limited to switching between applications, arranging the layout of windows and scrolling window content. Interfaces for 3D environments, on the other hand, are often based on a spatial "world" metaphor that relies on navigation as the primary form of interaction. Much of the intuitivity and potential of 3D interfaces stems from the use of a spatial "world" metaphor that enables the user to navigate through information spaces and use his everyday skills for interaction. However, navigation in 3D environments is not without problems: While occlusion of important information (for example through overlapping windows) can be problematic in conventional WIMP interfaces, the problem becomes significant in 3D environments where the interface is much more dynamic and the visibility and recognizability of interface components differs between different viewpoints, on which the application designer has little influence. It is therefore not sufficient to provide animation support in 3D user interfaces, but the animation techniques must also include the necessary functionality to ensure that the user is actually able to see and identify the animation. Manipulation in 3D interfaces is often simple direct manipulation of objects in the virtual world or the use of application specific 3D widgets. Due to the spatial nature of 3D "worlds", interfaces have to provide adequate answers to a number of questions concerning user orientation, interface presentation and interaction feedback:
• To improve user orientation the interface needs to answer to the questions "Where am I?", "Where have I come from?" and "Where can I go from here?".
• Interface presentation should primarily aid the user in recognizing interface objects and their functionality ("What is that object?", "What can I do with it and how?"). A more advanced function of interface presentation is to provide hints or suggestions for future actions ("What can/should I do now?").
• Interaction feedback informs the user about changes in the environment caused either by the user himself, the system, or other users ("What is happening and why?", "Where did that object come from?"). A 3D user interface that aims to answer these questions, requires a number of qualities, similar to those identified in cartoon animation: (1) Guidance of user attention toward important features and actions in the interface: Attention guidance can be achieved through anticipation to give cues about impending actions, attention getting actions and the use of exaggeration to increase the prominence of significant features.
(2) Objects with "believable" behaviour: This includes solidity, consistency, constancy of objects to avoid confusion (e.g. an explanation for object appearance/ disappearance: fly in, grow). Physical behaviour of objects can be approximated using "cartoon physics" to increase the appearance of "naturalness" and "realism". (3) Smoothness, perceptibility and visual appeal: Careful staging of objects and avoidance of technical artefacts, such as flicker and strobing serve to increase the impression of "real" objects with believable behaviour instead of artificial pictures. These desired qualities have to be translated into actions when presenting the behaviours (Goals) of objects in the 3D environment. Object behaviours that need to be supported in 3D interfaces include: Move, Attract attention, Appear / disappear, Change size / state / geometry /content, Identify, Demonstrate functionality. Object behaviours themselves are composed from a number of actions (e.g. move, transform (translate, rotate), wiggle, vibrate, change parameter, change geometry) that implement the animation of objects, based on the animation principles discussed above. The actions themselves are based on a set of building blocks provided by the underlying 3D library (geometry, timing functions, trajectory functions, transformation functions, rendering).
4 Agent Based Modeling and 3D GUI Before we present our system architecture we will briefly comment on agent based modelling in general and apply this AI-technique to the design of user interface elements. Since the work presented here is still in a preliminary stage we concentrate on some motivational scenarios to apply the presented ideas.
4.1 Agent Based Modeling For our agent architecture we adopt ideas from Shapiro’s Concurrent Prolog [25] and Rao’s AgentSpeak(L) [23] which is based on the well known BDI architecture. Agents are characterized as a set of concurrent objects acting locally according to specific incoming messages and perceived events. On a higher level these objects have complex internal states and act according to strategies. Properties of agents are autonomy, reactive / pro-active behaviour and structured messages for communication. A metaphor for explaining their functionality can assign certain „mental“ capabilities (e.g., beliefs, goals, desires, intentions) to these active objects. Many approaches of agent architectures exist in the AI-literature (see [31,32,33]) and a significant part of these is based on Rao’s and Georgeff’s BDI-approach [22] where agents have beliefs, desires or goals and build intentions to achieve their
goals by means of plans. For the purpose of our work the following naive view of agents is sufficient (see fig. 1 Events
Input
Messages
Messages
Goals
Beliefs
Strategies Output
Intentions control
Actions
Fig. 1: Naive Agent View An agent is composed of a knowledge base where the agent‘s beliefs are stored. Beliefs can be seen as facts or data describing the agent‘s model of the world. A plan store provides strategies for possible behaviour in form of rules. To achieve agent specific goals, certain strategies are activated based on available data (beliefs) and incoming messages (from other agents) as well as events generated by the user or the system. Executing these strategies results in certain actions which may generate new messages or change the agent’s environment. The control mechanism is an endless cycle where incoming messages and relevant events are observed. This results in a possible update of the knowledge base (if new information is available) and the selection of suitable strategies in order to fulfil the agent‘s goals. Agent behaviour is often modelled by operational means [23]. Based on definitions for plans, beliefs, goals, etc. an abstract interpreter is defined which describes the control cycle running in each agent. Strategies are described declaratively in terms of clauses similar to horn clauses known from logic programming. Additional test conditions ensure the applicability in a given context. The strategies used here are related to guarded horn clauses known from Concurrent Prolog [25]. A strategy or plan chosen to fulfil a goal is executed by pushing the elements to be processed on a stack. Processing these elements can generate new actions, goals, or tests that are then pushed on the stack. If more than one strategy is pursued at a time, the corresponding stacks are processed concurrently. The evaluation of a strategy can suspend another strategy through a explicit suspend-operation. In the following section we will show how characteristic aspects of agents like proactive behaviour, declarative behaviour specification and inter agent communication simplify the design of 3D interfaces.
4.2 Design structure of a 4D-Widgent Each 4D-Widgent has the structure illustrated in fig. 2. A more detailed description is given below: Input: The widgent’s input are events it is perceiving and messages from other widgents or communicating objects. Events are perceived by sensors which are currently limited to the detection of collisions / proximity
and visibility. Messages are structured, include message type, sender, receiver, and contents. They can be sent from other 4D widgents, special agents which serve as communication interface to the user (e.g., for handling new input devices) or other objects which can send messages. Sender and receiver can be individual elements or groups of elements Beliefs: Beliefs describe the widgent’s knowledge about the world (e.g., relevant parts of the interface). This includes information about the widgent’s visibility according to a user’s camera, its importance (a factor assigned by an external source), size, speed, or states of the widgent or other media information. Goals: In a mentalistic notion selected goals can be considered as adopted desires. They describe states the widgent wants to bring about. This includes the attraction of the user’s attention, the visualization of the widgent’s current states or the start of a dialogue with the user. In general, all goals are defined by the UI designer to provide a maximum usability of the interface. The agent selects goals dependent from its state, incoming messages and perceived events and decides what strategies will be used to achieve these goals. visibility, importance speed size, mass active, passive Collision Proximity Visibility
...
moveAtoB:: if visible(self) && important(self) :anticipate(move,A,B), slowIn(move), stretch(self), moveTo(self,B,speed), slowOut(move), followThrough(). if visible(self) && not(important(self)):moveTo(self,B,speed).
Events
„deep“ context condition may result in further strategies which have to be evaluated. This refers to flat and deep clauses in parallel logic programming languages like Concurrent Prolog and Flat Concurrent Prolog [25]. Intentions: Intentions are modelled as a data structure where partially instantiated strategies are stored and processed by the interpreter. Each strategy which is currently executed is stored in a run time stack where the individual tests, context conditions and actions are processed. If there is more than one strategy to be executed, the interpreter selects a stack in each run cycle and processes the next element. Output: The behaviour of an agent is the execution of actions and the sending of messages. Messages are structured and allow communication with agents and group of agents. Special messages exist which model specific communication acts (e.g., sendAndReceive, inform, ask, etc.). Each widgent has a number of primitive actions which modifies their visual representation or other abstract parts of the widgent’s environment. The animation principles described previously are encoded as parametrized actions which are performed if the corresponding strategy is executed. Other actions include state transitions, or modifying other media representations of the widgent like simple 3D-animations, sound, text, etc.
Goals Messages
Messages
Input
Beliefs
Strategies
Output
Fig. 3: Move of a Widgent (only slow-in slow-out) user interaction (sent from an input device agent) messages from other widgents
Intentions control Actions move, slowIn, slowOut stretch, followThrough simple 3D animations other actions
Fig. 2: Example of a 3D-Widgent Strategies: Widgents have rules which describe what actions have to be taken to achieve a goal. The selection of a specific strategy depends on the activated goal and a number of contextual conditions which have to be fulfilled before the strategy’s actions are performed. A contextual condition may be „flat“ i.e, its reduction is a simple test whether each condition matches the widgent’s belief base (e.g., visible() or important() ). A
As an example consider a graphical object that is moved from point A to point B. The corresponding widgent (described in fig. 2) receives a message (the input) that activates the goal to be achieved (moveAtoB(A, B)). Based on a number of beliefs, i. e., knowledge about the interface (e.g. visibility and importance), it then decides on a strategy to employ: An object that is not visible can be moved directly from A to B without any animation, for an unimportant object a very simple linear animation will be sufficient, while an important object should use all available cartoon animation techniques to ensure that the user recognizes the change. These strategies are then executed by a number of actions that effect the necessary changes
on the geometry (fig. 3).
5 Application Scenarios In this section we describe three applications scenarios in which we want to apply the presented ideas. As this project is part of a larger research effort [21] we differentiate into short term, mid term and long term goals. The application scenarios are taken from two other projects of our research team: a visual editor for the structured design of 3D widgets [8] and the design of a 3D visual programming language [9].
does not include multi-user interaction and the objects do not yet use proactive behaviour.
approximate Y-value
B A
y x
5.1 Snapping Objects and Approximating Coordinates As a first working example we started to implement the animation techniques in the snap-to-object and approximate function of a visual 3D editor [8]. The function snapto-object() enables precise alignment of arbitrary objects, by marking the snap target object and selecting an object to be repositioned. The selected object is then repositioned to align to the target object. The approximation function approximate() aligns the objects coordinate values in X-, Y-, Z-direction or the objects rotation angles. Since both objects may have arbitrary position and orientation before the alignment operations, it is often difficult for the user to follow which object was repositioned to which location, especially if many similar objects are present. The use of cartoon animation techniques (anticipation, slow-in, slowout and follow-through) clearly visualizes the change.
approximate Y-value
B A
Fig. 4: Example of Widgent Animation (Approximate) For the snap-to-object function exaggeration is also used. When the selected objects collide both move slightly back and forth and highlight for a short time. Fig. 4 and 5 visualize the described functions. In fig. 4 the first object B is selected as reference object and second object A is adjusted by approximating the Y-position. In fig. 5 the face x of object A is snapped to face y of object B. While the snap-to-object function demonstrates the usefulness of cartoon animation principles in 3D interfaces, it
Fig. 5: Example of a Widgent Animation (Snap)
5.2 Simple Multi-User Editor and Proactive Widgents The next scenario we want to realize is a small abstract shared workspace in which multi-user interaction is possible. Proactive behaviour of widgents can best be demonstrated in a multi-user 3D environments like 3D Chat Rooms or CAD tools for multi user access that make it hard for the user to distinguish between manipulable interface objects and static "decoration". The shared 3D Editor allows to create, copy, delete, transform (move, scale, rotate), group and ungroup objects by multiple users. In this scenario a number of different situations may occur where animation principles will enhance the “life-likeness“ of the interface: Passive and active users. In this class of scenarios (3D white-board) a user A is active (e.g., performs some operations) and the other users are passive observers. If an actions is applied to a widgent, it selects a suitable animation technique to attract the passive users attention. This depends on the widgent‘s state (importance, position, etc.) and other information like state of passive and active users, etc. In the following figures some interactions and possible corresponding animations are illustrated. The creation (or paste) of a widgent is animated by scaling the object to its original size. The deletion (or cut) of a widgent is animated vice versa. Grouping is animated by means of blob animation which exaggerate the grouping effect. This animation technique covers the separate shapes with an „organic like“ surface (like a thin skin of latex). After the animation the objects are no longer „covered“, but the grouping can interactively be visualized by highlighting if the cursor touches a group member.
group
delete
Fig. 7: Animated Grouping
create
While in this first prototype the notion of user intention will be very primitive (based on proximity and idle time), this can be extend to more complex user models and multi-user environment, where inter-agent communication will become important ????
Fig. 6: Animated Create / Delete of a Widgent Proactive widgents. Based on their beliefs about the users intention the widgents can autonomously adopt strategies to attract the attention of the user, in order to indicate interaction possibilities. As an example, consider a 3D world where a large number of elements exist and only some of them have interface functionality. If these UI elements detect that a user is confused (e.g., he continuously clicks on arbitrary elements in a region without further actions) it generates a goal to attract the users attention with a special „attract“-strategy which lets the element wiggle and change its color if the element is at the user‘s focus (cf. fig 8).
Fig. 8: Simple Proactive Widgent Multiple Active Users. If more than one user wants to interact with a widgent this needs special attention. If user B wants to move a widgent which is already selected by another user A the exclusive access is animated by a corresponding strategy. This animated feedback helps to avoid a misinterpretation of the denied access as a result of not selecting the 3D object correctly. When the grasped object is released, it springs back to its original shape. Other concurrent actions can be animated in a similar way. A:select B: move
Fig. 9: Animation of Concurrent Interactions
Shared 3D viewers and editors are currently developed in the CAD area where multiple users inspect and manipulate a virtual prototype [15]. An expressive visualisation of concurrent interactions of users might greatly enhance the effectiveness of such collaborative 3D design tools
5.3 Long Term Goal: Collaborative 3D Visual Programming Our long term goal is to employ these techniques in a complex 3D user interface: a structure editor for the 3D visual programming language SAM [9]. In contrast to their textual counterparts the language elements in a visual programming language are graphical. Visual Programming is sometimes referred to as programming with pictures and provides a new way of programming especially suited for non-expert users. During the last decade visual programming has grown into an important and well established research direction in HCI. A number of interesting visual languages have emerged in recent years (see the yearly IEEE Symposium on Visual Languages) and many of them are based on AI-concepts like concurrent constraints, logic programming, rule based or agent based concepts. SAM (Solid Agents in Motion) is one of the few visual languages which uses 3D shapes for describing programming elements. A SAM program is a set of synchronously running objects, called SAM agents, which communicate by exchanging messages. A SAM agent is defined by a set of simple production rules which execute a sequence of actions. An action can perform a local action (motion, sound, calculation, etc.), change the SAM agent's current state, send a message (including reply, broadcast) or kill / duplicate the SAM-agent. Rule
Agent
objects defining the condition at its outside and a sequence of actions inside. The execution of SAM agents is completely animated by a continuous smooth motion and scaling of objects. When a rule of an agent has been selected by pattern matching it is enlarged to the size of the agent and the currently executed action is marked by a scanner. In fig. 11 a small SAM program is illustrated which describes a producer which continuously produces a message, a transducer which transforms a message into another and a consumer which consumes all received messages. In its concrete representation a small 3D scene with farm, cow and cat is realised. The other example shows a process communication between three processes which uses the sojourner / pathfinder scenario as concrete view. Farm Cow Cat Hay
Milk
Farm Cow Hay
Cat Milk
Action Fig. 11: Abstract and Concrete SAM Programs
Message Fig. 10: SAM Language Elements SAM agents and messages can have an abstract or a concrete 3D representation, i.e., arbitrary forms which directly correspond to the objects’ meaning within the program. In their abstract representation, agents are initially represented as spheres with ports at the outside and a set of rules enclosed. Rules are basically visual copies of its agents with
Visual Programming in general is only effective if dedicated tools for program development exist. Direct manipulation editors, integrated debuggers and syntax checks are necessary elements for visual programming environments. For the SAM programming language this is especially useful because concrete and abstract representations of SAM programs have to be developed and a SAM program may consist of a large number of SAM agents. Our current prototype editor allows to interactively create abstract programs, assign concrete representations to them and save the result which can be executed by the SAM interpreter.
Three processes: Earth, Sojourner, Pathfinder
representations which are also modelled in this editor and finally tested within the programming environment. This highly complex task is an ideal application for our widgents and includes all previously described examples like 3D editor operations, multiple active users, the need for proactive widgents in complex scenes, etc.
6 Implementation
c
The software architecture of our system is illustrated in figure 14 and consists of tools which operate on a graphics system, a high level animation library and a multi-agent system.
1.) Earth sends request for photo 2.) Sojourner activates pathfinder 3.) Pathfinder moves to rock Yogi, takes shot and returns. 4.) Sojourner send photo to earth
Visual/textual design notation, scripting language, 3D Editor + visual representation component
Fig. 12: Visualizing Process Communication in SAM
+ abstract behavior component
Tools interactive 3D / 4D graphics agentbased system
=
Generation of 4D widgents
Fig. 14: Concept of the System Architecture
6.1 Agent System
Assign a concrete representation
Create a SAM agent
Fig. 13: SAM Editor The current prototype implementation of the SAM editor makes significant use of conventional 2D widgets (cf. fig. 13) for constructing SAM agents. In the next version we plan to integrate these 2D widgets in the 3D scene to reduce the mental load of the user to switch between several 2D and 3D windows. As a long term goal we plan to extend the SAM editor toward a multi-user editor where a SAM program is collaboratively developed, assigned concrete
The agent architecture described above is based on concepts developed in [11] and is further extended and implemented in a current diploma thesis using the Java programming language. The behaviour of agents is defined by three types of strategies. The reduction of a communicative strategy may require communication (message passing) with other elements and deep context conditions. Deliberative strategies do not use communication but may have deep conditions. Reactive strategies only have flat conditions. In case of multiple possible strategies reactive ones have highest priority and communicative strategies have lowest. This layered approach is similar to current approaches in multi agent systems like Müllers InterRaP [20]. Communication is realized using the FIPA/ACL specification.
6.2 Graphics system
7 Related Work
For the „visual parts“ we use a prototype of a custom animation layer put on top of the 3D graphics library OpenInventor [30]. AAL is a hierarchical library put on top of OpenInventor and was originally designed as a system for rapid prototyping of interactive real-time 3D animations [10]. It is organized along different levels of abstractions and uses a construction kit metaphor for designing interactive animated 3D applications. The first level above OpenInventor provides all necessary static objects for prototyping animations. We only use lights, cameras and primitive shapes and external objects as static animation elements but encapsulate them in completely object oriented way. Unlike OpenInventor or VRML97, properties like position, material, or transformation are attributes of the corresponding element and not a substructure in the complete scene graph. Abstract animation methods like keyframe animations (move, rotate, scale, colour) and algorithmic animations (pendulate, shuttle, inverse, rule-based) can also be defined at this level. At the next level animated objects are composed from elements of the lower levels. Objects are manipulated by predefined and user-defined animation methods that allow to generate complex animations based on animation lists and message passing. An animation list can start other lists at the same animated object or at other objects. Multiple lists are processed concurrently allowing to easily generate complex animated behaviour. The highest AAL-level provides concepts for agent-based functionality (e.g., states, transitions, handling of events and simple constraints).
The potential benefits of animation in user interfaces have been recognized for quite a while, see Baecker and Small [1]. In [13] Hudson and Tasko describe how animation techniques have been integrated in a 2D user interface toolkit. Chang and Ungar [4] give a detailed account of how cartoon animation techniques can (and have been) used in a 2D WIMP interface. Another important research stream considers AI based planning approaches for interface presentations and mainly concentrates on endowing interface agents with believable, life-like qualities [24,16,19]. For the purpose of presenting and explaining complex information spaces the expressive animation of gestures and other motions of such interface agents is essential. The use of knowledge based techniques for animation design which makes use of traditional film structures (sequences, scenes, shots) was described by Karp and Feiner [14]. Butz applied traditional film techniques to automatically generated 3D animations for the illustration of technical devices [3]. In contrast to this generation of 3D animations, Helbing and Preim used traditional film techniques like camera perspective, camera motion, shots and cuts to support the creation of intent based 3D animations for educational purpose [12]. The use of squash and stretch to enhance the expressiveness in direct manipulation editors was well recognized by B. Thomas [26]. He used a special warp algorithm to continuously animate transformations applied to 2D widgets. Recently, Thomas extended this idea to 3D objects and multi user editors [27]. Based on Conner’s pioneering work in 3D widgets [5] the extension of 3D interface elements to the fourth dimension using high level animation libraries is currently an increasing area of research [6].
6.3 Tools For a rapid development we are integrating this 3D-Animation functionality in the scripting language Tcl/Tk 8.0. Tcl/Tk is often used for prototyping 2D-WIMP-user interfaces and is well suited to be extend by new commands. Our system Ishells extends a Tcl/Tk shell by arbitrary 3D graphics functionality. The designer specifies the desired classes and methods of a 3D graphics library and Ishells automatically integrate these elements as additional Tcl/Tkcommands in a new shell. We successfully integrate a large proportion of Open Inventor in Tcl/Tk as well as extensions of OpenInventor developed by other researchers. Parts of the prototype implementation of our AAL library were integrated as well. We recognized the need for a visual/textual design notation for specifying a complex 3D user interface. The textual description based on Ishells will be extended with concepts taken from visual languages [7].
8 Conclusions and Perspectives Our first results support our belief that cartoon animation is well suited to convey changes in user interfaces and that the improved life-likeness achieved by cartoon animation techniques helps to avoid the breakdown of the virtual world metaphor in 3D user interfaces (even though the world might be less realistic!). Through the use of cartoon animation techniques 3D user interfaces can become more understandable, more "realistic", more engaging and thus more usable. By encapsulating the animation "rules" in agents, it becomes possible to reuse them in multiple applications and to simplify their application in user interfaces. Since the necessary knowledge of animation principles is encapsulated in the agents, 3D user interface components implemented as agents are simple and efficient for the in-
terface designer. While our current considerations indicate that „intelligent“ widgets can enhance 3D user interfaces, we do not propose to model 3D GUIs exclusively by means of proactive autonomous widgents. For simple structures the exhibition of agent-like behaviour is not necessary. Nevertheless, it is useful for many interface elements to provide the option of complex autonomous behaviour. If the interface adapts to its user, is applied in many different situations or the interface is used in very complex, dynamic applications like multi-user workspaces, advanced behaviour combined with expressive graphical means provides additional options for the design of future interfaces. Other areas in which a restricted environment has caused the development of techniques to communicate information clearly and "believable" include theatre (see Laurel [18] and stage magic (see Tognazzini [29]). Compared to the animation principles discussed in our work these techniques present a higher level of abstraction, therefore their use in interfaces would require more planning and intelligence. It would be interesting to see if and how far the agent based approach that we presented in this paper can be adapted to these higher level techniques.
[9]
[10] [11]
[12]
[13] [14]
9 Acknowledgments
[15]
The authors would like to thank the reviewers for their valuable comments and Sophie O‘Halloran for proof reading.
[16]
10 References [1]
[2] [3] [4] [5] [6] [7]
[8]
R. Baecker, I. Small. Animation at the Interface in: The Art of Human-Computer Interface Design, Laurel, B.(Ed.), Addison Wesley, 1990, pp. 251267. H. D. Burkhardt. Agent-Oriented Programming for Open Systems in [31]. A. Butz. Ein inkrementeller Ansatz zur Generierung informativer 3D-Animationen“, PhD Dissertation, Universitaet des Saarlandes 1997 (in german) B. Chang; D. Ungar. Animation: From Cartoons to the User Interface, Sun Technical Report SMU-TR95-33 D.B. Conner et al. Three-Dimensional Widgets. in: Proc. of Symposium on Interactive Graphics, Computer Graphics Vol. 26, March 1992. J. Döllner, K. Hinrichs. Interactive, Animated 3D Widgets. In: Proceedings of Computer Graphics International CGI‘98, Hannover, Germany, 1998. M. Duecker, C. Geiger, R. Hunstock, G. Lehrenfeld, W. Mueller. Visual-Textual Prototyping of 4D Scenes. Proc. of the IEEE Symposium on Visual Languages, Capri, Italy, Sep. 1997. C. Geiger, V. Paelke, R. Zelder. An integrated design tool for 3D widgets. IASTED International
[17]
[18] [19]
[20]
[21] [22]
[23]
Conference on Computer Graphics and Imaging, Halifax, Canada, June 1998 C. Geiger, W. Mueller, W. Rosenbach. SAM - An Animated 3D Programming Language. IEEE Symposium on Visual Languages, Hallifax, Canada, Sep. 1998. C. Geiger. Prototyping interactive animated object. Proc. Simulation & Visualisierung. Magdeburg, Germany, 1998, SCS C. Geiger. Rapid Prototyping of interactive threedimensional computer animations (in german). Dissertation Universität-GH Paderborn. in preperation. R. Helbing, B. Preim. Interaction Facilities and HighLevel Support for Animation Design“ In: Thomas Strothotte (ed): Abstraction in Interactive Computational Visualization: Exploring Complex Information Spaces. Springer-Verlag. S. Hudson, J. Tasko. Animation Support in a User Interface Toolkit: Flexible, Robust, and Reusable Abstractions in: Proc. UIST'93) P. Karp, S. Feiner. Automated presentation planning of animation using task decomposition with heuristic reasoning. Proc. Graphics Interface, Toronto, Canada, May 17-21, 1993 H. Kress. Shared 3D-Viewer. WWW page at http:// www.igd.fhg.de/www/documents/about/ igdhome_e.html, Fraunhofer Institut Grafische Datenverarbeitung, Darmstadt, Germany. D. Kurlander, D. T. Lings. Planning-based control of interface animation. Proceedings of the CHI 95, Denver, USA, 1995 J. Lasseter. Principles of Traditional Animation Applied to 3D Computer Animation in : Computer Graphics, Vol.21, No.4, July 1987, Proceedings of Siggraph '87, ACM Press. B. Laurel. Computers as Theatre, Addison Wesley, New York, 1991 J. Lester, J. Voerman, S. Towns, C. Callaway. Cosmo: A Life-like Animated Pedagogical Agent with Deictic Believability in Proc. of IJCAI97 Workshop on Animated Interface Agents: Making Them Intelligent, Nagoya, Japan, J. P. Müller. An Architecture for Dynamically Interacting Agents. Dissertation am Deutschen Forschungsinstitut für Künstliche Intelligenz (DFKI), Saarbrücken, 1996. V. Paelke. Structured design of interactive 3D animation for technical illustration purposes. PhD Thesis. in preperation. A.S. Rao, M. Georgeff. Modeling Agents Within a BDI-Architecture. In R. Fikes, E. Sandewall (Ed.): Proc. of the 2nd Int. Conf. on Principles of Knowledge Representation and Reasoning, 1991 A.S. Rao. AgentSpeak(L): BDI Agents Speak Out in a Logical Computable Language. in W. Van de
[24] [25] [26]
[27]
[28] [29]
[30] [31]
[32] [33]
Velde, J. P. Perram. Agents Breaking Away. 7th European Workshop on Modelling Autonomous Agents in a Multi-Agent World, MAAMAW’96, Eindhoven, The Netherlands, Springer 1996. T. Rist, E. André, J. Müller. Adding Animated Presentation Agents to the Interface, in: Inteligent User Interfaces, Orlando, FL, January 1997 E. Shapiro. The family of concurrent logic programming languages. ACM Computing Surveys, 21(3) 1989. B. H. Thomas, P. R. Calder. Animating direct manipulation interfaces. In Proc. of the ACM Symposium on User Interfaces Software and Technology, Pittsburgh, USA, 1995. B. H. Thomas, D. Stotts, L. Kumar. Warping Distributed System Configuration. In 4th Int. Conference on Configurable Distributed Systems. Maryland, USA, 1998 F. Thomas; O. Johnston. Disney Animation - The Illusion of Life, Bevelled Press, New York, 1981 B. Tognazzini. Principles, Techniques, and Ethics of Stage Magic and Their Application to Human Interface Design in: Proceedings INTERCHI '93, ACM Press, pp.355-362 J. Wernecke. The Inventor Mentor. Addison-Wesley, 1994. M. Wooldridge, N. R. Jennings Intelligent Agents I. ECAI-94 Workshop on Agent Theories, Architectures, and Languages (ATAL), Amsterdam, Niederlande, Springer, 1994. M. Wooldridge, J. P. Müller, M. Tambe. Intelligent Agents II. IJCAI’95 Workshop (ATAL), Montreal, Cananda, Springer, 1995. M. Wooldridge, J. P. Müller, N.R. Jennings. Intelligent Agents III. IJCAI’96 Workshop (ATAL), Budapest, Hungary, Springer.
