Real-time Planning for Interactive Storytelling

Real-time Planning for Interactive Storytelling R.J.S. Paul, M.D.J. McNeill, D.K. Charles, D.M.G. McSherry, P.J. Morrow School of Computing and Information Engineering, University of Ulster, Coleraine, Northern Ireland [email protected] {pj.morrow, mdj.mcneill, dk.charles, dmg.mcsherry}@ulster.ac.uk

ABSTRACT Stories are often implemented in contemporary computer games in a linear fashion. In general, a player follows a pre-scripted storyline from start to finish, which can lead to the player feeling that their actions and choices in the game have limited effect. This is particularly true for massively multiplayer online games where typically there are thousands of quests. A different approach, which introduces non-linearity, allows the narrative to be structured in such a way as to allow an element of choice, whereby the user can affect the direction of the story at key moments. However, creating such stories requires significant effort at design time to cover all the possible choices that may be made by the user at run time. One possible way to afford the player a significant amount of influence over the story direction while also keeping design costs relatively low is by allowing elements of the story to emerge as the game is played (interactive storytelling). A key challenge is maintenance of a coherent story within the context of an emergent narrative based on user actions in real time. This paper begins with a review of research in the field of interactive storytelling. An architecture to support emergent stories in the context of a multiplayer game is presented along with initial ideas for implementation of the real-time planner. Categories and Subject Descriptors (according to ACM CCS): D.0 [Software]: General

1. Introduction In contemporary games, stories are typically implemented using one or more standardised methods such as linear, branched or layered [TBC08]. Characteristically, first person and third person action-orientated genres often use movie-like cut-scenes to progress their story line in a linear fashion. Branched storytelling is a variation on this whereby the player is given choices that affect the direction of the narrative at certain points in the gameplay, often allowing for multiple endings dependent on the players choices in the game. With newer role-playing games (RPGs) we observe the layered-story technique in effect. This method comprises a central linear or branching storyline alongside sets of side stories within a virtual world. Layered stories provide players with a sense of empowerment over the direction of the story. However, one drawback is that the side stories often have little or no relevance to the main story and there is still a need for a large amount of authoring. Interactive storytelling (IS) an expanding research field [CCF*08] [TBC08] [DCA07] [BK08] that attempts to strike a balance between a designer and a player directed story by allowing the player to influence the ongoing narrative in some way. IS has many potential applications including military training [RS06], interactive drama [CCM02] and computer gaming [TTB06]. Debate on the role of stories within games continues and one point of

view is that the concepts of narrative and interactivity are diametrically opposed to one another [RS06] [C04]. This is because there is a significant difficulty to overcome in allowing a user to have freedom of interaction within a story while maintaining the coherence and avoiding inconsistencies in the narrative [CCF*08]. In dealing with this issue there have been two main approaches: techniques that favour autonomy and freedom of character interaction with the plot are labelled as character based approaches [BK08] [CCM02], and other approaches which focus more on the structure of the overall plot. In the later plot based approach [TBC08] [CPF*05] [F04] [MAG*04] there is a focus on maintaining coherence while attempting to allow a lesser degree of user interaction with the plot. Plot-based approaches favour the importance of narrative structure over player interactivity. They generally attempt to guide a player along some kind of partly predefined narrative structure and allow relatively small levels of direct interference from the user. This approach may afford a user less freedom and/or restrict the number of possible plot situations; however, it is understandable that with this approach focusing more on the plot, a more structured narrative will result. Character-based approaches often favour freedom of interaction and story possibilities over plot structure. They attempt the dynamic creation of stories from interactions between users, the virtual world and computer controlled characters.

R. J. S. Paul, M. D. J. McNeill, D. K. Charles, D. M. G. McSherry and P. J. Morrow

Interactive storytelling systems, then, allow the current story to be shaped by the user's actions. To enable this, a model of the current state of the system is required. This state model must be dynamic, reflecting user actions. It can be used to inform a reasoning or planning system which can then attempt to shape the current story. The user must be informed - either directly or indirectly - of changes to the story, thereby completing the feedback loop. We are primarily interested in interactive storytelling for real-time interactive games. Within this context, key challenges include:

•

Updating the knowledge of the planner or reasoning system to reflect user actions and other changes in the game state; for example, if a user moves location, or a character acquires some new knowledge.

•

Creating stories, or adapting from existing stories, by using a planner or reasoning system which processes the knowledge of the current state of the game world.

•

Depending on the new stories created, the roles of characters in the game world may be changed; for example, a neutral character may become an enemy.

•

Maintaining consistency in the story; for example, if a user moves or destroys an object that is required for a future step of the story, then that step is no longer valid and should not form part of the overall story. In many multiplayer online role-playing games objects are re-spawned in order to maintain the validity of stories. Such an approach, however, reflects a lack of consequences of the actions of the user that moved the object.

•

Maintaining interactivity; within the context of realtime games any reasoning or planning must be designed in such a way that the interactive nature of the game is not compromised.

•

Minimising authoring requirements; as much as possible, new stories should be created dynamically, thereby reducing the necessity to author stories which may never be played out, such as those in branching story systems.

The remainder of this paper is organised as follows; Section 2 summarises previous research in this field. Section 3 proposes an architecture for a multiplayer game system which supports dynamic storytelling. A prototype implementation of the real-time planner and ideas for an enhanced implementation of the planner are presented in Section 4. Conclusions are presented in Section 5. 2. Previous Work There is a substantial body of research within the interactive storytelling field. In this paper we focus on research that is most relevant to our proposed architecture. I-Storytelling [CCM02] is a well known character based interactive storytelling system which attempts to bring about narrative through interactions between autonomous agents with individual task networks which are tailored to fit their specific roles in the story [CCM02]. The user interacts with the system through hiding objects that are cen-

tral to the story and by influencing the automated characters through a speech recognition interface. These interactions, along with random character starting positions, provide a means of generating different storylines on each execution of the system. A real-time hierarchical task network (HTN) planning system [NCL*99] [NAO*] is used to automate character movement and actions. Each character has a number of context-specific subtasks and as such, careful authoring of each character's task network is needed in order to ensure that an interesting narrative occurs. Character roles are designed into the planners a priori for the story and therefore the actions that a character can take are scripted for a particular role. Characters’ roles are preselected at design time for a particular story. Other systems implement an interpretation of Propp’s ‘Morphology of the Folk Tale’, [CPF*05] [L05]. Propp describes a Russian Formalist approach to decomposing fairy tale stories into narratemes (plot element) and functions of characters (roles). The systematic structure of Propp’s analysis has led some researchers to incorporate his models into their systems. Fairclough and Cunningham’s [FC03] OPIATE is an example of this. This system includes a story-managing component (the story director), and uses case-based reasoning to support the creation of plots. The system comprises of a 3D adventure game environment, where the user can interact with objects and other characters freely and choose actions to perform through a drop down menu structure. In order to allow the progression of the story to be influenced by the player’s movements and actions within the world, a story director records character attitudes to the player. These attitudes change with each interaction, creating information which is used to cast characters into specific roles (e.g. villain or friend). Each character role contains a set of constraints relating to their state such as social rating, possessed objects, loyalties and current location. Whichever character matches the constraints of a particular role best is assigned that role for the particular story instance. Tomaszewski and Binsted [TB07] describe Eudaemon, an interactive storytelling system based on a Proppian approach and intended to produce a directed interactive drama in which the player can manipulate objects and interact with characters. It includes a story manager, similar to the one seen in the OPIATE system, whose purpose it is to build a story dynamically piece by piece from preauthored story scenes based on narratemes. Tomaszewski and Binsted noted some apparently intrinsic limitations to a Proppian approach to interactive drama during development of their system. They note that most of the functions that Propp describes are inflicted upon the hero rather than enacted by him. In the few cases where the player is given a choice, it is generally only a binary one. In one instance, the player can respond positively or negatively to a character with a donor role. However, these choices may be illusory because the player must choose the ‘correct’ option to allow the story to progress. For example, they must respond positively to the ‘donor’ to receive a magical agent, crucial to a later part of the story. These observations suggest that using Propp’s story narratemes, as they stand, in storytelling system management may not be suitable for interactive simulations.

Real-time Planning for Interactive Storytelling

None of the above systems fully realise the goal of a fully dynamic story world, where user actions lead to the generation of stories in an unscripted way. While it is expected that a central story exists, it could be argued that dynamic changes to the story world (as a result of user choices) should lead to the emergence of hitherto unscripted changes to the story. Character roles could change (e.g. a friend could become an enemy due to negative actions carried out against them), small sub-plots (quests) could change or emerge (e.g., a quest to find an object that another character has moved). The rest of this paper presents our thoughts on how such a system could be designed, and some early results from implementing a realtime planning system. Section 3 presents our architecture for an interactive storytelling system which attempts to address this challenge and others identified in the introduction. Our system uses AI planning methods rather than case-based reasoning for story creation and revision and character role selection. 3. An Architecture for Interactive Storytelling Our proposed architecture (Figure 1) has two main components: a Game Engine and a Drama Manager. A detailed discussion of each component follows

In general, characters use their planners to decide how to perform tasks (or achieve goals) assigned to them by the Drama Manager or Game Engine. Each character also converts its local knowledge, acquired from sensors, into a partial game state representation for use by its planner. The planner uses this information to guide the decomposition of an assigned task into primitive tasks whose preconditions are known to be satisfied. For example one of the primitive tasks (or plan steps) identified by the planner might be “pick up the sword”. The character passes these primitive tasks as commands to the Game Engine, which then changes the character’s state and behaviour accordingly. The tasks that characters receive from the Game Engine represent the character’s internal desires and are based on changes to the character's state; for example, if the hunger attribute of a character rises above some threshold, this initiates a ‘getFood()’ task, which is sent to the character's planner. The planner then tries to generate a plan for the task by decomposing it into primitive tasks to be sent back to the Game Engine. An example task decomposition can be seen in Figure 2. The selected decomposition depends on which pre-conditions are satisfied in the available methods. For example, for the method buyApple() to be selected, it may be necessary for a condition such as 'sells(Shop1, Apple)' to be satisfied. Similarly if there existed a fact 'has(John, Apple)" then this may have enabled the creation of a plan involving a 'steal' subtask.

getFood()

makeMoney()

buyApple()

goToWork() doWork() goTo(Shop1) purchase(Apple)

Figure 2: Example HTN Decomposition

Figure 1: Proposed Interactive Storytelling Architecture for Real-time Interactive Games

A character can also call a predicate in its planner to inform the planner that a task cannot be completed using the current plan, in which case the planner attempts to create a new plan for the uncompleted task.

3.1 Game Engine

3.2 Drama Manager

The Game Engine handles the display and updating of game world objects and also interacts with characters and the Drama Manager. This includes managing the primitive tasks performed by characters and changing character states and behaviours accordingly, as well as managing game world state information obtained from character sensors.

Initially the drama manager has a hierarchical network of story elements, which can be pieced together in different ways to form a story. Each story element has a set of preconditions which determine whether or not the element is valid for a particular game context and a cost function which can be used by the drama manager to choose an optimal story. The current state of the game world is passed to the drama manager periodically from the Game Engine. The drama manager then attempts to create a story that fits the current world state using its HTN planner and network of story elements (i.e., it checks whether or not

Each character in the game has its own HTN planner in order to facilitate the creation of a dynamic game world where characters can interact with each other in a nondeterministic way.


the preconditions of each story element are satisfied). If a story can be built, the drama manager assigns tasks related to story elements to individual character planners. Using an HTN planner in this way, as part of the drama management subsystem, could potentially enable the creation of many story variants based on the state of the world at a particular time. In the event that no story is valid in the current world state, the Drama Manager assigns special tasks to characters with the aim of reaching a new state in which a valid story can be created. A similar recovery method can be seen in work by Riedl [R06]. Section 4 discusses how the implementation of real-time planners within a game engine can be realised.

for the navy and many university projects [NAI*05]. However, JSHOP2 uses a grammar-parsing component called ANTLR which parses input, written in JSHOP2 syntax, into Java code. Producing a plan in JSHOP2 is therefore achieved through multiple steps. These steps include compiling the domain description and the problem descriptions from the JSHOP2 syntax into Java code using ANTLR and then compiling and running the Java file produced. When this file is executed it outputs the plan(s) for the planning problem. Due to the way that the input language must be parsed to a Java file before execution, the potential for use with real-time planning systems, without a good deal of modification to the code, may be limited. Hierarchical Task Network (HTN) planning was not

Figure 3: JPlan Implementation

4. Initial Implementation A major challenge in the implementation was to create a real-time planning system and integrate it with a game engine. Goal Orientated Action Planning (GOAP) [Ork] was considered as a possible way to achieve this. GOAP is a procedural STRIPS-like planning architecture designed to be used with real time games. However, it has to date mostly been used with first person shooter games with few implementations or discussions of its use in RPG type environments. additionally, in most implementations, a fixed array of symbols (state facts) are available, which limits the ability to create or destroy game world objects without modification to the architecture. JSHOP2 [NAO*] was considered for the planner, due to its popularity in research and the inclusion of many additional features like allowing for multiple plan outputs for a single problem and the ability to handle numerical values. Shop, Shop2 and JShop2 have been used for projects such as unmanned airborne vehicle control, evacuation planning

deemed a necessity in the initial implementation that we describe in this section JPlan [MANZ] is a Java implementation of Graphplan [BF97], a basic but efficient planning algorithm, which was chosen for use as the planning component of the system. To improve performance, in keeping with a real-time game engine scenario, the JPlan planner was modified to receive input and give output via procedural calls instead of using files. Figure 3 shows the implementation of a real-time game engine which utilises JPlan to automate characters. It incorporates the Game Engine and character planning components of the architectural model described earlier. The Golden T Game Engine [PAU] was chosen as a game development framework for the system as it too is implemented in Java and provides a basic but efficient game engine with inbuilt methods to deal with drawing and updating sprites.

Real-time Planning for Interactive Storytelling

The green rings show the radius of each character's sensory input, which updates the character's knowledge of the game world. Random starting positions are chosen for the characters as well as several apple and sword items. Over time the characters’ hunger levels increase as does their dislike for other characters. When either of these variables reaches a pre-defined threshold, the appropriate goal is fed into the character's planner. Characters can choose from several operators when planning: eat, steal, drop, pickup and attack. Output from their planning operations is displayed in the console on the right. 4.1 Discussion The objective of the initial prototype was to create a realtime planning system to help identify any issues surrounding the implementation of the proposed approach. As such, the prototype proved to be very useful in enabling the following issues to be identified:

•

•

•

The planning algorithm used in the prototype is efficient and optimal [BM07] but its features are limited for the purposes of our research. For example, JPlan only returns the first plan it finds and so multiple plans cannot be generated and compared for suitability. The lack of expressivity in the operator input language restricted scalability. As more locations were added to the system, the number of operators needed to enable travel to different locations increased exponentially. To overcome this character movement was assigned to the Game Engine and not included as plan steps.

which should make the game implementation a very efficient process. 5. Conclusions Interactive games are becoming increasingly complex and game players are becoming increasingly critical of games where the story appears to be too scripted or where appropriate, consistent feedback of user actions and choices is not supported. Consequently, there is considerable interest in the use of such techniques as interactive storytelling, where the game responds intelligently to user actions, reflected in a more dynamic, emergent game narrative. One way in which this can be realised is by a system which can dynamically adapt elements of the story to run-time events in the game world. This may be best achieved by a system capable of reasoning over a hierarchical network of story elements to produce a series of tasks that are capable of interpretation by a game engine. A major challenge is to maintain both consistency in the game world and interactivity. In this paper, we have proposed an architecture to support these requirements and taken a first step towards realising the planning sub-system. References [TBC08]

THUE D., BULITKO V., CAMANHO M.: Player Modeling for Interactive Storytelling: A Practical Approach, S. Rabin, ed., MA: Charles River Media, 2008, pp. 633645.

[CCF*08]

CAMANHO M.M., CIARLINI A.E.M., FURTADO A.L., POZZER C.T., FEIJÓ B.: Conciliating coherence and high responsiveness in interactive storytelling, DIMEA '08: Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts, New York, NY, USA: ACM, 2008, pp. 427-434.

[TB07]

TOMASZEWSKI Z., BINSTED K.: The Limitations of a Propp-based Approach to Interactive Drama, CA: AAAI Press, 2007, Menlo Park: 2007.

[DCA07]

DELMAS G., CHAMPAGNAT R., AUGERAUD M.: Plot monitoring for interactive narrative games, ACE '07: Proceedings of the international conference on Advances in computer entertainment technology, New York, NY, USA: ACM, 2007, pp. 17-20.

[BK08]

BARBER, KUDENKO: Generation of dilemma-based interactive narratives with a changeable story goal, INTETAIN '08: Proceedings of the 2nd international conference on INtelligent TEchnologies for interactive enterTAINment, ICST, Brussels, Belgium, Belgium: ICST (Institute for Computer Sciences, SocialInformatics and Telecommunications Engineering), 2007, pp. 1–10.

Adding and removing knowledge from characters could not be achieved due to the way that the knowledge state detector operates. Previous character planner facts are overwritten each time the conversion from Game Engine data to planner facts takes place. This meant that knowledge could not be transferred between agents. A redesign of this part of the architecture would be necessary to allow this transfer.

4.2 Towards a Complete Implementation Given the limitations of the graph planning algorithm identified in the initial implementation, it was decided that the most flexible solution would be to develop our own Hierarchical Task Network (HTN) planner in LPA WinProlog. HTN planners [NCL*99] [NAO*] create a plan through a decomposition process whereby each task is broken down into one or more subtasks until all that is left is a list of primitive tasks or ‘plan steps’. These plan steps can then be carried out by an agent until either the tasks are complete or re-planning is required due to a change in the world state. This allows us to build in methods to retrieve and sort multiple plans and handle specific flags sent from the Game Engine. The planner can be integrated with the Game Engine using LPA’s intelligence server architecture, which allows communication via DLL calls with other programming languages. This strategy also enables the use of Microsoft’s XNA framework as the Game Engine,


[RS06]

[CCM02]

[TTB06]

RIEDL M.O., STERN A.: Believable Agents and Intelligent Story Adaptation for Interactive Storytelling, Proceedings of the 3rd International Conference on Technologies for Interactive Digital Storytelling and Entertainment, Darmstadt, Germany: 2006. CAVAZZA M., CHARLES F., MEAD S.J.: Character-based interactive storytelling, IEEE Intelligent Systems, vol. 17, 2002, pp. 17-24. TYCHSEN A., TOSCA S., BROLUND T.: Personalizing the player experience in MMORPGs, Technologies for Interactive Digital Storytelling and Entertainment, Darmstadt, Germany: 2006, pp. 277-288.

[C04]

CRAWFORD C.: Chris Crawford on Interactive Storytelling, New Riders Games, 2004.

[CPF*05]

CIARLINI, POZZER, FURTADO, FEIJÓ: A logic-based tool for interactive generation and dramatization of stories, ACE '05: Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology, New York, NY, USA: ACM, 2005, pp. 133– 140.

[F04]

FAIRCLOUGH C.R., Story Games and the OPIATE System, Ph.D. Thesis., University of Dublin, Trinity College, 2004.

[MLA*04]

MAGERKO B., LAIRD J.E., ASSANIE M., KERFOOT A., STOKES D.: AI Characters and Directors for Interactive Computer Games, Proceedings of the 2004 Innovative Applications of Artificial Intelligence Conference, AAAI Press, 2004, pp. 877– 883.

[L05]

LINDLEY C.A.: Story and Narrative Structures in Computer Games, Developing Interactive Narrative Content, 2005.

[FC03]

FAIRCLOUGH C.R., CUNNINGHAM P.: A Multiplayer Case Based Story Engine, TCD-CS-2003-43, Computer Science Technical Report, Trinity College Dublin, Nov. 2003.

[YRB*04]

YOUNG R.M., RIEDL M.O., BRANLY M., MARTIN R.J., SARETTO C.J.: An architecture for integrating plan-based behaviour generation with interactive game environments, Journal of Game Development, 2004, p. 1.

[R05]

RIEDL M.O.: Towards Integrating AI Story Controllers and Game Engines: Reconciling World State Representations, Proceedings of the 2005 IJCAI Workshop on Reasoning, Representation, and Learning in Computer Games, Edinburgh: 2005.

[R06]

RIEDL M.O.: Automated Story Generation: Balancing Plot and Character, USC Institute for Creative Technologies, Marina del Rey: 2006.

[Ork]

J. ORKIN: Agent Architecture Considerations for Real-Time Planning in Games, Artificial Intelligence & Interactive Digital Entertainment (AIIDE}, AAAI Press, 2005 NAU D., AU T., OKHTAY I., UGUR K., MURDOCK B., WU D., FUSUN Y.: JSHOP2, http://www.cs.umd.edu/projects/shop/inde x.html NAU D., AU T., ILGHAMI O., KUTER U., WU D., YAMAN F., MUÑOZ-AVILA H., J.W. MURDOCK: Applications of SHOP and SHOP2, Intelligent Systems, IEEE, vol. 20, Apr. 2005, pp. 34-41.

[NAO*]

[NAI*05]

[BF97]

Blum A., Furst M.: Fast Planning Through Planning Graph Analysis, Artificial Intelligence, 90:281-300 (1997). http://www.cs.cmu.edu/~avrim/graphplan

[PAU]

PAULUS T.: Golden T Game Engine, http://www.goldenstudios.or.id/: Golden T Studios.

[BM07]

BARROS L.M., MUSSE S.R.: Planning algorithms for interactive storytelling, Computers in Entertainment, vol. 5, Jan. 2007.

[MANZ]

EL-MANZALAWY Y.: http://sourceforge.net/projects/jplan/

[NCL*99]

NAU D., CAO Y., LOTEM A., MUÑOZAVILA H.: SHOP: Simple Hierarchical Ordered Planner, In IJCAI-99, pp. 968973, 1999.