Using Planning with Action Preference in Story Generation Xiaobo Li
The Graduate School of Information, Production and Systems, Waseda University, Fukuoka, Japan
[email protected] Samiullah Paracha Research Center for Information, Production and Systems, Waseda University, Fukuoka, Japan
Jiao Wu
The Graduate School of Information, Production and Systems, Waseda University, Fukuoka, Japan
[email protected] Osamu Yoshie The Graduate School of Information, Production and Systems, Waseda University, Fukuoka, Japan
[email protected] [email protected] ABSTRACT Nowadays, plenty of researches focus on story generation which is widely used in computer games, education and training applications. It is highly desirable that the generated story should afford high user agency and at same time having capabilities to address user’s interventions. In this paper, we apply planning, which is derived from artificial intelligence, to achieve this objective. With the use of planning, several solutions are produced, which contains a sequence of user’s and system agents’ actions. In addition, we propose the concept of Action Preference, which takes into account user’s feedbacks, to evaluate all of the solutions after planning. Meanwhile a variant of hyperbolic tangent is utilized to calculate Action Preference. In order to evaluate its feasibility, an educational game was implemented on the basis of story generation. That result proves that planning with Action Preference is an effective approach in story generation.
Keywords Story generation, Gamification, Planning, Action preference
1. INTRODUCTION According to Riedl & Young (2006), there are two fundamental types of narratives used in story generation: linear narrative[5] and branching narrative[5]. Linear narrative is a traditional form of narrative in which a sequence of events is narrated from beginning to the end without allowing the user to alter the course of the unfolding narrative. This type of narrative is found in traditional storytelling media such as novels and movies. Computer games often employ linear plots. A plot is an outline of the most
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. MoMM2013 , 2-4 December, 2013, Vienna, Austria. Copyright 2013 ACM 978-1-4503-2106-8/13/12 ...$15.00.
significant occurrences in a narrative. Even though the user has a certain degree of control during level play, the only outcome is successful completion of some objective or failure, in which case the user must try again. Therefore, the events that occur during level play, although part of an emerging narrative, are not typically considered part of the plot. Every user experiences similar story during successive sessions. In branching narrative, there are many points in the story at where some action or decision made by the user alters the way in which a narrative has to unfold. Branching narratives are typically represented as directed graphs in which each node represents a linear, scripted scene followed by a decision point. Arcs between nodes represent decisions that can be made by the user. Even though a branching narrative may introduce variability into the user’s experience, the variability is typically built into the system at design time and it is, therefore, if a user were to make the same decisions during two consecutive sessions with the system, his or her experience would be the same. The alternative is Generative Approaches[5] to interactive narrative that construct narrative at runtime or preconstruct on a per-session basis highly interactive branching narrative structures. For every possible way the user can violate the story plan, an alternative story plan is generated. Both branching and dynamically generated narrative provide different dramatic experiences to the users however, the former restricts user’s agency, whereas the latter suffers from weak dramaturgic and learning consistencies. It is highly desirable that the story should afford high user agency and at same time having capabilities to address user’s interventions.
2.
PLANNING IN STORY GENERATION
As mentioned above, the scripted systems have strong narratives but do not scale with the freedom of choice; whereas emergent environments have strong freedom of choice, albeit provide a weak dramaturgic consistency. This paper incorporated hybrid narrative generation mechanism (see Figure 1) that combines the strength of both branching narrative and emergent narrative techniques borrowed from narrative intelligence[4]. Such reinforcements have offered us an en-
Figure 1: Hybrid narrative generation mechanism combines the strength of both branching narrative and emergent narrative techniques. Table 1: An example of actions, preconditions and postconditions. Action Precondition Postcondition move(leaner, char(learner), place(playground), library, place(library), char at(learner, playground) char at(learner, playground) library) make friend( char(victim), friend(victim, victim, char(learner), learner) learner) like(victim, learner) play with( friend(victim, played game(victim, victim, learner) learner) learner)
vironment that not only scales, but also provides a sense of story. To achieve this, planning (eg. a STRIPS-like representation planning algorithm[2]) has been applied that dynamically generates multiple stories at run time depending on user’s interventions. In planning, an action has preconditions and postconditions (effects). Whilst, the preconditions determine the applicability of an action, the postconditions are an indication of a changed state resulted by the application of that action. Given a goal state, and the current world state, the story generator (planner) selects an action to execute from the produced solutions. Table 1 presents an example of three actions, represented with variable arguments for which different instances can be substituted. The story generator applies planning to dynamically generate different narratives during the run-time and deals with user’s interventions that threaten the causality of a plan by re-planning. Now we focus on how to utilize planning in hybrid narrative. In emergent narrative, each node is regarded as a branching narrative. Branching narrative has its own precondition which differs from the action’s precondition. Branching narrative takes place when world state satisfy its precondition. In the first place, a world state is given. Afterward, planning figures out several solutions which are able to move world state from one branching narrative to another. In another word, each arc in emergent narrative corresponds to a solution produced by planning.
3. CHOOSING OPTIMAL SOLUTION AFTER PLANNING For planning, there are many kinds of algorithms, such as
Figure 2: A variant of hyperbolic tangent which is used to calculate action preference. total order planning[6], partial order planing[6], and planning graph[1]. Planning utilizes a set of actions to combine a sequence to lead the initial state to goal state. According to user’s different purpose, one or more possible solutions can be figured out after planning. Here’s the problem, which solution is the most appropriate for user. (i) The simplest algorithm to find a goal is to select the first solution whose conditions match the premises. It has an advantage in saving computational time, but it’s not reasonable. Even though computational time is an important factor, this algorithm may miss the best action that can exist in later solutions. (ii) Another situation is, if we use the number of actions to judge how well a solution is, it means we only consider the length of the solution, we will always get a same solution if we use a same input. Solutions came from planning give us a large range to choose. To evaluate a solution, we need more factors not only the two mentioned above. Which solution is the optimal one, it really does depend on user’s preference.
3.1
Action Preference
First of all, whether or not an action can be executed depends on user’s will. User’s choice is a potential factor to reflect his or her preference. In other word, user’s feedback affects action preference. It inspires us to find a type of sigmoid function[8], all types of which have S shape. Here we choose hyperbolic tangent. Figure 2 proposed a variant of hyperbolic tangent which is used to deal with calculating action preference in this paper. tanh(x) + 1 2 In this formula, ’P’ means action preference, its value is between decimal 0.0 and 1.0. And at first time, ’x’ is assigned to 0. As a result, the initial action preference is 0.5. If we accept an action during the generation of story, the action preference will be changed to f(x+1). Whereas, if we reject the action, the preference will be changed to f(x-1). Hyperbolic tangent is suitable for calculating preference of each action. According to Figure 2, we can see that action preference ’P’ changes in a small range when absolute value of ’x’ gets larger and larger. It indicates that a more attractive action will get a high value corresponding to its preference. When absolute value of ’x’ is small, the value of P = f (x) =
Table 2: Action preferences are calculated by using hyperbolic tangent which is described before. Yes Yes No Yes No No A1 0.5 0.88 0.98 A2 0.5 0.88 0.5 A3 0.5 0.12 0.02
Figure 3: A set of solutions after planning, solution is a sequence of actions. action preference ’P’ will be sensitive. It coincides with the real situation at beginning. As time goes on, those value of preference will be more and more precise. For example, there are three kinds of actions: take gift (A1), move to (A2), and make friend (A3). At first time, each action preference is 0.5. If a solution is a sequence of [A1, A2, A3, A1, A2, A3]. Now we focus on user’s feedbacks, we assume that, a sequence of user’s feedbacks corresponding to this solution is [Yes, Yes, No, Yes, No, No]. Then we get Table 2. Here, (tanh(0)+1)/2 = 0.5, (tanh(1)+1)/2 = 0.88, (tanh(2)+1)/2 = 0.98, (tanh(-1)+1)/2 = 0.12, and (tanh(-2)+1)/2 = 0.02.
3.2 Evaluation Function Once we get all of the possible solutions after planning. We make a recommendation to user. Here we use an evaluation function to compute the score for each solution. A solution with the highest score will be recommended to user. The evaluation function is defined as follows: E1(s) + E2(s) + E3(s) 3 Here, ’s’ means solution, evaluation score E(s) is the mean value of E1(s), E2(s), and E3(s). For each solution, we consider in three aspects: (i) length of the solution, (ii) mean preference of all the actions in current solution, (iii) preference of first action in current solution. A given example in Figure 3 shows how to calculate evaluation score. In Figure 3, ’S’ corresponds to solution, ’L’ corresponds to length of solution, ’A’ corresponds to action, ’P’ corresponds to action preference, and ’M’ corresponds to mean preference of actions in solution. Then E(s) =
E1(Sk ) = 1 − !
Lk L21 + L22 + ... + L2k + ... + L2n
Mk =
"L k
i=1
Lk
Pki
Figure 4: Screenshots of SMuyou which is an educational game with story generation. E2(Sk ) = !
E3(Sk ) = !
Mk M12 + M22 + ... + Mk2 + ... + Mn2
Pk1 2 2 2 2 P11 + P21 + ... + Pk1 + ... + Pn1
E1, E2, and E3 use regularization method to normalize parameters they care about respectively. Using formulas described above, we can easily get evaluation score of each solution. And a solution with the highest score will be adopted and returned to user finally.
4.
SIMULATION
As gamification[3] is one of the most significant factors in education. We implement an educational game (SMuyou) with story generation learned from existing theories[7] to tackle bullying at schools. Using Prolog to achieve process of planning and C++ to complete the graphical user interface. Several screenshots are showed in Figure 4. SMuyou places a player in a virtual school to interact with other characters in bullying scenarios. It is consisted of short dramatic plots that describe ethical dilemmas in bullying context, allowing the player to experience bullying dilemmas at a safe emotional distance; make values-based decision; and change relationships in positive ways. Planning is used to find solutions for leading current world state to next dilemma’s precondition. In SMuyou, initially, using planning to achieve all of the original solutions. Then, adopting evaluation function to rank those solutions. Finally, the solution with the highest score will be recommended to user. As a result, the optimal solution trades-off between length of solution and user’s preference. We review some virtual learning environments of particular relevance to SMuyou. In each case, similarities to and differences from SMuyou have been identified, covering scripted narrative, emergent narrative, and continuous story. According to Table 3, compared with presented systems, both
Table 3: Comparison between current existing story generation systems and SMuyou. Educational Games Branching Emergent Continuous Narrative Narrative Story Carmen’s Bright Yes No No IDEAS BAT ILE Yes(Linear) No No ELECT BiLAT Yes No Yes TLTS No Yes No FearNot! No Yes Yes Crystal Island No Yes No Teatrix No Yes No ORIENT No Yes Yes SAM Yes No No STEVE Yes No No SMuyou Yes Yes Yes
scripted narrative and emergent narrative are supported in SMuyou, meanwhile, the two narratives’ combination make the effect of continuous story. As a result, the SMuyou has a great advantages in story generation.
5. CONCLUSIONS AND FUTURE WORK In this paper, using Planning in Story Generation, it affords high user agency, meanwhile, it has capabilities to address user’s interventions. In addition, we propose an approach, Action Preference, which extends traditional planning methods. And a variant of hyperbolic tangent is utilized to achieve Action Preference. We construct an Evaluation Function to rank all of the solutions after planning. Our simulation shows that it is successful to apply Action Preference in story generation. In the longer term, to evaluate the accuracy of Action Preference and Evaluation Function, we need a statistical method to make the experimental result more quantitative. One feasible implementation requires a wide range of statistics which direct at different user’s feedbacks. Once we obtain those standard data, a numerical result will be done.
ACKNOWLEDGMENTS This research was supported by Waseda University Grant for Special Research Projects 2013B-195.
6. REFERENCES [1] A. Blum and M. Furst. Fast planning through planning graph analysis. Artificial Intelligence, 90(1-2):281–300, 1997. [2] S. LaValle. Planning Algorithms. Cambridge University Press, Cambridge, UK, 2006. [3] J. Lee and J. Hammer. Gamification in education: What, how, why bother? Academic Exchange Quarterly, 15(2):146–151, 2011. [4] M. Mateas and P. Sengers. Narrative intelligence. In Proceedings of the Narrative Intelligence, AAAI Fall Symposium, pages Technical Report FS–99–01. AAAI Press, 1999. [5] M. Riedl and R. Young. From linear story generation to branching story graphs. IEEE Computer Graphics and Applications, 26(3):21–31, 2006.
[6] S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, 1995. [7] K. Schrier and D. Gibson. Designing Games for Ethics: Models, Techniques and Frameworks. IGI Global, USA, 2011. [8] T. Segaran. Programming Collective Intelligence: Building Smart Web 2.0 Applications. O’Reilly Media, Beijing, 2007.
