Web-Based Learning with Non-linear Multimedia Stories Marc Spaniol1 , Ralf Klamma1 , Nalin Sharda2 , and Matthias Jarke1,3 1
RWTH Aachen University, Informatik 5, Ahornstr. 55, D-52056 Aachen, Germany {mspaniol, klamma, jarke}@cs.rwth-aachen.de 2 School of Computer Science and Mathematics, Victoria University Melbourne, Australia
[email protected] 3 Fraunhofer FIT, Schloss Birlinghoven, D-53754 Sankt Augustin, Germany
[email protected]
Abstract. Stories and story-telling are a cultural achievement of significant relevance, even in modern times. Since ancient times stories have served to entertain and teach mankind to “transmit” knowledge from generation to generation. Story-telling ranges from simple aural narrations to traditional inter-generational discourse and, in modern times, workflow-oriented organizational learning. Web-based systems are by nature well-suited to support learning from digital stories in communities of practice. Despite the potential of story-telling to foster knowledge sharing in communities its full power to stimulate community-based learning processes in yet only marginally exploited. Although there are many story-telling approaches, most of them are not suitable for non-linear story creation and consumption. In addition, most of these are not based on a well defined methodology that underpins the story development process. In this paper we present the implementation of a non-linear multimedia story-telling environment based on the movement oriented design (MOD) paradigm. Finally, using a media-theoretic approach, we utilize structural aspects of story telling and identify patterns related to successful non-linear multimedia stories.
1
Introduction
Organizational knowledge management and professional learning are closely connected. The knowledge creation theory by Nonaka and Takeuchi [13] has become widely acknowledged in management theory and practice. Also, in the fields of computer supported cooperative learning (CSCL) and professional learning the most prominent knowledge management theories are those of Bereiter, Engestrm and Nonaka [14]. Especially, the socialization, externalization, combination and internalization (SECI) model by Nonaka and Takeuchi [13] has been accepted as a standard model for organizational knowledge creation. We are building learning environments on the assumption that in organizations informal communities of practice want to share knowledge about their profession. Learning in this setting is primarily a social process [3,25]. We refine the SECI model here by adding W. Liu, Q. Li, and R.W.H. Lau (Eds.): ICWL 2006, LNCS 4181, pp. 249–263, 2006. c Springer-Verlag Berlin Heidelberg 2006
250
M. Spaniol et al.
new distinctions in order to apply more comprehensive knowledge sharing and professional learning strategies based on the community-of-practice approach. We present an overview of these theories in the following, as full coverage of the SECI model, and other knowledge and media theories is beyond the scope of this paper, refer to [8] for a more comprehensive discussion. In Figure 1 we depict a hierarchy of knowledge types with examples. The SECI model makes a basic distinction between tacit or procedural and explicit or declarative knowledge [12,13,16]. While even this distinction is under debate, we add one refinement to the SECI model by introducing a distinction between semantic and episodic knowledge [23,24] in a community/organizational context. While semantic knowledge represents semiotic and conceptual knowledge such as documentation in organizational charts, business process definitions and so forth, episodic knowledge is knowledge about experiences such as episodes and narratives, e.g. war stories. This distinction is also being debated. Nevertheless, our claim is that through a combination of semantic and episodic knowledge can be used more effectively in organizations. While situational context may be lost by externalizing stories, outreach and impact of stories may be enhanced by this process. Documentation as a means of semantic knowledge can further be classified as verbal (linguistic data) and non-verbal (e.g. visual image, video, diagram). The Digital Storytelling Association (DSA) defines story-telling as follows: “Digital Storytelling uses digital media to create media-rich stories to tell, share and to preserve. Digital stories derive their power through weaving images, music, narrative and voice together, thereby giving deep dimension and vivid color to characters, situations, and insights” [4]. This illustrates that story telling can be used for sharing knowledge, and thus, learning. While most e-learning and knowledge management systems focus only on semantic knowledge of various kinds, story-telling is much more. Story-telling intertwines semantic knowledge, i.e. already reified concepts of communities stored as documents, by linking it with the narrative experiences gained from episodic knowledge. Thus story-telling can be seen as an approach to develop learning histories [18] by creating knowledge hyper stories [19]. Consequently, story-telling is an important aspect for knowledge sharing and learning in communities of practice. Therefore, telling, sharing and experiencing stories are common ways to overcome problems by learning from the experiences of other members. While story telling is recognized to be important in organizational knowledge creation, there are only a limited number of approaches to improve the influence and impact of story telling by using organization wide information systems. One of the major obstacles may be that authoring of stories is extremely challenging if not supported by a clear and simple methodology. Furthermore, suitable tools need to be in place to support story authors in the use of different media. Thus, the different requirements of authors and recipients of stories can be brought together. We present a proof-of-concept implementation which enables non-formal communities of practice in organizations to create multimedia narratives, to share
Web-Based Learning with Non-linear Multimedia Stories
251
Knowledge
Declarative Knowledge
Procedural Knowledge sensomotoric skills, procedural scripts non-documented routines and operations
Semantic Knowledge
Episodic Knowledge
semiotic concepts documentation
memory of experiencing past episodes web blogs, narratives
Legend: Verbal
Non-verbal
words linguistic data
image, icon, index video blogs, diagrams, images, photographies
individual level community/organizational level knowledge processes by [13] knowledge in story-telling
Fig. 1. Individual and community/organizational levels of knowledge processing. (Adapted from Nonaka and Takeuchi [13] with refinements on declarative knowledge done by Ullman [24]).
and to learn from such narratives. First, we introduce (non-)linear story-telling in more detail. Then we give an overview on existing story-telling systems, explain their underlying theoretical methodologies and discuss their suitability to combine semantic and episodic knowledge. Subsequently, we present the Movement Oriented Design (MOD) paradigm and explain its application in creating nonlinear multimedia stories. In the following, we’ll introduce our implementation of a Media Integrated Story-Telling environment in web-based communities called MIST. We demonstrate the novelty of our approach by combining MPEG-7 based multimedia metadata interoperability for sharing semantic knowledge in a multimedia story-telling environment. From the proof of concept implementation, we present initial test results with an example application for entrepreneurial training. The paper closes with conclusions and an outlook on further research.
2
Linear and Non-linear Digital Story-Telling
Story-telling has a long tradition going back to 330 BC, with the Poetics by Aristotle [1] a seminal work for construction of dramas and epics. Over the millennia a lot has changed in the way stories are produced, in terms of the media used and how they are consumed. With the underlying intention to transfer knowledge and to stimulate learning, the basic idea of story-telling has remained unaltered. However, stories embedded in a story-telling paradigm differ in the way they are produced, consumed and stored. In the next section, we explain the different facets of story-telling.
252
2.1
M. Spaniol et al.
Linear vs. Non-Linear Story-Telling
In general, a story might be linear or non-linear. In a linear story, the plot is completely specified during story creation. Here, the author prescribes a linear path the user will have to follow in order to consume the story. Linear means that each part of the content is meant to be seen or heard in the same order every time it is experienced. Therefore, the only point of interaction on the recipient side is to stop or forward the story-process without being able to influence the story’s narrative path, or the plot. Thus, the only interactivity allowed to “jump” back or forward on the navigation path. Since there is no opportunity to change the outcome of the overall story, this behavior is still considered linear. Non-linearity breaches the “classical” sequential narrative. A non-linear story may have different endings depending on the user interactions taking place during story consumption. Therefore, during the story creation process there is not just a single path through the story, but many alternative paths. Thus, the author combines story elements and links them with one of many successor elements. As a result, this may lead to different stories depending on the recipient’s selection at the interaction points. 2.2
Web-Based Digital Story-Telling
Another aspect for classifying stories is the way these are processed and consumed. Apart from the “classical” way to present stories, new media allow new means to process and present stories. Digital media allows speedier creation, sharing and consumption of interactive content. What makes digital media most suitable for story-telling it the ability to recombine various media types, making stories more effective and interactive. Web-based systems are the ultimate step in the evolution of story-telling by making interactive multimedia contents not only available 24/7, but also allowing community-wide distribution. Despite the opportunities offered by story-telling to combine semantic and episodic knowledge, its full potential for knowledge sharing and e learning is thus far marginally exploited. In the next section, we compare different story-telling approaches and discuss their suitability for knowledge sharing and e-learning.
3
Related Work
There are several models of story-telling; most of these are commercial and aim at the creation fiction. Our overview will focus on systems that are suitable for sharing semantic and episodic knowledge in communities. Dramatica is a comprehensive framework suitable for creating multimedia stories [15]. However, it does not allow any kind of non-linearity. In Dramatica a story represents a particular model called the “story mind”. It is left to the creativity of the authors to express their episodic knowledge as a linear story so that dedicated aspects of the story are filled with content. Dramatica is also capable supporting semantic knowledge.
Web-Based Learning with Non-linear Multimedia Stories
253
Adaptive Digital Storytelling(Adaptive DST) is a computer-based form of narration that tries to integrate basic principles of narratives and dramatic art into interactive digital stories [5]. Adaptive DST subdivides episodic knowledge into selected- and must-phases, and specifies their interdependencies. Another key concept in Adaptive DST is the option to manipulate the story a priori. Here, a variation of a story can be generated based on pre-defined tags used to specify the level of information a user wants to obtain. Based on a 4-ary classification scheme users can select from a superficial to a fine-grained story adaptation. Thus, non-linearity is only supported to a certain extent: The existing “core story” might not be changed completely in its outcome, but might be altered depending on the user’s interest in the topic. While the concept is applicable to knowledge sharing on scholarly level, in general, it is doubtful that such a labor-intense, and mostly unguided creation process might be applicable in a community, or at a larger scale. Storylining Suspense and Story Engine are closely related systems for the creation and consumption of non-linear multimedia stories. While Storylining Suspense is an approach to a new authoring method for interactive storytelling [20], Story Engine is used to capture episodic knowledge by narrating interactive non-linear stories (e.g. created by Storylining Suspense) [2]. The focus in Storylining Suspense is on authoring of non-linear stories based on a set of morphological functions defined by Vladimir Propp [17]. These functions are mapped within the system based on a scene model thus creating variants of a story based on the underlying model and the user’s interaction. Additionally, there are options to store semantic knowledge about multimedia contents, but it is left open, whether these contents are available only to support the creation process or will be accessible upon consumption as well. Despite their client/server structure, Storylining Suspense and Story Engine are suited only on a limited scale for multi-authoring. Hypermedia Novel (Hymn) is a new story-telling approach that extends the classical narration concept of Graphic Novel [6]. Hymn is a modular concept that allows the creation and consumption of hypermedia stories. The main concept of Hymn is the so-called narration module which can be accessed by an authoring tool. A narration module captures the episodic knowledge and stands for a scene within a story. These modules may be linked with other narration modules thus defining the story graph. Despite its clear graph oriented narration structure, Hymn does not seem to apply any theoretical concepts. The “Hymn-player” is a conventional web-browser using the Java Media Framework. Here, different media might be visualized but there is currently no support to store and retrieve semantic knowledge within media related metadata. The Digital Storytelling Cookbook and Travelling Companion is considered to be a handbook for the creation of digital stories based on the “heuristics” gathered in a community of users associated with the center for digital storytelling [10,22]. For that purpose, the DSC breaks down episodic knowledge of digital stories into subcomponents and gives practical advices how to make stories out of user experiences. Besides some practical advices on how to find
254
M. Spaniol et al.
ideas about stories there are seven theoretical elements specified which should be fulfilled in a good story. However, there are no concepts described suitable to process media related semantic knowledge. On the technical level, the DSC only gives hints on how to use proprietary software as a common technical platform to create and share these stories has not yet been developed. In general, the DSC is suitable to support digital storytelling in various areas of application without going into details. The Movement Oriented Design (MOD) paradigm [21] is a new methodology for the creation of linear and non-linear multimedia stories. Its core idea is to bring together different theories, models and tools under one roof. Thus, it integrates features from Dramatica [15] as well as the Aristotelean Poetic [1]. The result is a novel methodology and a formalism in order to create multimedia stories by combining three facets of stories: Motivation (verbal and non-verbal knowledge), Exigency (semantic knowledge) and Structure (episodic knowledge). Thus, the MOD methodology is a comprehensive framework for the creation of non-linear digital stories. However, a prototypical implementation is missing yet. In the previous subsections we have introduced several implementations and methodologies applied in the area of story-telling. As we have pointed out, current approaches are not suitable for combining community support with a comprehensive methodological concept to process semantic and episodic knowledge. Table 1 gives a condensed overview on these approaches by highlighting their key features. For the sake of comparing these features, our Media Integrated Story-Telling (MIST) environment, which contains the theoretical concepts of MOD, is included. Since we want to foster community learning from non-linear multimedia stories we will pursue in MIST an approach that combines both: The benefits of the Movement Oriented Design methodology, and a community centric media support. Table 1. Comparison of existing story-telling environments and methodologies Adaptive Digital Storytelling
Storylining Suspense & Story Engine
Hypermedia Novel
Story concept
n.a.
Must & should dependencies
Morphological functions
Extended Graphic Novel
Semantic Knowledge
Verbal & non verbal
Available
Available
n.a.
Linear
Linear & nonlinear
Linear & non-linear Viewer: Public Editor: ???
Episodic Knowledge Product Type
Viewer: Public Commercial Editor: Commercial / Proprietary
???
n.a.
Verbal & non verbal
Verbal & non verbal
n.a.
Linear & non-linear
Linear & non-linear
Not implemented
Not implemented
Research
Advices only
n.a.
Automatic consistency checks
Creation
Individual
Individual
Individual
Community wide
Not implemented
Not implemented
Community wide
Sharing
n.a.
Proprietary Webserver
Integrated Story Engine
Proprietary Webserver
Not implemented
Not implemented
Affiliated Community Webserver
Validation
Community support
???
Digital Storytelling Movement Cookbook and Media Integrated Oriented Travelling Story-Telling (MIST) Design (MOD) Companion (DSC) Motivation, Motivation, Community Exigency and Exigency and “heuristics” Structure Structure Adopted from MOD
Dramatica
n.a.
Not implemented
Not implemented
Automatic validation of MOD compliance
Web-Based Learning with Non-linear Multimedia Stories
4
255
Media Integrated Story-Telling (MIST)
As we have introduced in the beginning, the more fine grained distinction of declarative knowledge into semantic and episodic knowledge allows a more appropriate classification of the media especially when used for processing the knowledge via multimedia stories. A key concept of MIST is the combination of “static” media related knowledge captured in multimedia documentations with the “dynamic” knowledge stored in the narrator’s memories. Consequently, story-telling in MIST is based on (potentially different) media combined into non-linear multimedia stories. In order to accomplish the management of versatile media types and their “high-level” semantic descriptions we make use of the MPEG-7 multimedia metadata standard [7]. We have adopted a sub-set of this extensive standard to allow sophisticated media specific information to be connected with the MOD story elements. We decompose non-linear multimedia stories into elements that contain structural information and a set of media specific descriptions. Multimedia stories in MIST are decomposed into structural elements and media related contents. At the highest structural level there is the so-called “problem hierarchy” which specifies the problems described by this multimedia story, their interdependencies as well as (potentially many) references to related story elements. Declarative knowledge is captured at this level pre-dominantly. Here, the plot of the multimedia story is crafted. In order to achieve a more fine grained decomposition of the declarative knowledge of a multimedia story, the second structural level specifies transitions between media. Through this process a non-linear story is created and episodic knowledge is depicted. Finally, the semantic knowledge of a multimedia story, such as verbal descriptions and corresponding non-verbal contents, are connected with the story structure itself. For that purpose, story elements are linked with (potentially many) media files and their metadata descriptions. In order to allow a fine grained description of the verbal knowledge associated with the non-verbal knowledge of a media file, all media in MIST contain detailed low-level as well as high-level semantic descriptions. MIST extracts all low-level technical information about the media automatically, while the highlevel semantic descriptions have to be added manually. The wide range of media related information in MIST is covered by a sub-set of the MPEG-7 multimedia metadata standard [7]. Media files - which contain mostly technical information about a medium such as visual encoding, frame rate, color space, etc. - are linked with five more document types (places, objects, time, agents of action, classification scheme) that might contain additional high-level semantic descriptions about a medium (cf. figure 2). By dividing the high-level semantic knowledge stored in MIST into arbitrary document types we allow users to annotate media with high-level semantic entities that might have been created; thus making the use of MIST easier. Since MIST is a community centric client/server application all stories, media and media descriptions are stored on a central server. Technically, the server functionality is split into to an eXist-DB [11] for the storage of the MPEG-7 metadata and an affiliated ftp-server, which is used for automated uploading or downloading of multimedia artifacts by the community members.
256
M. Spaniol et al. Audiovisual features Title Subject/Keywords Description Date Identifier Source Language Relation Coverage Rights
Role
“Media“
“Classification Scheme“
Type Format
AgentObjectType “Places“
“Objects“
SemanticPlaceType ObjectType
“Time“
“Agents of Action“
Creator Publisher Contributor
SemanticTimeType
Fig. 2. Low- and high-level media semantics of media captured in MIST
MIST supports the community wide creation and exchange of multimedia stories. Next, we give a formal notation of stories in MIST and then explain its user interfaces. 4.1
Formal Notation of MIST
All stories created in MIST are compliant to the MOD paradigm. Formally, we define two structures: Non-linear multimedia stories S and their problem hierarchies PS . The universes of both structures are based on a common set of identifiers defined as follows: I ⊆ N, 0
