Towards a Holistic Methodology for Engineering 3D Virtual World Applications Ingo Seidel, Markus Gärtner, Josef Froschauer, Helmut Berger Matrixware Information Services GmbH Operngasse 20b A-1040 Vienna, Austria {firstname.lastname}@matrixware.com
Abstract 3D Virtual World applications are developed in an ad-hoc fashion and the development does not follow predefined processes or methodologies. However, as with any other application, a well informed development methodology increases the efficiency and helps to avoid potential errors. In this paper we are going to present a methodology for the development of applications in 3D Virtual Worlds that is based on results from previous related works, our experiences in developing 3D Virtual Worlds and influences from agile and Web 2.0 software development practices. The methodology is defined on a high level and serves as a guideline covering all stages of the development life cycle. We present two new stages, a familiarization stage and a technology selection stage, which have not been considered in this detail in other methodologies so far and discuss how the methodology has been established within the context of creating a 3D e-Tourism environment.
1. Introduction The development of 3D environments has been going on for approximately 20 years, but there still does not exist a generally agreed upon process for building such environments. The need for a structured development process which takes into account the added complexity of the third dimension, the different nature of interaction as well as different usability challenges has been expressed in the early years already and several researchers have proposed methodologies [9, 18, 22]. Such methodologies usually focus on certain stages of the development cycle and/or are designed for a specific technology. Few methodologies exist that cover the whole life cycle from the start of the project to the end of the project. However, especially for people who are new to application development in 3D environments, a
Dieter Merkl Institute of Software Technology and Interactive Systems Vienna University of Technology Favoritenstrasse 9-11/188 A-1040 Vienna, Austria
[email protected] holistic guideline that provides help during all stages of the development is essential in order to build effective applications. Therefore, the goal of our work is to propose a holistic methodology that covers the whole development life cycle, enables 3D technology novices to get engaged in the realization of 3D applications and is general enough to be applied in different types of 3D environments. The target audience of the methodology are experienced application developers that are new to the domain of 3D Virtual Worlds as well as 3D application developers that want to optimize their development practices and get new insights. The principal structure of the methodology is based on related works that deal with methodologies in the Virtual Reality and 3D Virtual World domain. Key characteristics and methods have been identified and an iterative, user-centered, evaluation driven principal structure has been developed. This structure has then been extended according to our experiences in the Itchy Feet project where we created a 3D Virtual World in the e-Tourism domain [19]. We identified two stages that we have found to be essential in the development of 3D Virtual Worlds and which have not received much attention in other works yet. One stage is a familiarization stage that is intended for novices and conveys the unique characteristics of 3D environments as well as the differences to 2D interfaces such as interaction and navigation metaphors. The other stage deals with the selection of an appropriate 3D Virtual World technology. Each application has a unique set of features and what might be easily achievable in one 3D Virtual World can be impossible in another. Great care needs therefore to be taken when deciding upon the technology to realize the application. As an additional influence, the methodology has been inspired by strategies from agile software development and Web 2.0 application development practices. We believe that the fast paced, user centered and iterative nature of these approaches is
perfectly suited for developing 3D Virtual World applications as well and present a Web 2.0 deployment strategy that ensures a quick availability of the application. The paper is organized as follows. In Section 2 related works are presented. The proposed methodology is introduced in Section 3 and every stage of the methodology is presented in detail. In Section 4 we outline how the methodology was applied in the Itchy Feet project and in Section 5 the paper is concluded.
2. Literature Review There exist several methodologies and procedures for the development of applications in 3D environments. An excellent overview of these methodologies can be found in [13]. In the same paper, Molina et al. propose the TRES-D (ThREe dimensional uSer interface Development) methodology, a methodology for the development of Virtual Reality applications. The TRES-D methodology consists of six major stages that describe the main development activities, but no details on low level processes are given. The methodology has been applied during the development of a web-based 3D online store [17]. Furthermore, the same team of researchers has developed the IDEAS methodology for the development of 2D and 3D user interfaces [14]. This methodology follows a model-based approach where 2D as well as 3D user interfaces are automatically generated. This approach is related to the work of González-Calleros et al. [10] who describe a method to automatically generate 3D user interfaces in three steps. User-centered methodologies with a stronger focus on the usability aspects and the evaluation of Virtual Environments have been proposed by Gabbard et al. [9,11] and Tromp et al. [21]. The process presented by Gabbard et al. consists of four stages with the first one being dedicated to user task analysis while the other stages are dedicated to evaluation. In [21], Tromp et al. present the usercentered development of a Collaborative Virtual Environment. The environment has been build in three iterations starting with feasibility studies and a user needs analysis. Evaluations take a central role to improve the user interface and were carried out in four threads of work. Another framework for the design and evaluation of Collaborative Virtual Environments has been proposed by Dang et al. [3]. In an iterative process, the environment is first designed, then developed and then evaluated. A Collaborative Virtual Environment has been created to verify the suitability of this process and was evaluated in an informal user study. The work of Menchaca et al. also deals with Collaborative Virtual Environments and they present
specific tools for the realization of web-based 3D Collaborative Virtual Environments [12]. These tools are integrated in the first three stages of a development cycle (Analysis, Design, Implementation) in an effort to define a methodology. Fencott focuses on the aesthetic characteristics of Virtual Environments and has developed a design methodology [5]. He distinguishes structural and conceptual/perceptual modeling. While the former is related to the specification of the software system, the latter deals with the design of the environment and how the world is perceived by the users. Wilson et al. have expressed the need for a structured development of Virtual Environments and created a methodology that not only covers the whole life cycle of the development but also provides techniques and tools to implement the different stages [22]. The methodology is called VEDS (Virtual Environment Development Structure) and has been developed over several years with the constant collaboration of potential users. Another methodology that covers the whole development life cycle is named SENDA and has been developed by Sánchez-Segura et al. [18]. The SENDA methodology is based on software engineering processes that have been defined by the IEEE and furthermore integrates techniques from other fields such as Human Computer Interaction and Artificial Intelligence. The presented methodologies share certain characteristics that provide a good basis for the definition of a holistic methodology. These essential characteristics include an iterative process, a strong focus on the user and the tasks the user is going to execute and the repeated conduction of evaluations. These insights have contributed to the principle design of our methodology which is presented in the following.
3. Methodology for 3D Virtual Worlds Our application development methodology for 3D Virtual Worlds is structured into six main stages. These stages are interconnected and certain activities are to be performed during each stage. In the specification pool all documents are collected that are related to the specification of the application. Different types of documents are contributed to the specification pool in each stage and the existing documents in the pool are updated in later stages. This emphasizes the fact that requirements are expected to change during the development and are not immutably defined at the beginning of the project [6]. Additionally, the methodology exhibits an iterative nature as all activities are carried out in an iterative fashion and some of the stages are also linked in a way to enable iteration. A graphical representation of the methodology is depicted in
Specification Pool Requirements Use Cases Scenarios User Tasks
Need for a 3D Virtual World
Stage 1 What is Achievable?
Interface Mockups Sketches Floor Plans
Stage 2 Requirements Specification
Identify
Specify
update existing
update existing
Evaluation Reports Software Specifications
Stage 4 Identify the 3D Virtual World Technology
Stage 3 Initial Design
Design
3D Virtual World Criteria
Evaluate
Stage 5 Realization (Prototype)
Design
Implement
update existing
Bug Reports Evaluation Reports
new features
Evaluate
update existing
Stage 6 Deployment
Maintain
Evaluate
Figure 1. An overview of the methodology. Figure 1. The dashed lines between the stages indicate that an iteration at this point is more costly than the iteration of the activities within a stage itself. The methodology is consulted when 3D Virtual Worlds have been identified as a possible target technology for realizing an application and a need for a 3D Virtual World has emerged.
3.1. Stage 1: What is Achievable? The first stage is dedicated to the familiarization with 3D Virtual Worlds and for getting a deeper understanding on their possibilities and limitations. This stage is especially intended for novices that have not worked with this technology before. In order to get a feeling for the technology, the first step is to explore different 3D Virtual Worlds. A hands-on experience provides an easy access and helps to quickly learn and familiarize oneself with this technology. This exploration is conducted systematically through the provision of a document with good and bad examples of 3D Virtual World applications. This illustrates in which situations 3D Virtual Worlds are more powerful than 2D interfaces and in which situations this is not the case. Additionally, we have compiled a list of 3D Virtual World criteria which is based on a master thesis [1] and has been updated according to our experiences. This list conveys key characteristics and will also be used later in the methodology during the selection of the 3D Virtual World. Furthermore, the knowledge of usability principles is essential for being able to assess the applicability of 3D Virtual Worlds and to be able to properly realize the requirements. This understanding is developed by studying usability guidelines for 3D Virtual Worlds. These guidelines are summarized in a document that is based on revised versions of the usability taxonomy presented by Gabbard and Hix [8] and the usability criteria identified by Stanney et al. [20]. After this stage the technology is better understood regarding the applicability of 3D Virtual Worlds and whether they are the appropriate technology for this project or not. In this stage as well as in the following stage one can come to the conclusion that 3D Virtual Worlds are not suited for the task at hand and that another
technology should be considered for the realization instead.
3.2. Stage 2: Requirements Specification In the requirements specification stage the goal is to develop an understanding of the user needs and to identify the tasks and purpose of the application. The interactive nature of 3D Virtual Worlds and the established practices in related methodologies indicate that the needs of the user should be the central interest. It is advisable to already integrate users from the target population in this stage to get valuable input for the formulation of the requirements. Furthermore, the application domain is an important facet of this stage. Domain experts are consulted in order to properly formulate the requirements with respect to the domain characteristics. In cooperation with domain experts, a user task analysis is conducted and the requirements of the application are derived. For example, Robles et al. [17] observed the behavior of people in a shop environment and conducted interviews with employees to define the requirements of a 3D online store and Tromp et al. [21] have used consumer evaluations to find out the requirements. When the user tasks and requirements have been established, use cases and usage scenarios are created. Use cases capture the different activities that are carried out in the environment while usage scenarios describe longer interaction sequences defining the scenarios in which the application will be used. These methods have proven to be effective means in other methodologies as well [5, 9, 11, 13, 22].
3.3. Stage 3: Initial Design The first set of requirements is used to create an initial design of the application. This corresponds to the phase of abstract design in [13] and conceptual design in [5]. The design created at this point is platform independent and no decision for a technology has been made yet. Similar to other methodologies [3, 5, 13, 17, 22], paper prototypes, user interface mockups, sketches and floor plans are created.
The design documents are developed in an iterative Design-Evaluate cycle where evaluations are carried out to constantly refine and improve the designs. Evaluations in this stage take the form of expert-guideline based evaluations and formative evaluations [9]. In an expert-guideline based evaluation the interfaces and designs are evaluated by experts according to usability and design guidelines. Identified problems and issues are reported and updated versions of the designs are created. In a formative evaluation the application is evaluated with target users that execute tasks and whose behavior is observed. In order to involve users at this stage, paper-based prototypes and interface mockups are used to create a first usage experience of the application.
importance of every characteristic is defined by a weight that ranges from 1 to 3. The scores for each technology are calculated serving as a basis for the decision on the 3D Virtual World technology. Especially in this stage a change of previous design or requirements documents might become necessary due to characteristics and limitations of the chosen technology that were not known beforehand.
3.5. Stage 5: Realization The implementation of the application is carried out in an iterative cycle consisting of the activities Design, Implement and Evaluate. Design in this context refers to the design of the software itself. The details of this activity as well as the implementation depend on the technology that has been chosen in the previous stage and the established practices in the work environment. In general we suggest to use an agile software development approach [6] for an iterative fast paced development. With this approach prototypes become available quickly and it is easier to react to changes in the requirements that are determined in evaluations of these prototypes. In terms of the evaluation, formative and summative evaluations [9] are conducted in this stage. Formative evaluations are executed on functional prototypes in this stage. The degree of detail of these evaluations ranges from ad-hoc evaluations in the first iterations to more systematically executed evaluations in later phases. We have found that qualitative evaluations have proven to be especially useful at this point. In a qualitative evaluation the focus is on the opinion of the user and it is easier to discover the attitudes towards the application than in quantitative experiments. A summative evaluation is carried out when the application is finished and has the goal of comparing the task performance in the developed application with the performance of the same tasks in other systems.
3.4. Stage 4: Identify the 3D Virtual World The 3D Virtual World technology that is chosen has a great influence on the feasibility of the implementation. We believe that a process for choosing the appropriate technology is essential and have therefore created a method for selecting the best suited technology. The method is based on research in the domain of software evaluation where the goal is to evaluate different software products against a set of requirements and to chose the best suited software product for the given requirements. In particular, the method is based on research by Punter [16] and Colombo et al. [2] which use checklists for evaluating qualitative criteria of software products. A checklist contains questions that are related to quality characteristics which are of importance for the application to be developed. These questions are derived from the catalog of 3D Virtual World characteristics that has been introduced in Section 3.1. Those characteristics that are relevant for the application are included as questions in the checklist and different candidate technologies are evaluated against them. An example of a checklist is shown in Table 1. For each candidate technology the question is evaluated on a rating scale ranging from 0 to 5. This value indicates the degree to which the characteristic is fulfilled by the technology. Example questions are: Is avatar customization supported? How well is the privacy protected? Is it possible to fly around with the avatar? Additionally, the
3.6. Stage 6: Deployment In the last stage, the application is deployed. The deployment strategy can vary greatly depending on the type of application that was developed. In the deployment stage we therefore distinguish between
Table 1. Checklist for the evaluation of different 3D Virtual World technologies. Question
VW 1 𝑣11
Question 1 ⋮ Question n Sum
𝑣12
∗ 𝑤1
𝑠1 =
𝑣1𝑘 ∗ 𝑤1
∗ 𝑤𝑛
⋯
𝑣𝑛𝑘
𝑣𝑖2 ∗ 𝑤𝑖
⋯
∗ 𝑤1 ⋮
𝑣𝑛2
∗ 𝑤𝑛
𝑣𝑖1 ∗ 𝑤𝑖
VW k
⋯
⋮ 𝑣𝑛1
⋯
VW 2
𝑠2 =
⋮
𝑠𝑘 =
∗ 𝑤𝑛
𝑣𝑖𝑘 ∗ 𝑤𝑖
rapid deployment and traditional deployment. Under the term traditional deployment we understand an application that is developed in several iterations until all features are integrated and which is then released as a final product. In contrast, in rapid deployment a functional application is delivered quickly, possibly after the first iteration in the realization stage already. At this point only a small subset of features is available, but users are able to use the application much earlier than in the traditional deployment strategy. New features are constantly added to the application in the following and the number of available functions steadily grows. This approach is based on development practices of Web 2.0 applications where a first version is released early and new features are slipstreamed into the online application [15]. We believe that this deployment strategy is especially well suited for applications in online 3D Virtual Worlds. In this case no re-distribution of the application is required and users instantly profit from the added value of new features. In contrast, if the application is designed to be installed and executed on target machines, the traditional deployment strategy is better suited. Besides the deployment strategy, maintenance and evaluation are activities that take place in this stage. Formative as well as summative evaluations are carried out, requirements and design documents are updated according to the results and the application is constantly improved in an iterative loop with the Realization stage.
4. The Itchy Feet Project An initial version of this methodology has been used in the Itchy Feet project for the development of a 3D e-Tourism environment [19]. In the following we outline the development process of the 3D Virtual World and show what we have learned. The 3D Virtual World of Second Life has proven to be a manifold environment for the familiarization stage and later we also conducted research in this world [7]. In contrast to the present version of the methodology, we had conducted the selection of a technology in the beginning of the project already. At this point we did have several ideas for an application in a 3D Virtual World, but no detailed analysis had been made yet. In later stages we then found out that the chosen technology lacked support for the implementation of some of the application features and that we should have considered application specific requirements in the technology selection as well. Therefore, the technology selection has been moved to a later stage in the methodology in order to establish a detailed understanding of the features before the selection process takes place. We then carried out a task analysis where we identified the essential functions for an application in
the e-Tourism domain. This was done through the analysis of e-Tourism portals where we compared the different characteristics of such platforms and identified the key features [4]. For the identified tasks and roles we then modeled use cases to specify the task sequences in more detail. As a next step we started with the initial design and created userinterface mockups, drew floor plans and made sketches of the 3D Virtual World. During this stage we also had created a detailed software requirements specification which was intended to be used as a blueprint for the implementation. A great effort was required to update the specification according to changes in the requirements and we came to the insight that a smaller level of detail is favorable in order to be able to react quicker to any changes in the specification. After finishing the implementation we conducted a formative user evaluation with the goal of evaluating the acceptance of a 3D Virtual World in the tourism domain. As mentioned previously, a qualitative approach has proven to be useful and we were able to get valuable insights on the users attitudes towards the created environment.
5. Conclusion & Future Work In this paper we have presented a new methodology for the development of applications in 3D Virtual Worlds. The six stages of the methodology were introduced and we have discussed how the methodology evolved in the Itchy Feet project and what lessons we have learned. The methodology is currently applied in the development of a cultural learning application which extends the existing 3D e-Tourism environment. This will help us to establish further knowledge on methodological properties and possibly contribute to a revised version of the methodology. Furthermore, we will apply and evaluate the rapid deployment approach in another project which has the goal of creating an online 3D museum.
6. Acknowledgments This work was funded by the FWF Austrian Science Fund (project reference: L363).
7. References [1] M. Baumgartner. Real-world business in synthetic worlds: World characteristics and their relevance for commercial activities. Master’s thesis, Vienna University of Economics and Business, 2008. [2] R. Colombo and A. Guerra. The evaluation method for software product. In Proc. of the International Conference on Software & Systems Engineering and their Applications (ICSSEA ’02), 2002.
[3] N.-T. Dang, C. Chatelain, J.-M. Pergandi, and D. Mestre. A framework for design and evaluation of collaborative virtual environments. In Proc. of the 3ème Journées de l’Association Francaise de Réalité Virtuelle, Bordeaux, France, 2008. [4] B. Dippelreiter, C. Grün, M. Pöttler, I. Seidel, H. Berger, M. Dittenbach, and A. Pesenhofer. Online tourism communities on the path to Web 2.0 - an evaluation. Information Technology & Tourism, 10(4):329–353, 2008.
[13] J. P. Molina, A. S. García, V. López-Jaquero, and P. González. Developing VR applications: the TRES-D methodology. In Proc. of the First International Workshop on Methods & Tools for Designing VR Applications, Ghent, Belgium, 2005. [14] J. P. Molina, P. González, M. D. Lozano, F. Montero, and V. López-Jaquero. Bridging the gap: Developing 2D and 3D user interfaces with the ideas methodology. In Proc. of the 10th Workshop on the Design, Specification and Verification of Interactive Systems, pp. 303–315, 2003.
[5] C. Fencott. Developing Future Interactive Systems, chapter A Methodology of Design for Virtual Environments, pp. 66–91. Idea Group Inc., 2004.
[15] T. O’Reilly. What is Web 2.0. http://oreilly.com/web2 /archive/what-is-web-20.html (acc. 13.01.2010), 2005.
[6] M. Fowler. The new methodology. http://martinfowler. com/articles/newMethodology.html (acc. 13.01.2010), 2005.
[16] T. Punter. Using checklists to evaluate software product quality. In Proc. of 8th European Conference on Software Cost Estimation (ESCOM), pp. 143–150, 1997.
[7] J. Froschauer, I. Seidel, M. Gärtner, H. Berger, and D. Merkl. New directions in science communication: A virtual research & experience landscape. In Proc. of the World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, pp. 2590–2597, Vancouver, Canada, 2009. AACE.
[17] A. Robles, J. P. Molina, V. López-Jaquero, and A. S. García. Even better than reality: The development of a 3-D online store that adapts to every user and every platform. In Proc. of the 11th International Conference on HumanComputer Interaction (HCI ’05), Las Vegas, NV, USA, 2005.
[8] J. L. Gabbard and D. Hix. A taxonomy of usability characteristics in virtual environments. Technical report, Virginia Polytechnic Institute and State University, Blacksburg, Virginia, November 1997.
[18] M.-I. Sánchez-Segura, A. de Amescua Seco, and J.-J. Cuadrado. Software engineering and HCI techniques joined to develop virtual environments. In Proc. of the Workshop on Bridging the Gaps Between Software Engineering and Human-Computer Interaction, pp. 20– 25, Portland, OR, USA, 2003.
[9] J. L. Gabbard, D. Hix, and J. E. Swan. User-centered design and evaluation of virtual environments. IEEE Computer Graphics & Applications, 19(6):51–59, 1999. [10] J. M. González-Calleros, J. Vanderdonckt, and J. Muñoz-Arteaga. A structured approach to support 3D user interface development. In Proc. of the 2009 Second International Conferences on Advances in ComputerHuman Interactions (ACHI ’09), pp. 75–81, Washington, DC, USA, 2009. IEEE Computer Society. [11] D. Hix and J. L. Gabbard. Handbook of Virtual Environments: Design, Implementation and Applications, chapter Usability engineering of virtual environments, pp. 681–699. Lawrence Erlbaum Associates, 2002. [12] R. Menchaca, L. Balladares, R. Quintero, and C. Carreto. Software engineering, HCI techniques and Java technologies joined to develop web-based 3D collaborative virtual environments. In Proc. of the 2005 Latin American conference on Human-computer interaction (CLIHC ’05), pp. 40–51, New York, NY, USA, 2005. ACM.
[19] I. Seidel, M. Gärtner, M. Pöttler, H. Berger, M. Dittenbach, and D. Merkl. Tourism Informatics: Visual Travel Recommender Systems, Social Communities, and User Interface Design, chapter Itchy Feet: A 3D e-Tourism Environment, pp. 209–242. IGI Global, 2009. [20] K. M. Stanney, M. Mollaghasemi, L. Reeves, R. Breaux, and D. A. Graeber. Usability engineering of virtual environments (VEs): identifying multiple criteria that drive effective ve system design. International Journal of Human-Computer Studies, 58(4):447–481, 2003. [21] J. G. Tromp, A. Steed, and J. R. Wilson. Systematic usability evaluation and design issues for collaborative virtual environments. Presence: Teleoperators and Virtual Environments, 12(3):241–267, 2003. [22] J. R.Wilson, R. M. Eastgate, and M. D’Cruz. Handbook of Virtual Environments: Design, Implementation and Applications, chapter Structured Development of Virtual Environments, pp. 353–378. Lawrence Erlbaum Associates, 2002.
