SOFTWARE ENGINEERING CHALLENGES IN NEW MEDIA APPLICATIONS Amitava Biswas Department of Computer Science, Texas A&M University College Station, TX 77840, US
[email protected]
ABSTRACT: Pervasive computing and mobile technologies have provided non-technical designers with new media to design unique user experiences. Traditional software engineering methodologies and tools are poorly suited to develop these new media applications. All these pose nontrivial challenges when engineers develop new media applications in co-operation with designers and artists. We propose a novel software engineering approach and a roadmap to tackle such problems.
KEYWORDS:
Design collaboration, prototyping, new media application.
rapid
1. Introduction Due to the rising emphasis on satisfaction of higher order human needs, leisure over work, and socio-economic shifts [1], expectations from technologies are evolving and new kind of applications [2 to 6] are emerging. Non-technical designers are defining alternative uses of technology that go beyond the common notions of functionality [3 to 8]. They often call mobile, wearable, pervasive, ubiquitous, context aware, inter-active technologies, as "new media" because they use these as new design material/media to express creative ideas. Through their design, they are creating new “values” for users, which can be: “user experience”, “aesthetics”, “social engagement”, “user commitment”, “co-operation”, etc. [3,4,6,8] Development of these new media applications often can not follow the traditional software engineering or life cycle development trajectory that is suitable for well understood, archetypal problem contexts, like enterprise/business system development. This is largely because new media application development contexts may not support the basic assumptions on which these software engineering methodologies and tools are based upon. Traditional engineering practice assumes that system requirements can be inventoried a priori [9,10] to a reasonable extent. Often in "new media art" applications, the emergent system specifications can not be defined in sufficiently tangible terms till the very end of the project (case study 1 & 2, next pg.). In some interactive social applications, user expectations from systems are difficult to specify in formal or precise terms or even in natural language narratives. Software engineers are not familiar with extremely intangible, amorphous, ambiguous and difficult to comprehend concepts like “user experience” and “values” [4 to 6]. Yet they have to develop systems that meet these real-world goals. The process of capturing
Jagmit Singh Ontario College of Arts & Design Toronto, ON, Canada
[email protected] requirements and then specifying the system attributes is further complicated when the system usage context is entirely absent or it is not well understood. Even the sociologists, ethonologists, skilled requirement engineers or human computer interaction specialists, who are deemed experts in such matters, may not yet understand them. These situations have emerged because user’s expectations from systems now go beyond well defined, well understood and easy to comprehend quality attributes like functionality, usability, performance etc. Such development contexts pose significant challenges to the mainstream software engineering and associated process management practice. To cater to this accelerating evolution of user expectations, new software engineering methods and processes are required to: capture new class of highly intangible and volatile requirements; adequately define system specifications; successfully embody the expected quality attributes in the emergent system; and satisfactorily deliver the expected values to stake holders. This paper motivates adoption of new software engineering methods, processes and practices to serve new media development contexts. It presents two case studies to highlight specific software engineering issues in new media production environments. A case study on a toolkit is presented to introduce a new software engineering approach that may be followed to enable a multidisciplinary, co-operative, new media application development process. The learnings derived from these project experiences are integrated with the available ideas from design and human computer interaction domain, and presented in form of a software engineering roadmap.
2. Need for new engineering methods "New media art" is type of new media application where an artistic idea is expressed using technology or new media artifacts (case study 1). New media interactive applications, which are distinct from new media art, may have specific expectations behind them [case study 2, 2,4,6]. Traditional requirement engineering practice is yet to develop a systematic approach to deal with these kinds of requirements. Rather than the technologists, the nontechnical designers, who have the necessary skill sets, generally carry out the requirement engineering tasks in these experimental and exploratory new media projects. Therefore technologists are forced to play a very limited role unless they had sensitized and educated themselves about non-technical design and contextual issues. Due to heightened interests from mobile, telecom and
entertainment industries, these applications are gaining ground. Hence software engineering practice needs to develop adequate concepts, ideas, methodologies and usable tools, and assimilate emerging ideas from other domains to serve these kinds of application development needs. The following section identifies specific problems that thwart usage of traditional software engineering methods in new media art and interactive application development contexts. Case study 1 (Locative sound-scape in Park Emile Gamelin, Montreal): An artist created an audio-visual installation in an urban park without any specific mandate about the nature of the deliverable. This software application ran on a handheld computer/PDA (HP iPAQ). Users enjoyed audio (music, interview, recordings, etc.) or video (pictures, video clippings, animations) as they roamed freely in a large urban park with a miniature GPS receiver. The audio-video playback was triggered by user's location (GPS co-ordinates). The content was a collage of audio-video components which were artistically rendered to deliver unique user experience, entertainment and information (history of the location). No coding was involved as the artist used the GUI based Mobile Bristol toolkit [10], to bind the audio and video objects to specific GPS co-ordinates and location change events. However the artistic creativity was severely limited within the rudimentary locative media and design style, as envisioned by the tool kit designers. No other form of context awareness or flexibility was offered by the toolkit. It could only run on Windows Pocket PC for the HP iPAQ. Full motion video was impossible due to limited CPU power and inefficiencies of the Flash video player. However this toolkit allowed the artist to rapidly develop this application without any help from software engineers.
3. Software engineering challenges In new media art installations, artists often require nonquotidian technological artifacts that do not exist and have to be developed. Artists often lack technical capabilities to develop the required variety of technological artifacts themselves, so they depend on technologists. Therefore, unlike in traditional art, in new media, the artist is forced to maintain dialogue with his media via technologists. Thus, much complexity and difficulty arise from the process in which practitioners of two disparate domains work together. Artists find it difficult to maintaining high fidelity dialogue with the media via agents, whereas technologists face difficulty in eliciting requirements and embody these requirements in the emergent system. Technologists can not take part in the artistic design process and the artists do not intend to participate in the rigorous technology development process. So both practitioners have to follow their own separate work processes. The artistic design can not be and should not be decided a priori, to preserve the core value of the artistic creativity. Creative artist’s work processes do not necessarily follow “analyze-model-design-build” trajectories like engineers. They iteratively and intutively generate creative ideas and evolve their design based on their perception and experience. Hence, most often they can not visualize and express even a low-fidelity version of their content design at early stages of the project. As, the requirements and system specifications for the technological artifacts are tied to the artistic content
design, therefore the system specifications for all the technological artifacts can not be satisfactorily determined a priori. On the otherhand, in every iteration, artists prefer a near-complete high-fidelity system to review their design before they decide to modify it in the following design cycle. But system developers need specifications to be clearly spelt out to build even the first high-fidelity system. Case study 2 (Memichi at Banff national park, Alberta, Canada): Memichi was a context and location aware game application that ran on smart mobile phones. The game was supposed to: provide unique user experience, entertain, and engage visitors at Banff national park. A multi-disciplinary team of technologists, designers and multi-media production crew developed it. Broad game content ideas were generated in "participatory design" brain storming and creativity sessions by the team. Later the game designers evolved this content through multiple iterative design cycles. In each cycle, they required reasonably complete software version for each alternate content scenario before they could choose one and freeze some specifications for the final software. To make this possible, the software engineers built a tool which automatically generated software code within few hours, once the content is formally specified. The designers specified alternate contents and the tool automatically generated software versions to choose from. The tool architecture and its development were carefully planned so that the core modules were available in the early phases of the project. During the project the software engineers went on developing and maturing the tool whose design was decided a priori. This insulated them from the frequent changes in the content specification. The content production crew went on producing the multi-media content artifacts, whereas, creative designers went on experimenting with the alternative version of the design and evolved it. This tool and appropriate project management allowed the three application development flows to independently progress and yet interface at well defined points. It allowed incorporation of multiple system features at any time, hence the game application manifested rich context awareness. The application recognized a pre-determined set of hand gestures in addition to location. It also allowed generation of device optimized high performance codes, hence the application enjoyed high quality audio-video rendering even on mobile phones. It could build different versions of the application for different handheld devices other than smart phones.
Conventional software engineering method that can break this chicken-egg deadlock is quite expensive. Traditionally, "evolutionary prototyping" is used in such uncertain/evolution contexts [12]. In such method, artists will generate creative ideas, technologists will receive briefing from artists, build prototypes, elicit modifications/corrections and further requirements from the artists and this cycle will be repeated till the artist is progressively satisfied (Fig. 1). For each artistic design cycle there can be multiple expensive prototyping cycles. Requirement Engineer 2. Briefs Artist/ Designer
4. Low fidelity feedback
1. Creative idea generation 8. High fidelity feedback System prototype
3. Analyzes & documents System Requirements
6. Low fidelity feedback
7. Build process
5. Design process System Design
Fig. 1: New media art production by traditional approach
Other than prototyping we have not come across any other workable solution. Even this kind of prototyping scenarios need appropriate software engineering tools and processes to reduce the cost of each cycle. Definitely, there are opportunities for improvements and novel software engineering solutions. The subsequent paragraphs elaborate the software engineering issues associated with such prototyping methods. Requirement engineering challenges: The “prototypingrequirement elicitation” cycles can be expensive due to difficulty in communication and idea sharing. Capturing artist’s visions, even as informal qualitative narrative form, is quite difficult because artist and technologists do not share a common vocabulary. Both practitioners have to go through a learning process to understand the subtle meanings and connotations of each other’s domain specific terminologies, models and their semantics. Secondly the team has to concoct new methods of design representations to share and exchange design ideas, which both artists and technologists can comprehend and which are suitable for the adopted art genre. These group learning processes consume time, effort, intellectual energy and creates distractions. Art practice is rich with variability in regards to art genre and artist’s individual style. This variability is associated with creativity, hence it needs to be preserved from technologist’s temptation of standardization. Thus, this constraint on standardization hinders reuse of the shared vocabulary, design visualization and communication tools. This is likely to keep the process expensive over longer term, unlike enterprise/business system applications, which benefited from standardization. A lot of high fidelity content design elements may not get translated in to system design in first few iterations. Many design information get lost in the process of briefing, analysis, documentation and initial system design. This is due to technologist’s lack of sensitivity towards highfidelity detailing, which is influenced by their propensity for abstraction over contextual details. System analysis and design obstacles: Technology domain information models are insufficient to capture the quantity, richness, and depth of high fidelity contextual information that the artists or creative designers work with. For example, they often deal with socio-cultural contexts. These complex contexts are human interpretations that assume complex knowledge in form of cultural exposure as necessary pre-conditions. These are difficult to model with the present state-of-the-art context meta-models, ontologies and artificial intelligence technologies, therefore systems rarely behaves as demanded/expected by the artists. Thus both artists and engineers have to negotiate and agree upon low fidelity proxy system features that can be delivered within limits of feature bloat and yet deliver the basic minimum value to the artistic installation. Even the rudimentary expectations may translate into a large quantity of unmanageable complex system design artifacts due to unsuitability of technical design methodologies, analytical tools, and ontological
artifacts. The sheer volume of contextual information also makes management and traceability of requirements quite difficult across system design and build phases. System developed by evolutionary prototyping may suffer from lack of coherency in its architecture due to inadequate planning. Coherency is important for separation of concerns, easy and rapid modifications. On the otherhand, these systems may not be tractable to the usual split architecture design strategy, where the core components are designed in planned manner to deliver functionality but the interactive and presentation parts are iteratively prototyped with user feedback. This is because interactivity is the core functionality for new media applications, therefore the core parts may not be easily insulated from frequent change in requirements. Development and coding issues: Manual code development is too expensive in terms of effort and need for elaborate and expensive regression testing, thus it can not support multiple design cycles and frequent change in requirements, unless the application is small and simple. GUI based toolkits along with specific run time environments are available [10] but they have limitations (case study 1), which severely restricts artistic creativity. These limitations arose because toolkit designers could not possible envision all possible design media, art genre and technology platforms a priori. They attempted to follow the traditional product development strategy, where a basic feature set is chosen that is assumed to satisfy some basic needs of a large number of users. This assumption did not hold true due to wide variety in personal design-art styles. Toolkits need to have user modifiable features to extend the design space, to incorporate new vocabularies or to enable designer's own work processes. Testing and validation problems: Testing ensures correct technical operation of the system, but it does not ensure its appropriateness or its effectiveness in delivering the expectations. Acceptability is decided by the designer/artist. This validation may involve situated rehearsals involving complete cycles of unit testing, build, installation, integration testing in representative usage contexts. So it is expensive and it slows down the iteration cycles. Thus all these factors: need for intensive high fidelity dialogue between the creative artist and the emerging technological artifacts; complexity and richness of the context; wide variability in artistic design process, content genres, and technology, etc; thwarts traditional software engineering methods. New media interactive application development may pose additional challenges. Today neither technology nor any other domain can provide any straightforward roadmap on how to transform high level expectations to non-ambiguous system functionalities or architecture attributes. However a multi-disciplinary team will be are more adept in visualizing system requirements [13]. But even these experts lack the required contextual knowledge,
hence they will need to study the usage context before identifying probable sets of requirements. Due to insufficient knowledge base, these multi-disciplinary contextual studies are more expensive than the requirement elicitation-negotiation processes for enterprise/business system development. These new media requirement elicitation processes will also require appropriate multidisciplinary research methods, knowledge acquisitionmanagement tools and processes, much of which are yet to be developed. Even this requirement elicitation method is a trial and error process, which have to be part of iterative evolutionary prototyping. The probable requirements that are identified, may have to be implemented as low or high fidelity prototypes and then validated through a series of usability studies. This means that the requirement engineering process will be even more expensive than the case of new media art. There is also an issue about scale in these applications. A large number and variety of users may use these applications, therefore requirement elicitation should encompass sufficiently large variety of usage situations. This increases the scale of contextual study and validation. To allow validation across large variety of usage contexts, the application may be put in production and then evolved as more users use it and provide feedback. Under such circumstances, the architecture and the application deployment should support rapid user validation and evolution even when it is under production.
4. Prototype generator: an appropriate tool The part of the application development workflow which are standard and well understood, can be automated to reduce its cost and tighten the loop. A significant portion the software code can be generated from formal requirement specifications and tested automatically by an appropriately designed tool (Fig. 1). Thus a prototype generator consisting a code generator and tester and can facilitate development over multiple iterative cycles. Code generator automates code generation whereas the tester can ensure consistency of the code generator, the code generation process and the integration of the build. Contextual knowledge assimilation, design and requirement generation phases can be left to the experts who can utilize their personalized skills and knowledge. Communication between artists and technologists can be facilitated to ease the design knowledge and information transfer bottleneck. Low-fidelity prototyping can facilitate this communication and stimulate creative design idea generation. Thus it is possible to transform the “system design and development” problem to a manageable “prototype generator design” problem and the development work can be split up into two fairly independent workflows. Mobile Experience Engine (MEE) toolkit and associated project management methods materialize this approach (Fig. 2). The toolkit development workflow concentrates on evolving the prototype generator. While the application production workflow focus on delivering application
prototypes as required by the designer/artist. Utilizing this toolkit the requirement engineer in the production team can formally specify artist's requirements, rapidly generate code and build the prototype executable. The code generator builds high fidelity prototypes, whereas the visualizer generates low-fidelity visualizations from the design specifications. For the Memichi game project, the visualizer was designed to illustrate the conceptual design and flow of events in form of high level state diagrams, which artists could comprehend and relate to. Requirement engineering
System architect
Specifies
Visualizer, Code generator & Tester requirements
Offline design process Engineer 3. Analyzes &
2. Briefs Artist 1. Creative idea generation 7. High fidelity feedback System prototype
5. Low fidelity feedback 4. Generation process Visualization
6. Build process
System requirement Generates visualization Generates code
System Design Offline development Visualizer Code Generator
Checks
Tests the build
Consistency integration, build tester
Fig. 2: New media art production process with prototype generator. The low-fidelity prototypes or "visualizations" delivered multiple benefits. They facilitated the communication between the designer/artist and the requirement engineer. They also provided quick low-fidelity feedback about the design idea, so that the designer/artist could evolve the design without going through the entire cycle of highfidelity situated prototyping. These visualizations might have even stimulated creative design ideas. All these benefited the requirement elicitation process. The consistency of the code generator, the code generation process and the integration of the build was verified by test scripts which constitutes the tester component of the prototype generator. The system architect of the Memichi project had some experience with different new media art genres and thus could decide significant portion of requirements for the prototype generator beforehand. This solved the problem of inability of contemplating requirements a priori. The key parts of the code generator could be developed before the production workflow started and the toolkit could contribute in the prototyping process. Detailed architecture of MEE has been discussed in [14], and briefly described here. The contextual information model is embodied in XML schemas, the system requirement specifications are formally specified as XML descriptions which conforms to the schema. The C++ source code is generated directly from the XML
requirement specification descriptions though multiple steps as defined in the XSL style sheet files (Fig. 3). Requirement Engineer
Specifies
Genre specific XML description of requirement specification Transforms
Genre specific XSL Generated by partial hand coding
Genre independent XML description of requirement specification
Core Code generator
Transforms
Code generator XSL Generated C++ source code for prototype Code generator C++ library
Is linked
Compilation & build process
Prototype executable
Fig. 3: Automatic code generation The code generator is embodied in the style sheets and core C++ libraries. The visualizer generates XML based Scalable Vector Graphics (SVG) file that can be rendered by a SVG browser to get the design representation (Fig. 4). The code generator and visualizer can be customized by limited hand coding effort to cater to different art genres or design styles. Developer
Generates by hand coding
Visualizer core XSL (Genre independent) Generates by partial transformation and partly by hand coding
Genre specific XML description of requirement specification Generated by transformation and partial hand coding Genre independent XML description of requirement specification Transforms
Visualizer XSL XML SVG file Rendered by SVG browser Visualization (low-fidelity prototype)
Fig. 4: Visualization generation The MEE library and style sheets consist of three parts: (1) a core part, which is extensible but relatively static; and customizable parts which are specific to (2) design style, content/art genre and (3) the run time platform/device. The design style and content genre specific parts aim to reduce the semantic gap between the artist's or designer's model and formal system specifications and system model. A XML based content specification language was developed which can be customized and extended to suit different kinds of art/content genres or design styles. The information models embodied in the specification language closely follow the models used by the artist or designer for the specific project. Thus artist's/designer's thoughts can be more easily transformed in to formal specifications. The genre specific style sheet XSL transforms this specification to genre/style independent content specification that are suitable for source code
generation transformations by the core code generator style sheets and library. The platform/device specific part of MEE ensures generation of performance optimized codes so that the application executable delivers maximum quality and performance on a given device. This is an important issue for mobile applications which are supposed to run on a wide variety of handheld devices. Rapid introduction of mobile devices in market and their obsolescence necessitate this requirement. Such partitioning of MEE aids adaptation and evolution of the code generator to match availability of mobile/handheld devices. The C++ libraries are encapsulated within XML descriptions to allow reflection by other prototype generator components, like consistency tester scripts.
5. Software engineering roadmap Two distinct schools of thoughts assuming two different sets of beliefs have attempted to address this question "how to develop new media applications/installations". The traditional outward looking, engineering influenced approaches begin with identification of user contexts and needs, which is followed by iterative prototyping and evaluation cycles to progressively decide the application design and materialization to attain satisfactory level of user contentment. An alternate creativity inspired, inward looking introspective and reflective approach, involves creating an installation to deliver unique user experience. The first process oriented approach gives a confidence about the basic minimum quality of the deliverable. Whereas, the second inspiration driven approach, allows artistic creativity that may deliver a better quality of deliverable, if certain preconditions are met, for example, presence of a creative director. This second approach is favored by creative director/ artists. However engineering concerns and solutions are quite similar in both the approaches, and are addressed below. The first approach is suitable for research and commercial projects where a primary objective is to identify which human needs can be satisfied today with the available state of art technologies, and how those needs can be satisfied [1,6]. Therefore in this approach a multi-disciplinary design team having a broader collective perspective and knowledge about users should pursue application development by iterative prototyping. The prototyping tasks are: researching and modeling the usage context; design idea generation and validation; transferring the design to technologists; prototype development; and situated validation. In the creativity inspired approach, researching and design idea are best left to the creative artist/ director. However for process oriented approaches, research productivity can be improved by adoption of appropriate research methods. Creative design idea generation methods can facilitate quicker requirement identification. Design transfer can be improved by adopting communication aids, sensitizing technologists and involving them in multi-disciplinary contextual research. In case of large applications, wherever possible, system development productivity should be improved by
wisely splitting the application architecture into two parts, one dealing with interactivity, the other tackling core functionality. The interactive part of the application can be progressively prototyped by outward looking, user assisted methods. Whereas the core functionality part can be engineered by inward looking, expert lead method. Automatic code generation like MEE can tighten the system development loop or GUI based design tools like [11] can eliminate it. Research and creative idea generation are the two most untractable bottlenecks, but a variety of research and creative idea generation methods can be adopted from design and human computer interaction domain literatures. Some of these are described below. Focused research like [4,5] can clarify and aid modeling of apparently complex, ambiguous and volatile requirements. Contextual, ethnographic studies, naturalistic inquiry, usability analysis, role analysis, informal expert review, can generate factual information and improve understanding about usage behavior [15, 16,17]. User assisted prototyping, “participatory prototyping” [18], and prototyping combined with usability studies [6,12] are other popular techniques to extract situated knowledge about user behavior. To extract and assimilate knowledge from prototyping, the process have to be managed and appropriate knowledge management tools and methodologies have to be adopted [19,20]. Each prototyping cycle should have associated research and development objectives, and at the end of each cycle an assessment should be made about the achievements in that cycle. A combination of multidisciplinary research strategy involving “general field study”, “descriptive modeling”, “prototyping”, “model revision” and “focused field study” can assimilate contextual knowledge across multiple observation-design/prototyping cycles [21]. After information extraction, modeling is the next step for knowledge building. Modeling aids, meta-modeling [22 to 25] and ontological artifacts will facilitate this. Managing the identification, collection, analysis and representation of contextual knowledge will be key to improve productivity. Hence, study, co-ordination [26] and knowledge assimilation protocols and tools that embody these methodologies are needed in production environments. A significant part of requirement engineering for new media applications is about envisioning future and creating innovative design ideas. A wide variety of inward or outward looking methods are available. Outward looking methods, either participatory or experimental, involve users and the usage environment to generate design ideas. Participatory methods involve users as participants of the design/requirement articulation and generation process. Experimental methods carry out experiments on users. These experiments are distinct from non-invasive ethnographic observations. Various types of prototypes may be used to observe user's interaction with the system. High fidelity computer based prototypes elicits more ideas and participant involvement compared to paper based prototypes for computer applications. But paper based
ones may help to elaborate each individual idea [13]. Even low-fidelity paper based, role play based, “experience prototype” can stimulate design ideas for systems that intends to deliver “user experience” [4]. Social “disruptions” and “breaching experiments” may provoke “sociological or design imaginations” and design ideas [27] for innovative “social circumstances” applications whose usage context have not yet emerged. Many mobile, wireless and GPS applications fall under this category. To ensure sufficient generality of the generated requirements, a larger user group should be targeted by scaling up some participatory requirement elicitation techniques [13]. In inward looking methods, multi-disciplinary team of experts, designers and technologists introspect, identify and analyze themselves to gain more in-depth understanding of the requirements. To identify new applications of technology related to leisure and entertainment, multi-discplinary team can take part in a “technology inspired” approach which involves “playful visions of technology”, “playful explorations” and reflections. Analysis of the ethnographic observations on the participating experts can further enrich the requirement identification [28]. Creative exercises and “Brain storming”, “Method 635”, “Random input” methods can stimulate creative idea generation and refine ideas. In “Future workshops” the stakeholders can clarify current problems, create visions about the future and generate ideas about how to realize them [29]. Aesthetically inclined requirement engineers can follow and assimilate certain design methodologies suitable for interactive applications, which have some form of physical expressions. They can start with general notions of functions and general usability criteria, then progressively and iteratively refine them into concrete design features for the interactive parts of the application by performing a series of exploratory and reflective exercises involving “expression” and “functionality” [8]. However structured processes like [30] are required to distill requirements from the creative idea generation processes. Readily customizable and extensible object oriented information models implemented by XML along with object oriented programming can partially remedy the lack of rich high-fidelity information models. A flexible and superior alternative is to use ontological artifacts and metamodels, but this requires identifying or developing ontologies suitable for the given usage context. It may possible to progressively build and extend ontologies suitable for various art genres and design situations. Ontologies can be shared, reused, enriched and extended by community sharing and participation facilitated by web service technologies [31]. Wider participation and contribution will gradually cover all variations of art genres, design practices and contexts. Indexed library of art and new media domain ontologies can be envisioned and implemented. Automated high and low fidelity prototype development tools and environments like MEE and their development should be actively considered. To facilitate large scale validation and quantitative analysis,
the application should have features for automatic usability data collection as in [11].
6. Related works Rapid prototyping had been used as a requirement elicitation tool in technical [32] and context-aware applications [33], when the usage context was unknown. Automatic code generation [32] or visual interface programming [34] has been explored in rapid prototyping. Management of prototyping process have been discussed by [19,20]. Software engineering approaches suitable for interactive applications have been argued for [1,6] and introduced by [6,35].
7. Conclusion In this paper we have articulated the need for new software engineering processes, methods and tools that are suitable for new media applications. We analyzed two examples of such system development contexts, where conventional software engineering assumptions do not hold true. We have identified the key engineering problems, then we showed how automatic software code and design visualization generation facilitate the iterative prototyping process in such circumstances. Finally we presented a software engineering roadmap for similar system development projects.
References [1] Oulasvirta, A., (2004), Finding meaningful uses for context-aware technologies: the humanistic research strategy, Proce. of the SIGCHI Conf. on Human factors in computing systems, 247 - 254. [2] Fujinami, K., Yamabe, T., Nakajima, T., (2004) "Take me with you!": a case study of context-aware application integrating cyber and physical spaces, Proce. of the 2004 ACM symposium on Applied computing. [3] Kjeldskov, J., Paay, J., (2005), Just-for-us: a context-aware mobile information system facilitating sociality, Proce. of the 7th Int. Conf. on Human computer interaction with mobile devices & services MobileHCI '05. [4] Buchenau, M., Suri, J, F, (2000) Experience prototyping, Symposium on Designing Interactive Systems, Proce of the Conf. on Designing Interactive Systems, New York City, New York, US, 424 -433. [5] Forlizzi, J., Battarbee, K. (2004), Understanding experience in interactive systems, Proce. of the 2004 Conf. on Designing Interactive Systems, Cambridge, MA, USA, 261 - 268. [6] Cockton, G. (2005), A development framework for value-centred design, Conf. on Human Factors in Computing Systems, Portland, OR, USA, 1292 - 1295. [7] Rogers, Y. et al., (2002), Things aren't what they seem to be: innovation through technology inspiration, Proce. of the Conf. on Designing Interactive Systems, London, England, 373 - 378. [8] Hallnas, L. , Redstorm, J. (2002), Abstract information appliances: methodological exercises in conceptual design of computational things, Proce. of the Conf. on Designing Interactive Systems, London, England , 105 - 116. [9] Finkelstein, A., Kramer, J., (2000) Software engineering: a roadmap, Proce. of the Conf. on The Future of Software Engineering, 3 - 22. [10]Nuseibeh, B., Easterbrook, S. (2000) Requirements Engineering: A Roadmap, Proce. of the Conf. on The Future of Software Engineering, ACM Press. [11] Mobile Bristol Toolkit. http://www.mobilebristol.com [12] Alvi, M. (1984), An assessment of the prototyping approach to information systems development, Communications of the ACM, Vol. 27 (6), 556 - 563. [13] Mackay, W. E. (2004), The interactive thread: exploring methods for multi-disciplinary design, Proce. of the 2004 Conf. on Designing Interactive Systems, Cambridge, MA, USA, 103 - 112.
[14] Biswas, A. et. al. (2006), Assessment of Mobile Experience Engine, the development toolkit for context aware mobile applications, Proce. of the Int. Conf. on Advances in Computer Entertainment Technology, LA, US, ACM Press. [15] Sommerville, I., T. Rodden, P. Sawyer, R. Bentley, and M. Twidale (1993), Integrating ethnography into the requirements engineering process, Proce. of The IEEE Int. Requirement Engineering Conf., San Diego, CA., US, 165-173. [16] Potts, C. (1997) Requirements Models in Context, Proce. of the 3rd IEEE Int. Symposium on Requirements Engineering, 102 Gutierrez, O. (1989), Prototyping techniques for different problem contexts , Proce. of Conf. on Human Factors in Computing Systems, 259 - 264. [17] Vredenburg, K., Mao Ji-Ye, Smith, P. W., Carey, T. (2002) A survey of user-centered design practice, Proce. of the Conf. on Human factors in computing systems, Minneapolis, US, 471- 478. [18] Maze, R. , Bueno, M. (2002), Mixers: a participatory approach to design prototyping, Proce. of the Conf. on Designing Interactive Systems, London, England, 341 - 344. [19] Purtilo, J. Larson, A., Clark, J. (1991), A methodology for prototyping-in-the-large, Proce. of the 13th Int. Conf. on Software engineering, Austin, Texas, United States, 2 - 12. [20] Schneider, K. (1996), Prototypes as assets, not toys: why and how to extract knowledge from prototypes, Proce. of the 18th Int. Conf. on Software engineering , Berlin, Germany, 522 - 531. [21] Mackay, W.E., Fayard, A-L (1997), HCI, Natural Science and Design: A Framework for Triangulation Across Disciplines, Proce. 64 of ACM Designing Interactive Systems, ACM Press, 223-234. [22] Rolland, C., Grosz, G. Kla, R. (1999), Experience with GoalScenario Coupling in Requirements Engineering , Proce. of the 4th IEEE Int. Symposium on Requirements Engineering, 74. [23]Dardenne, A., Lamsweerde, A,V., Ficks, S. (1993), Goal-directed requirements acquisition, Science of Computer Programming, Volume 20 , Issue 1-2, 3 - 50. [24] Nurcan, S. , Rolland , C. , (1997), Meta-Modelling for cooperative processes in The 7th European-Japanese Conf. on Information Modelling and Knowledge Bases, 27-30 Toulouse, 361-377. [25] Williams, A. (2002), Assessing prototypes' role in design, Proce. of the 20th annual Int. Conf. on Computer documentation, Toronto, Ontario, Canada , 248 - 257. [26] Tai, G. (2005), A communication architecture from rapid prototyping, Int. Conf. on Software Engineering, Proce. of the 2005 workshop on Human and social factors of software engineering, St. Louis, Missouri , 1 - 3. [27] Crabtree, A. (2004) Design in the absence of practice: breaching experiments, Proce. of the 2004 Conf. on Designing Interactive Systems, Cambridge, MA, USA. 59 - 68. [28] Rogers, Y. et al., (2002), Things aren't what they seem to be: innovation through technology inspiration, Proce. of the Conf. on Designing Interactive Systems, London, England, 373 - 378. [29] Loewgren, J ., Stolterman, E ., (1999), Methods & tools: Design methodology and design practice. Interactions, Vol. 6, No. 1, 13-20. [30] Boehm, B. W. Gray, T. E. , Seewaldt, T. (1984), Prototyping versus specifying: a multiproject experiment, IEEE Transactions on Software Engineering, Vol 10 , Issue 3 , 290 - 302. [31] Kim, C-S, Park, L-L., Choi, C., Kim, S-D, Sensor abstraction and context delivery model based on web service for context-aware computing, Proce of the 17th IASTED Int. Conf. Parallel and Distributed Computing and Systems, Nov 2005, Phoenix, AZ, USA. [32] Luqi, Steigerwald, R.; Huges, G.; Berzins, V. (1991) CAPS as a requirements engineering tool, Proce. of the Conf. on TRI-Ada '91, San Jose, California, US, 75 - 83. [33] Long, S., Kooper, R., Abowd, G.D., Atkeson, C.G., (1996), Rapid prototyping of mobile context-aware applications: the Cyberguide case study, Proce. of the 2nd annual Int. Conf. on Mobile computing and networking, NY, US, 97 - 107. [34] Sohn, T.Y., Dey, A.K., (2003), iCAP: an informal tool for interactive prototyping of context-aware applications, Extended abstracts on Human factors in computing systems, Conf. on Human Factors in Computing Systems CHI '03, Ft. Lauderdale, USA, 974 - 975. [35] Abowd, G. D., (1999), Software engineering issues for ubiquitous computing, Proce. of the 21st Int. Conf. on Software engineering, LA, US, 75 - 84.
