Using Modular Abstract Prototypes as Evolving ...

Proceedings of the ASME 2012 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference IDETC/CIE 2012 August 12-15, 2012, Chicago, IL, USA

DETC2012-70050 USING MODULAR ABSTRACT PROTOTYPES AS EVOLVING RESEARCH MEANS IN DESIGN INCLUSIVE RESEARCH Imre Horváth Faculty of Industrial Design Engineering Delft University of Technology Delft, the Netherlands [email protected]

Els Du Bois Engineering Product Development Artesis University College Antwerp, Belgium [email protected]

ABSTRACT Various manifestations of products, prototypes and tools are commonly used in design research to discover and describe novel phenomena, or to test specific research theories, or to explore intrinsic data that cannot be accessed and validated otherwise. However, as research means, the above physical artifacts are over-detailed and inflexible, in particular when phenomena associated with design creativity and product ideation are investigated. To support design inclusive research in the context of conceptualization and early testing of complex, knowledge-intensive software tools, the authors propose modular abstract prototyping. The original goal of abstract prototyping was to demonstrate the real life processes established by new artifact-service combinations, as well as the interactions of humans with them in various application scenarios. A modular abstract prototype relies on a comprehensive information structure. The demonstration contents of the modules are defined by a stakeholder and purpose oriented logical dissecting of this information structure, and implemented as digitally recorded, multi-media enabled narrations and enactments. This paper discusses the technical aspects of developing modular abstract prototypes, and their use as flexible and evolving research means. A complex application example is presented in which modular abstract prototyping was used in focus group sessions to assess the conceptualization of a trade-off forecasting software tool by various stakeholders. This tool is being developed for forecasting energy saving and financial benefits that can be achieved by ubiquitous augmentation. The stakeholders have formulated positive opinion about the level of immersion and the articulation of informing that can be achieved by using modular abstract prototypes. Future research focuses on the development of a

web-hosted engine for real-time interactive abstract prototyping in participatory research sessions. 1. INTRODUCING THE ADDRESSED RESEARCH ISSUE This paper presents the concepts and results of our multidisciplinary research which focuses on the inquiry intensification in design research, and aims at placing the early development of software products on a scrutinized information basis. The concrete phenomena studied in our work is assisted informing in design research in the context of participatory (stakeholder inclusive) conceptualization and development of software products. The relevance of this research comes from the fact that there is a growing demand in the software industry to develop products according to the wishes of the various stakeholders, and to consider the explicit demands as early as possible in software development [1] [2]. There are, however, two concerns with respect to these objectives. First, stakeholders are usually not experts in software conceptualization and development. The abstract and incomplete information that is available in the fuzzy front end and early development phases may be too abstract for them and does not contribute to their dispositional context. Consequently, informing stakeholders about the functionalities, behavior, usage and other features of software tools, and demonstrating how the developed tools influence their real-life processes is an important issue [3]. Second, software designers and engineers need to conduct operative research outside their laboratory environments where the means of a comprehensive demonstration may be constrained. Therefore, there is a growing need for effective and rigorous methodologies and appropriate assisting instruments (adaptable research means)

1

Copyright © 2012 by ASME

of modular abstract prototypes (MAPs) through a real-life application case. The exemplified design inclusive research cycle of a promotion research project has made an intense use of multiple MAPs. They were presented to various stakeholders in focus group sessions to support their assessing of the conceptualization of a software tool, which is being developed for forecasting energy saving and financial benefits that can be achieved by ubiquitous augmentation. This paper discusses not only the methodological aspects of implementation, but also the technical issues concerning the development of MAPs and their use as evolving research means in systematic design research. The next Section of the paper discusses some important and relevant principles of informing science. Section 3 explains the main concepts and the process flow of design inclusive research. Section 4 elaborates on the need for using early prototypes as tangible hypotheses and, in particular, on the concept and implementation of modular abstract prototypes. Section 5 discusses the main issues of stakeholder-inclusive early software development. Section 6 presents a demonstrative application case and explains the way of using modular abstract prototypes in presenting the functional and structural framework of a design support software tool. Section 7 discusses the findings and concludes about the work and the results, respectively.

Figure 1 Knowledge domains involved in the presented research that allow them to elicit data and information that they could not get to otherwise [4]. Meeting all these requirements is not as easy as it seems at the first sight because there is a hiding challenge in bringing together all necessary knowledge from multiple disjunctive disciplinary fields. These fields are indicated in Figure 1. As shown, our research work focuses on the issue of synergic software product development, and spreads over and combines knowledge from four domains of interest: (i) using the principles of informing science in order to build effective communication bridges to stakeholders of software developers, (ii) applying the methodological framing and completing research cycles as design inclusive research (DIR), which concentrates on the development of explanatory and predictive theories through designing or using artifacts, (iii) enabling (disciplinary or operative) design research by dynamically changeable research means in order to have access to facts and knowledge that are not accessible otherwise, and (iv) developing an approach of abstract prototyping that offers high flexibility and expressiveness, while maintains efficiency and utility. In the forefront of our research is finding a defendable answer to the question: How to support design inclusive research with adaptable research means in the early phases of software design processes? Our research hypothesis has been that, by further articulating the concept of abstract prototyping, a sufficiently flexible research means can be developed and included in DIR processes. We also assumed that this research means can be used widely, no matter if research is done in a human context, or in an artifact context. The highlights of our research can be summarized as follows: We propose modular abstract prototyping (MAP) as a structured methodology for producing and using early, rich and demonstrative digital representations of software product ideas and as a supportive means for interrogative, experimental and interventional design research. We have obtained information about the applicability

2. RELEVANT PRINCIPLES OF INFORMING SCIENCE There are three major issues related to conducting effective participatory research: (i) the amount of information to be provided for the participants without causing large perceptive and cognitive biases, (ii) the manner of providing information with respect to the mental models of the participants, and (iii) extraction and aggregation of research data and interpretation of them in context. All these issues are addressed by informing science that is a novel, but a rapidly developing domain of knowing. Cohen, E. defined the goal of informing science as “to inform the clients with proper information so that their effectiveness is maximized” [5]. Towards this end, informing science investigates how to provide clientele with information in a form, format and schedule that maximizes its effectiveness” [6]. In general, informing science considers: (i) the cognitive information processing mechanisms of humans, (ii) the essence, content, forms and context of communicated information, and (iii) the approaches through which informing can be adapted to people, systems and environments [7]. Information is a cognitive content. It is well-known that human information processing is associated with the structure and characteristics of human cognitive architecture that is studied by cognitive informatics. Sweller, J. identified five basic principles for natural information processing in complex systems (e.g. in human cognition system). These are: (i) the information store, (ii) the borrowing and reorganizing, (iii) the randomness as genesis, (iv) the narrow limits of change, and (v) the environmental organizing and linking principle [8]. Kalyuga, S. investigated the implications of the cognitive load framework theory for informing and identified several ways of

2


are performed within the context of pre-existing social structures, which are neither inviolable nor permanent [19]. This has also become an essential principle for informing science, and many researchers have tried to convert it into structural, modality and interaction principles for advanced information systems [20]. In our work, we considered the above discussed principles of informing in informational design of MAPs, as well as in developing the scenarios and information flows for focus group sessions.

reducing cognitive load [9]. Specific recommendations for improving the process of informing include: (i) providing sufficient guidance in the absence of relevant knowledge in long-term memory, (ii) reducing split-attention, (iii) using multiple modalities, (iv) eliminating redundancy, (v) taking into account the levels of client’s expertise, (vi) considering the evolutionary type of communicated information, (vii) regarding the transitory nature of information, and (viii) considering cognitive transaction costs in multi-agent informing processes. Further influencing factors having been investigated by other researchers are: (i) interpersonal reasoning [10], (ii) human information behavior [11], (iii) personal cognitive attitude to communication [12], and (iv) different professional backgrounds and mindsets [13]. The informing environment is typically a complex one that can be viewed at three levels of abstraction: (i) the instance of being informed by an existing system, (ii) the creation of new instances of informing, and (iii) the creation of new designs for informing [14]. One strong assumption of informing science about information is that it is true, or at least, it can be considered as such in appropriate contexts. As argued by Stahl, B.C., if information is not true, then the meaning associated with it is wrong or misleading, therefore untrue information cannot inform perception or action [15]. Many researchers argued about the importance of matching the process of informing with the human mind and studied various manifestations of this. Obviously, incompleteness, vagueness, or complexity of information can have dramatic impact on the nature of informing. Koh, C.E. et al. hypothesized that a higher information quality is correlated with a higher level of satisfaction with the information, and a higher level of satisfaction with information is correlated with a higher level of performance expectancy [16]. Lang, M. identified four critical elements of successful communication between academics and non-academics professionals: (i) establishing appropriate channels toward a different non-academic audience, (ii) use of a commonly understood language, (iii) creating overlap in terms of common knowledge between participants in the communication process, and (iv) eliminating the lack of a cumulative tradition [17]. Walton, D.N. and Krabbe, E.C. distinguished six primary functions of human dialogs: (i) information-seeking (expecting answer to some questions), (ii) inquiry (collaborating with someone to answer some questions), (iii) persuading (persuade someone to accept a statement), (iv) negotiation (bargaining over sharing some resource), (v) deliberation (joint decision on action in situations), and (vi) argumentation (eristically seeking to promote believes). Several theories, frameworks and methodologies have been proposed to include the above and other principles of informing science in various electronic communication systems, artificial reasoning systems, executive information systems, group decision support systems, computer-aided collaborative work systems, and other advanced information technology-based systems for sophisticated information management. One is Giddens’ structuration theory that argues that all human actions

3. FRAMING DESIGN RESEARCH BY SYSTEMATIC METHODOLOGIES The co-conduct of research and design, as well as the challenges originating in the need to achieve a substantial methodological rigor in the practical implementation of design research, and to extend the opportunities of an effective and focused knowledge inquiry, have been addressed by design researchers for a long time [21]. Having different philosophical stances, positivist, phenomenologist, constructionist, instrumentalist and pragmatist investigators casted light on the differences and the similarities between the essence of designing artifacts and inquiry through research variously, and offered different and not converging explanations [22]. Part of the extremists claims that designing and researching are practically the same, while the other part claims that they are completely different activities. The reason why we brought up this dilemma here is to clarify that, on the one hand, we recognize the cognitive similarities of designing and doing research (as human learning processes), but, on the other hand, we also recognize their ontological, epistemological, methodological and praxiological differences. Actually, we think that there are some key dissimilarity indicators, such as: (i) the different objectives of the processes (creating useful artifacts vs. generating tested knowledge), (ii) dependence of the outcomes (results) on the process and contexts (producing the outcome in a subjective manner or process vs. striving after objectification and decontextualization), (iii) the role and the way of handling uncertainties (overbridging by human intuitions/experiences vs. striving after eliminating biases and errors), (iv) the rate of independence of the knowledge produced from its producer (being a manifestation of personal beliefs vs. a tested proposition agreed upon by many peers), (v) the type of theories that are applied and produced (dominantly specific and vague vs. reasonably generic and justified), and (vi) the criteria of validation of the results (fulfillment of pragmatic requirements vs. satisfying rigorous criteria) [23]. Several theories have been proposed to explain the proper interoperation of design and research processes (or vice versa) [24]. The related studies tried to clarify how design and research can be interwoven in a single process, in which both a rigorous inquiry and experiential creativity are addressed concurrently [25]. The commonly discussed approaches are such as research through designing, practice-driven design research, design action research, case study-based design

3


stakeholders, (ii) facilitating the alignment of mental models and developing a shared awareness, (iii) creating agreement (consensus) about a policy or decision, and (iv) nurturing the commitment of stakeholders with design decisions. In research through designing, prototypes serve to instantiate hypotheses from contributing disciplines and communicate principles, beliefs and considerations among subjects and investigators. Using prototypes is in line with the task statement of constructivist researchers, whose intent is to explore the various constructions, or “created realities”, based on collective reasoning and reflections [32]. In an earlier work, various framing methodologies, such as ‘research in design context’, ‘design inclusive research’ and ‘operative design research’, have been proposed to support conducting rigorous design research [33]. These methodological frameworks are underpinned by respective theories. They can be operationalized (instantiated) in a concrete research cycle by organizing the flow of research and design activities, selecting appropriate research methods, and using them according to the logics imposed by the conceptual frame. Design inclusive research stipulates the use of a generic procedural structure that is shown in Figure 2. On the other hand, this framing methodology can be instantiated into many different concrete methodologies by choosing different sets of methods. The DIR process decomposes to an explorative stage, a constructive stage, and a confirmative stage. These stages further decompose to various phases of particular objectives and actions as described in [33]. This procedural structure can be applied to one or more research cycles within one research project, and can be combined with the other two framing methodologies depending on the objectives. One generic example can be seen in Figure 3. Here DIR has been used as the methodological framing of research cycles DIR#1 and DIR#2, and facilitates the use of two manifestations (abstract and tangible) of the prototype. It should be noted that we used the word ‘framing’ above in a meaning that follows from the combination of its specific conceptualizations in cognitive science and social science. In cognitive science, framing expresses a particular state of the mind. In social science, framing is a schema of interpretation that supports understanding and responding to events. A framing methodology is also a semantic filter in the sense what is proper from reasoning and procedural perspectives. Framing of research cycles defines the packaging of the reasoning, procedural and content elements of the methodological conduct of research so as to encourage certain interpretations and carrying-outs, and to discourage others. Like a mechanical frame, which includes a structure of pieces to support nonstructural elements, DIR, as a methodological frame, also includes a structure of activities distributed over various phases. Nevertheless, framing research cycles as DIR is seen as a ‘rule of thumb’, rather than as a law that always leads to the desired and correct results. The goals of the explorative activities are: (i) aggregation of knowledge and constructing new knowledge related to the

research, and design inclusive research. The desire of a fluent interaction of the two activities in experimental design research is also reflected by concepts such as ‘tangible hypothesis’ [26], ‘embodied conjectures’, or ‘evolving exemplars’ [27]. These concepts indicate that the synergy between design and research should be achieved via instruments other than (the conduct of the) processes. They also cast light on the necessity (and the possibility) of applying some new ways of reasoning, such as e.g., ‘reasoning with consequences’. Sandoval, W.A. et al. introduced the concept of embodied conjectures in order to provide a tangible practical means for testing in experimental research, instead of applying formal hypotheses for logical justification [28]. It sounds obvious that converting the knowledge (theories), derived in the explorative phase of research, into various prototypes provides a research means for design researchers, which is close to their mindset and typical way of thinking [29]. Less obvious is, however, how the logical and methodological issues related to an indirect (and/or implicit) justification of hypotheses and theories can be managed based on reasoning with consequences, and how the evolution of design requirements, concepts and contexts can be grasped in design research processes. One of the most often mentioned strategy of carrying out design with a research flavor is research through designing. This approach, however, has been variously interpreted, described and conducted by different researchers. The reason is that it still misses a rigorous methodological framing. Some researchers claimed that the essence of research through designing is to develop design prototypes of various fidelities and to use them in the design process as support means and catalysts of inquiry towards non-specific knowledge [30]. Other researchers claimed that the essence of research through design is making extrapolations and generalization from repeated, evolving, or multiple design processes [31]. Prototypes, like other perceptible forms of expression, such as sketches, diagrams and scenarios, are seen to be the primary means by which designers can build connections between different domains of knowledge, and iterate towards an end product. Prototypes are supposed to be relatively high-fidelity replicas of designs. Generally, they are created and used in design processes for the following purposes: (i) presenting the results of the activities either in digital or in material form for

Figure 2 The three stages of design inclusive research

4


Figure 3 Methodological framing of subsequent research cycles in a research project operative knowledge (principles), or both, depending on the objectives and conduct. It has been observed in various practical applications of DIR that the used product samples or fully detailed materialized models are often over-detailed and inflexible. This observation casts light on the fact that probably there is optimum amount information that research means should capture and deliver to interrogative, observational and experimental studies. In particular, this is an issue when results of early product conceptualization are to be demonstrated and investigated. For this reason, using abstract prototypes with adaptable contents for testing software in the early phase of development has been considered in our research. In the next section, we introduce the concept of modular abstract prototyping and its use to support informing in design inclusive research.

research problem and its surroundings, (ii) formulation of a critique of the current understanding and existing approaches, (iii) defining the research questions and development of hypotheses, (iv) setting the goals of the design activities, and (v) development of comprehensive theories to solve the research/design problem. The theory generated at the end of the explorative stage explains what information a tangible design prototype should be based on. This information is combined with the less rigorous design information in the implementation. It means that both internal and external views are concurrently present in design inclusive research. The control of the knowledge blending process is ensured by the confirmative phase of research. Various manifestations and evolving forms of the object of designing can be considered as a research means in DIR. Most frequently, this means manifests in various forms of prototypes, which are conceptualized and detailed in the creative stage, and then used in the confirmative stage to support of data elicitation and collection. A prototype acts as a dual-face tangible hypothesis, which is proper and useful for doing research, as well as for designing products. Based on the findings of the latter stage, both the explanatory theory and the prototype can be changed. From an epistemological point of view, the recurrent involvement of design processes contributes to an evolving theory building in context [34]. This way, DIR makes it possible to gain access to research data that cannot be obtained and tested without having an abstract or tangible implementation of the object of design. The confirmative stage comprises actions that are orientated to: (i) justification of the constructed theory and the outcome of the creative activities based on the strategy of ‘reasoning with consequences’, (ii) internal validation of the research methods and the design methods, (iii) external validation of the findings of the research and the results of the artifact development, and (vi) consolidation of the results by matching them against the existing body of knowledge and generalizing them towards other applications. Thus, DIR allows generating either disciplinary research knowledge (insights) or

4. APPROACHES OF ABSTRACT PROTOTYPING Using abstractions is becoming important in software engineering due to the growing complexity of software products, and the necessity of reducing uncertainties at the beginning of the development process. Abstract models support the comprehension of complexities and simplification of complex systems. They also help extend the available design information by means of conceptualization and imposing reasoning frameworks. Abstract prototyping has been considered in software engineering as an effective technique for the representation of functional contents and navigational maps, and as an instrument of early implementation of various parts of the software, in particular, its user interfaces [35]. As discussed by Constantine, L. and et al, abstract prototypes may range from simple content inventories of the parts of the user interface to highly structured and formalized abstract prototypes based on canonical abstract components [36]. Traditional abstract prototyping supports functional specification of possible contents and logical organization of the various components of the user interface without jumping into the details of layout, appearance and behavior. In canonical abstract prototyping, a

5


without actually producing a workable tangible prototype. Three approaches have been identified, namely: (i) generic abstract prototyping (GAP), (ii) modular abstract prototyping (MAP), and (iii) interactive abstract prototyping (IAP). By definition, GAP is an implementation of the underpinning information structure in a demonstration means of a monolithic architecture. It presents the foreseen real-life process(es) in their embedding environments, including the involved humans and their actions. GAP assumes that the demonstration is orientated towards one stakeholder, or a homogeneous group of stakeholders. As opposite, MAP assumes that the demonstration is for multiple stakeholders of different backgrounds, mind sets, interests, and demands, and that they are interested in the assessment and improvement of the demonstrated concepts and implementations from multiple different perspectives. Including all pieces of information in one single abstract prototype would be unpractical either from a cognitive, or from a practical point of view. Hence, MAP reflects a context-dependent dissecting of the information structure into functional modules that can be combined according to the stakeholders and their demands. From an information technological point of view, a MAP is a modular architecture, which allows an independent development and flexible adaptation of the modules according to multiple stakeholders and demonstration sessions. The information sub-structures encapsulated in the modules necessitates both a semantic and a technical analysis. The modules are supposed to contain complementing, rather than overlapping information sub-structures. A demonstration session can be decomposed to a series of separate sub-sessions, in which the dedicated MAP contents are presented to the different stakeholders. This way, (i) the development of the demonstration materials can be more efficient and flexible, (ii) the information overload of the stakeholders can be reduced in comparison with that of generic APs, and (iii) and the demonstration sessions can be more intensive and stakeholder oriented. It is also an advantage that, by taking into consideration the stakeholders’ opinions, the number of necessary iterations can be reduced and iteration cycles can be made shorter. The number of modules (i.e. the resolution of the MAP) is closely related to the number and interests of the different stakeholders involved in the assessment process. A specific combination of the modules should ensure that the optimal amount and pieces of information are provided to each of the stakeholders, in a cognitively controlled way. In addition, other principles, for instance, template-based development or maximal reusability can also be considered in the methodology of modular abstract prototyping. A MAP has been defined as a comprehensive (selfcontained), content-wise dissected information structure, which is operationalized for demonstration as a combination of digitally recorded, multi-media enabled narrations and enactments (Figure 5) [42]. Eventually, the module development involves an analysis of the necessary information from the perspective of efficient informing, structuring the contents, and designing the narration and enactment. An

Figure 4 The main constituents of the information structure of abstract prototypes canonical collection of abstract (visual) components are used to construct a specification of user interfaces. An overview of the recent abstract prototyping concepts, approaches, technologies and developments is given in [34]. However, the concept of abstract prototyping is redefined and extended in that paper. The overall task of abstract prototypes (APs) has been defined as bringing future realities into present situations. The targeted application field has been identified as conceptualization and design of artifact-service combinations. In this context, the goal of abstract prototyping is to demonstrate the functioning of these complex products in various application environments and to make the foreseen reallife application, operation and interaction processes assessable. The theory of abstract prototyping relies on a number of assumptions, one of which is that an AP is supposed to allow a systematic demonstration of concepts for stakeholders in early phases of the product development process. It has been shown that the proposed approach is actually application domain independent, that is, it can equally well be used in other domains, such as innovation of industrial products and participatory development of software and service products. The distinguishing features of APs is that they (i) are based on a sophisticated information structure, (ii) contain all information constructs that are needed for human understanding of the proposed concepts, (iii) are multi-media enabled and digitally recorded means, and (iv) dominantly and concurrently operate in the perceptive and cognitive domains of human intellect. The generic information structure of APs has been formally specified in [37]. As shown in Figure 4, the involved information constructs are to describe: (i) the novel artifact and service combination, (ii) the foreseen real-life process induced by it, (iii) the involved human actors, and (iv) the surrounding environment. The process of development of APs is driven by the context information concerning: (i) the above constituents, and (ii) the concerned stakeholders. The demonstration of the information structure is facilitated by the use of a set of appropriate media means. Abstract prototyping is about designing realistic scenarios

6


5. ISSUES OF STAKEHOLDER-INCLUSIVE EARLY SOFTWARE DEVELOPMENT Preceded by the clarification of the objectives, assessment of the alternatives, and reduction of risks, an industrial software development process is typically decomposed to (i) requirements specification, (ii) high level architectural design, (iii) abstract functional design, (iv) human interface design, (v) functional component design, (vi) data structure and flow design, (vii) algorithm and component design, (viii) component implementation and testing, (ix) component integration and testing, (x) system operation testing and optimization, (xi) use acceptance testing, and (xii) operation and maintenance phases. As the list of phases shows, testing plays an important and critical role in the process. Software testing efforts also account for a large part of the software development costs. Comprehensive testing is much more than just finding and eliminating bugs in the software. Testing should extend to the evaluation of functional and non-functional properties, and to the satisfaction of the potential users. It is obvious that if testing happens as an early confirmation with the involvement of various stakeholders, rather than as a retrospective analysis, then many iteration and adjustment steps can be eliminated and the confidence of the stakeholders stakeholders can be increased. Software assessments may manifest in three forms, namely as (i) software demonstration contexts inspection, (ii) software walkthrough, and (iii) software testing. Software inspection abstract prototype contents focuses on examining the source representation with the aim of discovering anomalies and defects with the involvement narration enactment of people. It may be applied to any representation of the software system (requirements, design, test data, etc.) and, because it does not require execution of a system, can be used before implementation. Software inspection is mainly concerned narration enactment with analysis of the static system media 1.A episode #1 segment #1.1 representation to discover problems (static gnmn anchor #1 kjfdfdffdesd verbal segment #1.2 anchor #2 piyrsdjhgffse aaa communication verification). It has been reported by many media 1.B segment #1.3 asdeew anchor #3 mhdsyeu dfweissswert anchor #4 ffdfd software development experts as a very segment #1.4 anchor#5 o bdst yldd dfdgdg effective technique for discovering errors. segment #1.5 media 1.C Software walkthroughs are informal examinations of a software product (or of episode #i segment #i.1 media i.A gnmn anchor #1 kjfdfdffdesd textual its technical documents). It is usually segment #i.2 anchor #2 piyrsdjhgffse aaa communication asdeew anchor #3 mbvttrteu media i.B completed by a cooperating team of segment #i.3 software experts, such as concept developers, architecture designers, code media n.A segment #n.1 episode #n developers, interface specialists, media n.B segment #n.2 sert tt anchor #1 rrrrrdffdesd verbal/textual rtrtrt anchor #2 piyrsrt aeew data/knowledge base experts, clients/users, communication segment #n.3 trrttrttrrt anchor #3 mhdsyeu media n.C next phase developers, and software quality dfweit anchor #4 fftrtttrtrrtdfd segment #n.4 managers. Apart from the common and traditional computer tools, this work is typically supported by insufficient specialized software. For this reason, the Figure 5 The relationships between contents of narration and enactment advantage of MAP is that different parts of narration and enactment can be produced by different abstract prototype developers or knowledge engineers, or by using significantly different media. In addition, a MAP allows concurrent content development for and parallel implementation of multiple modules, and, if needed, enables an easier change of the modules. Certain kernel modules can be repeatedly used in demonstrations for different stakeholders. Another practical advantage of using MAP is that decomposing a complex problem to more manageable modules reduces the challenges and the risks, (but raises the need for a careful structural design). The modules may have volatile relationships with the whole of the MAP and with each other. In principle, they may be arranged according to various structural patterns, such as a linear chain, a tree, a loop, or even a web. In practice, the structure of the modules is determined by the body and flow of information needed for an efficient and comprehensive informing.

7


in the software, which is also supposed to allow a highly interactive usage, to provide opportunities for extending the case base, to include new (more sophisticated) cost and energy calculation algorithms, and to give a context-sensitive design guidance for the users. In order to be able to find the best implementation form of the functions and the interaction with the users, both operative research and participatory development of the software tool should be considered. This explains why the concept development and testing of the software tool has been framed in the research project as a series of design inclusive research cycles. Participatory research methods were used both in the explorative and in the confirmative stages in multiple cycles. Below, we give a concise overview of how stakeholder-specific MAPs were developed and then used in the confirmative stage of a design inclusive research cycle. The development of the MAPs commenced with an information synthesis that involved information about the concept of the software tool, the foreseen end-users, and the assumed use environment. In addition, context information was aggregated concerning these constituents, as well as about the stakeholders. On the basis of these chunks of information, the so-called demonstration contents of the MAPs have been defined. Recall that the demonstration contents are structured and built into the abstract prototype modules through using various media resources. The demonstration content is structured so as to show the whole and the constituents of the foreseen process, i.e., the conceptualized functions and implementation of the software, the end-users as personas, the actions of the personas and their interactions with the software, and the surrounding environment (including its constituting entities, their characteristics and relationships). The demonstration contents have been dissected into various chunks based on the consideration of three categories of stakeholders: (i) designers of household appliances, (ii) software programmers, and (iii) knowledge engineers. The decision on the number of the modules, the contents of the narration, and the forms of the enactments was informed by the context information related to the stakeholders and their interests. The narrations, as synthetic descriptions of the outline and the highlights of the foreseen real-life process, have been implemented by using animated text, human voice, and synthetic sounds. The enactments were designed so as to visualize all significant arrangements, happenings and contributors of the process. The narrations are composed of logical units, called episodes. These were edited as text, and, within each episode, some important key terms and phrases have been identified. They were used as links to, or anchors of the parts of the enactments (called segments). The segments were implemented by using different visualization media (modalities ranging from animated hand sketches to live movie streams) and were included in the enactments in the sequence indicated by the anchors. A smooth transition between them was sought for. As media representation for the contents of the MAPs, we decided to use

collaboration of the participating experts is sub-optimal, and documentation and sharing the walkthrough results among the participants is not supported sufficiently. Software testing is concerned with exercising and observing product behavior (dynamic verification). It is typically not done just in general, but with a particular objective in mind. These objectives can be: (i) functional, (ii) compatibility, (iii) robustness/sensitivity, (iv) computational performance, (v) interfacing, (vi) output, (vii) usability, (viii) security, and (ix) connectivity testing. The software (system) is executed with test data and its operational behavior is observed. Typically two major stages can be differentiated, which are called alpha-testing and beta-testing. Alpha-testing happens within the software development company, mainly completed by the developers with the involvement of representatives (test groups) of users. Beta-testing happens outside the developer company and is made by specific groups of the targeted endusers and software application engineers in real-life use environments. The objective of alpha-testing is to test the fulfillment of the implemented functions and check the stability on test data, while beta-testing provides information on the operation and possible failures of the software under real life circumstances with real users and data. As the objectives show, these two testing cycles serve corrective purposes, rather than preventive purposes. However, the earlier a defect is found, the cheaper is to fix it. 6. USING MODULAR ABSTRACT PROTOTYPES AS RESEARCH MEANS IN DESIGN INCLUSIVE RESEARCH As explained above, our research focused on a structured and participatory development of a software tool for forecasting financial trade-off when ubiquitous controllers are used to support energy saving. This is a frequently occurring task in designing household appliances, but non-trivial, because several technical and intangible factors have to be taken into consideration by product designers. For instance, to forecast the financial balance at ubiquitous augmentation, on the one hand: (i) the production costs of the existing product, (ii) the energy consumption costs of the existing product, (iii) the highest possible selling price of the existing product, and (iv) the longterm use behavior of the end-users, and on the other hand: (v) the production costs of the new product, (vi) the costs of the installed ubiquitous controllers, (vii) the energy consumption of costs of the augmented product, (viii) the additional energy consumption costs of the installed ubiquitous controllers, and (ix) the long-term gains of energy saving by the installed ubiquitous controllers, (x) the highest possible selling price of the augmented product, and (xi) a different long-term use behavior of the end-users of the augmented products need to be taken into consideration. Actually, the software should inform the designers of household appliances how find balance between these two sides when ubiquitous augmentation is applied. To support the designers, a case-based reasoning mechanism will be included

8


stakeholders in the demonstration session was provided. That is, it was possible for the participants to make immediate reflections, to pose interrupting questions and to follow up with a discussion. The working principle of the tool was demonstrated for each of the software end-user, software designer, and knowledge engineer stakeholder groups, but they also received specific information about the plan of programming and computations, and the knowledge processing mechanism, respectively. The discussions with the different groups of stakeholders were recorded, and, after coding, data were elicited from these by both quantitative and qualitative analyses. Data reduction techniques were applied to rank the data. The obtained working data were processed and converted into information about how to change, or further improve and optimize the demonstrated software concept. The proposed changes in the software were evaluated for feasibility. The theory of theoretical saturation was used to define the number of subsequent focus group sessions needed to collect sufficient data and knowledge. Data saturation was considered to have been reached when no significant new information emerged in a focus group session. That explains why four focus group sessions were held with the involvement of household electronic appliance designers. As discussed above, the whole process consisted of four phases. The first phase concentrated on the aggregation of technical information about the software concept for MAP development. The process started with an investigation of information available for the software concept selected for demonstration. There were two major goals here, namely (i) making the specifications as complete as possible, and (ii) clarifying the context of prototyping. The second phase involved the compilation and testing of the technical contents for the individual modules of the abstract prototype. The third phase dealt with the demonstration of the abstract prototypes, assembled from multiple different modules, to the stakeholders. The specific objective of this phase was to gather the stakeholders’ opinion about the functionality, interfacing and the expected performance of the demonstrated software concept. In the last phase, the information obtained from the stakeholders was coded and assessed and possible refinements of the software concept were identified.

Figure 6 Examples of linking the elements of the narration and the enactment parts of the MAPs a mixed-modality digital recording, in which, different forms of visual representations and animations could be included depending on the objectives and contents of the modules. Our striving after a perceptive and cognitive linking resulted in a high level synergy between the narrations and the enactments, both within and between the modules of the abstract prototypes (Figure 6). The modules have been tested in a scaled-down, but real-life pilot session, before they were used in the focus groups. Therefore, they could be optimized for informing the stakeholders, to underpin their opinion forming and to facilitate a well-informed decision making. Another stream of activities concerned the planning, setting up and completing the demonstration sessions. As mentioned above, they were designed and conducted as focus group sessions with on-site or videoconferencing-based participation. The stakeholders participating in the focus groups were recruited based on stratified systematic sampling. The opportunity of an active (interactive) participation of the

7. DISCUSSION, CONCLUSIONS AND FUTURE RESEARCH OPPORTUNITIES In general all stakeholders were positive about the use of modular abstract prototypes in the focus group sessions of the confirmative phase of the discussed research cycle. They argued that the major advantage of the proposed method was that a lot of information could be presented in an immersive manner in a relatively short period of time. They also thought that the proposed methodology is appropriate and useful for not only software products, but also for other complex products and product-service combinations, under the condition that the effort to make the modular abstract prototype is worth. Another

9


are provided to each of the stakeholders. It seems that the optimal number of modules (i.e. the resolution of the AP) is very difficult to be defined formally, but future research may explore certain formal rules and/or principles. Currently, it is defined based on the number of different stakeholders, who are involved in the software concept assessment process, and of the topics the stakeholders are interested in, but it may also be influenced by different other principles. It has been found that MAPs are well applicable as demonstration means in the case of interrogative research sessions, such as on-site focus group sessions and dislocated expert interrogation sessions. In addition to flexibility, modular development of abstract prototypes also increases productivity because different modules may be developed by different researchers and designers. MAPs must be carefully designed in context not only in terms of their cognitive contents and the presentation media, but also in terms of the internal structure of the modules and the architecture of the modules. The motivation for our research came from the observation of the lack of innovative and conceptual research means that can be applied in a flexible and adaptive manner to various research cases. Our experiences obtained in the conducted focus group sessions encourage us to claim that MAP is an efficient method to present concepts of software products to multiple stakeholders in an early phase of their development. On the other hand, we are fully aware of the fact that this research area requires more attention and exploration. It needs the integration of knowledge at least of four domains, namely, informing science, scientific prototyping in context, design research methodology, and participatory software development. Our observation is that the academia is lagging behind clarifying what the proper uses and expectable benefits of non-traditional early prototyping are, and the industry is yet hardly coping with these problems. The proposed methodological approach is capable to: (i) provide reasonable level of flexibility in demonstrative prototyping and concept assessment by stakeholders, (ii) encapsulate all information constructs that are needed and sufficient to frame the required information, and (iii) offer an easy-to-apply procedure for the development and applications of rich modular abstract prototypes. A demonstration session should be decomposed to a series of sub-sessions, in which the dedicated MAP contents can be presented to the different stakeholders. This way, the information overload of the stakeholders can be reduced and the development of the demonstration material can be more efficient and flexible. In general, the major benefits of using modular abstract prototyping in the context of software conceptualization and design are that it (i) influences the most creative phases of software innovation and development, and (ii) opens the way towards intelligence that cannot be obtained otherwise. The major deficits are that it needs not only extra efforts and knowledge, but also a comprehensive thinking and a systematic way of working. Due to: (i) the required extra efforts and resources, (ii) the vagueness and incompleteness of knowledge

advantages mentioned by the participants was the achieved reallife fidelity and clarity. Nevertheless, some drawbacks of the MAP approach were also put forward. Some participants remarked that, based on the sophisticated demonstration, it was easy to imagine how the software works, and how can it be used, but there is a chance that their feeling may differ when they are really using it in the case of real life design problems. Therefore, they proposed to follow up with research sessions with a testing of a surrogate prototype, or a fully-fledged functional prototype, in the next phase of the software development. Many of the participants also mentioned that much information was presented and therefore they had to make efforts to maintain their concentration and pay attention selectively during the demonstration parts. For some of them it proved to be difficult to recall everything precisely and correctly during the discussion parts of the sessions. Some suggested including more examples when complex functional structures, information flows, and interaction procedures are presented, to lower the level of abstractness in thinking. In addition to the analysis of the advantages and disadvantages of the MAP methodology, the designers participating in the focus group sessions also gave advice on other stakeholders who they believed needed to be involved. They proposed that the software developers as stakeholders require completely different set of information and perhaps other enactments for presenting functional and use information. They actually said that software developers think differently, namely in algorithms and codes. The end-users proposed to include additional groups of stakeholders, i.e. usability experts. The knowledge engineer stakeholders found it important to include experts from the field of designing ubiquitous controllers and also digital energy saving experts in particular in the time of the optimization of the trade-off calculation mechanism (algorithm) and when the content of the case base is prepared and filled in. We see the important elements of our work as follows: The research objective was to develop an early prototyping approach that can be used to demonstrate software concepts to the involved stakeholders in a rapid and flexible way in the framework of design inclusive research. Towards this end, the methodology of modular abstract prototyping has been worked out and tested in a number of applications. A modular abstract prototype has been conceptualized as a digitally recorded, multi-media enabled information structure, which is able to represent the operation and use processes in the case of various real-life tasks, users and environments. In the context of design inclusive research, modular abstract prototypes manifest as a research means that can be reconfigured and adapted rapidly in terms of its contents, depending on the viewpoint, knowledge and demands of the stakeholders. MAPs make informing of subjects in research processes more efficient and, for this reason, lead to more complete and reliable research data. We propose that a specific combination of the modules should ensure that the optimal amount and pieces of information

10


[8]

available for conceptualization, and (iii) the possible concerns about the trade-off of the investments and the professional gains, industrial software developers need more evidence about the effectiveness and usefulness of abstract prototyping. On the other hand, APs can efficiently support intense non-tangible communication with and demonstrations for stakeholders. Our understanding is that the return of the investments in abstract prototyping should be expected in the downstream phases and in the better quality of the software concepts, rather than in saving time, costs and efforts in the ideation and conceptualization phases. Abstract prototyping is targeting not only end users as stakeholders, but all those who can contribute to a correct and integral definition of the requirements, architectures, information flows, and data management constructs. However, we consider MAP just as a milestone towards a fully interactive abstract prototyping (IAP). Our current followup research is focusing on the exploration of possibilities of a highly interactive real-time abstract prototyping by using IAP skeletons and web-hosted object and knowledge repositories to support prototype development. We are also doing research to explore new application domains and application contexts where intensity, uniformity and efficiency of informing are critical and physical demonstration means cannot be considered due to the nature of the investigated systems, the long implementation and modification times, and the lack of demonstrative power and immersion. We are paying specific attention to early interactive abstract prototyping of complex, distributed and smart cyber physical systems with collaborating autonomous agents and ad-hoc internal connectivity.

[9] [10]

[11] [12]

[13]

[14]

[15]

[16]

[17]

REFERENCES [1] Neill, C. J. and Laplante, P. A., 2003, Requirements engineering: The state of the practice, IEEE Software, Vol. 20, No. 6, pp. 40-45. [2] Gulliksen, J., Göransson, B., Boivie, I., Blomkvist, S., Persson, J. and Cajander, A., 2003, Key principles for user-centred systems design, Behaviour and Information Technology, Vol. 22, No. 6, pp. 397-409. [3] Lee, J. and Xue, N. L., 1999, Analyzing user requirements by use cases: A goal-driven approach, IEEE Software, Vol. 16, No. 4, pp. 92-101. [4] Constantine, L. L. and Lockwood, L. A., 2011, Software for use: A practical guide to the models and methods of usage-centered design. Addison-Wesley, Reading, MA, pp. 1-608. [5] Cohen, E. B., 1999, Reconceptualizing information systems as a field of the transdiscipline informing science: From ugly duckling to swan, Journal of Computing and Information Technology, Vol. 7, No. 3, pp. 213-219. [6] Cohen, E. B., 2009, A philosophy of informing science, Informing Science, Vol. 12, pp. 1-15. [7] Gackowski, Z. J., 2010, Informing as a discipline: An initial proposal, Informing Science, Vol. 13, pp. 165-175.

[18]

[19]

[20]

[21]

[22]

[22] [23]

11

Sweller, J., 2003, Evolution of human cognitive architecture, in The psychology of learning and motivation, Ross, B. (Ed.), Vol. 43, Academic Press, San Diego, pp. 215-266. Kalyuga, S., 2011, Informing: A cognitive load perspective, Informing Science, Vol. 14, pp. 33-45. Kumpulainen, K. and Kaartinen, S., 2003, The interpersonal dynamics of collaborative reasoning in peer interactive dyads, The Journal of Experimental Education, Vol. 71, No. 4, pp. 333-370. Wilson, T. D., 2000, Human information behavior, Informing Science, Vol. 3, No. 2, pp. 49-56. Gill, T. G., 2008, The single client resonance model: Beyond rigor and relevance, Informing Science, Vol. 11, pp. 281-310. Perry, W., Signori, D. and Boon, J., 2004, Exploring information superiority: A methodology for measuring the quality of information and its impact on shared awareness, RAND Corporation, Santa Monica, CA, pp. 1-141. Gill, G. and Bhattacherjee, A., 2009, Whom are we informing? Issues and recommendations for MIS research from an informing sciences perspective, MIS Quarterly, Vol. 33, No. 2, pp. 217-235. Stahl, B. C., 2006, On the difference or equality of information, misinformation, and disinformation: A critical research perspective, Informing Science, Vol. 9, pp. 83-96. Koh, C. E., Prybutok, V. R., Ryan, S. D. and Wu, Y., 2010, A model for mandatory use of software technologies: An integrative approach by applying multiple levels of abstraction of informing science, Informing Science, Vol. 13, pp. 177-203. Lang, M., 2003, Communicating academic research findings to IS professionals: An analysis of problems, Informing Science, Vol. 6, pp. 21-19. Walton, D. N. and Krabbe, E. C., 1995, Basic concepts of interpersonal reasoning, State University of New York Press, pp. 1-223. Giddens, A., 1991, Structuration theory: Past, present and future, in Giddens’ Theory of Structuration: A Critical Appreciation, Bryant, C. G. and Jary, D. (eds.), Routledge, London, pp. 201-221. Jones, M. R. and Karsten, H., 2008, Giddens’s structuration theory and IS research, MIS Quarterly, Vol. 32 No. 1, pp. 127-157. Bayazit, N., 2004, Investigating design: A review of forty years of design research, Design Issues, Vol. 20, No. 1, pp. 16-29. Sevaldson, B., 2010, Discussions and movements in design research: A systems approach to practice research in design, FORMakademisk, Vol. 3, No. 1, pp. 8-35. Pedgley, O., 2007, Capturing and analysing own design activity, Design Studies, Vol. 28, No. 5, pp. 463-483. Horváth, I. and Duhovnik, J., 2005, Towards a better understanding of the methodological characteristics of engineering design research, in Proceedings of ASME


[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

2005 International Design Engineering Technical Conferences - IDETC/CIE 2005, Long Beach, CA, IDETC2005/85715, pp. 1-15. Stapleton, A. J., 2005, Research as design - Design as research, in Proceedings of DiGRA 2005 Conference: Changing Views - Worlds in Play, pp. 1-7. Zimmerman, J., Forlizzi, J. and Evenson, S., 2007, Research through design as a method for interaction design research in HCI, in Proceedings of CHI 2007, San Jose, CA, pp. 493-502. Zimmerman, J. and Forlizzi, J., 2008, The role of design artifacts in design theory construction. Artifact, Vol. 2, No. 1, pp. 41-45. van den Hoven, E., Frens, J., Aliakseyeu, D., Martens, J.B., Overbeeke, K. and Peters, P., 2007, Design research and tangible interaction, in Proceedings of TEI '07, Baton Rouge, LA, pp. 109-115. Sandoval, W. A. and Bell, P., 2004, Design-based research methods for studying learning in context: Introduction, Educational Psychologist, Vol. 39, No. 4, pp. 199-201. Stappers, P. J., 2007, Doing design as a part of doing research, in Design research now: Essays and selected projects, by Michel, R. (ed.), Birkhäuser Verlag, Basel, pp. 81-97. Horváth, I., 2007, Comparison of three methodological approaches of design research, in Proceedings of the 16th International Conference on Engineering Design - ICED '07, Ecole Central Paris, Paris, pp. 1-11. Horváth, I., 2008, Differences between ‘research in design context’ and ‘design inclusive research’ in the domain of industrial design engineering, Journal of Design Research, Vol. 7, No. 1, pp. 61-83. Zimmerman, J., Stolterman, E. and Forlizzi, J., 2010, An analysis and critique of research through design: Towards a formalization of a research approach, in Proceedings of DIS 2010, Aarhus, Denmark, pp. 310-319. Polson, P. O., Lewis, C., Rieman, J. and Wharton, C., 1992, Cognitive walkthroughs: A method for theory-based evaluation of user interfaces, International Journal of Man-Machine Studies, Vol. 3, pp. 741-773. Horváth, I., 2011, Theoretical framework for comprehensive abstract prototyping methodology, in Proceedings of the International Conference on Engineering Design - ICED’11, 15-18 August 2011, T. U. of Denmark, Kobenhaven, pp. 1-12. Kordon, F. and Luqi, 2002, An introduction to rapid system prototyping, IEEE Transaction on Software Engineering, Vol. 28, No. 9, pp. 817-821. Ulrich, W., 2001, A philosophical staircase for information systems definition, design and development: A discursive approach to reflective practice in ISD (Part 1), The Journal of Information Technology Theory and Application, Vol. 3, No. 3, pp. 55-84. Horváth, I., Rusák, Z., van der Vegte, W., Koojman, A., Opiyo, E. Z. and Peck, D., 2010, An information

[38]

[39]

[40]

[41]

[42]

12

technological specification of abstract prototyping for artifact and service combinations, in Proceedings of the ASME 2011 International Design Engineering Technical Conferences - IDETC/CIE 2011, August 29-31, 2011, Washington D.C., DTM-47079, pp. 1-14. Kordon, F. and Henkel, J., 2003, An overview of rapid system prototyping today, Kluwer Academic Publishers. Amsterdam, pp. 1-9. Dix, K. L., 2007, DBRIEF: A research paradigm for ICT adoption, International Education Journal, Vol. 8, No. 2, pp. 113-124. Medvidovic, N. and Taylor, R., 2000, A classification and comparison framework for software architecture description languages, Software Engineering, Vol. 26, No. 1, pp. 70-93. Leveson, N., 1997, Software engineering: Stretching the limits of complexity, Communications of the ACM, Vol. 40, No. 2, pp. 129-131. Horváth, I., Rusák, Z., Vegte, W. v.d., Opiyo, E.Z., and Koojman, A., (2011), Demonstration and assessment of innovation: abstract prototyping of artifact and service combinations, accepted for publication in Journal of Computing and Information Science in Engineering.
