248
Int. J. Simulation and Process Modelling, Vol.
Challenges in collaborative modelling: a literature review and research agenda Michiel Renger and Gwendolyn L. Kolfschoten* Faculty of Technology Policy and Management, Department of Systems Engineering, Delft University of Technology, Jaffalaan 5, 2628 BX Delft, The Netherlands E-mail:
[email protected] E-mail:
[email protected] *Corresponding author
Gert-Jan de Vreede Center for Collaboration Science, University of Nebraska at Omaha, 1110 South 67th Street, Omaha, NE 68182, USA E-mail:
[email protected] Abstract: Modelling is a key activity in conceptual design and system design. Through collaborative modelling, end-users, stakeholders, experts and entrepreneurs are able to create shared understanding about a system representation. This paper reports on a literature review to provide an overview of the commonalities and differences in collaborative modelling research. We develop a framework of analysis and conduct a quantitative meta-analysis to understand the challenges in collaborative modelling. Next, we performed a qualitative analysis to further elicit these challenges and discuss requirements for tools to support collaborative modelling, using Group Support Systems (GSSs) or Interactive Whiteboards (IWBs). We found critical challenges in supporting the understanding and adoption of modelling rules, and in integration of sub-models and different perspectives of systems to achieve shared understanding visualised in a system representation. Keywords: modelling; collaborative modelling; conceptual design; systems design; team modelling; group modelling. Reference to this paper should be made as follows: Renger, M., Kolfschoten, G.L. and de Vreede, G.J. (2008) ‘Challenges in collaborative modelling: a literature review and research agenda’, Int. J. Simulation and Process Modelling, Vol. Biographical notes: Michiel Renger is a PhD student at the Delft University of Technology in The Netherlands. He researches tools and facilitation techniques to support collaborative modelling. His work was presented at the GDN, CRIWG and EOMAS conferences, of which the last received the SIGMAS Best Paper Award ’08. Gwendolyn L. Kolfschoten is an Assistant Professor at Delft University of Technology in the Netherlands. She is an experienced facilitator of thinkLets-based Group Support Systems workshop having worked with numerous public and private organisations. Her research focuses on the quality of thinkLet-based collaboration process design for complex tasks. She developed the first example of Computer-Supported Collaboration Engineering (CACE) technology – an integrated support suite to assist collaboration engineers in process design. Her research has been presented at HICSS and CRIWG, AMCIS, Engineering Education, EuroPlop and GDN conferences and has been published in International Journal of Computer Applications in Technology, International Journal of Human-Computer Studies and Group Decision and Negotiation. Gert-Jan de Vreede is the Kayser Distinguished Professor at the Department of Information Systems and Quantitative Analysis at the University of Nebraska at Omaha where he is director of the Center for Collaboration Science. He is also affiliated with the Faculty of Technology, Policy and Management of Delft University of Technology in the Netherlands from where he received his PhD. His research focuses on the application, adoption, and diffusion of collaboration technology in organisations, the development of repeatable collaboration processes, facilitation of group meetings, and the application of collaboration technology in different
Copyright © 2008 Inderscience Enterprises Ltd.
Challenges in collaborative modelling: a literature review and research agenda
249
sociocultural environments. His papers have appeared in various journals, including Journal of Management Information Systems, Journal of Decision Systems, Communications of the ACM, Journal of Creativity and Innovation Management, DataBase, Group Decision and Negotiation, Simulation and Journal of Simulation Practice and Theory.
1
Introduction
Modelling is a key activity in conceptual design and system design, and a crucial step in the development of simulation models (Barjis, 2008; Brooks, 2006; Robinson, 2008). There is broad agreement that it is important to involve various experts, stakeholders and users in a development cycle (Boehm et al., 2001; Fruhling and de Vreede, 2006; Standish Group, 1995). While these parties are often interviewed or in other ways heard, they often lack the skills to actively participate in the modelling effort. If users are not involved in systems analysis tasks, their problems, solutions and ideas are difficult to communicate to the analyst. This often results in poor requirements definition, which is the leading cause for failed IT projects (Boehm et al., 2001). Further, analysts and entrepreneurs might have mental models, visions of a solution or system design, but might lack the adequate means of articulating these in terms familiar to all stakeholders involved (Hill and Levenhagen, 1995). While there are means to verbally explain models, such as metaphors, a graphical representation is often more effective. (“A picture tells more than a thousand words”). To use graphical representations as a basis for discussion, it would be useful if all the stakeholders can be actively engaged in the construction and modification of such models. With increasing complexity of systems and organisations, creating shared understanding and joint representations of those systems becomes increasingly important. Analytical skills become more wanted and more important to function in these complex contexts. However, creating one’s own system representation is in many ways different from creating a joint system representation. With the increasing need for collaboration among experts and knowledge workers (Frost and Sullivan, 2007), collaborative modelling becomes increasingly important. While meta-analysis has been performed to gain insight in metrics and effects in collaborative modelling (Rouwette et al., 2002), an overview of methods and role deviations has been described (Andersen and Richardson, 1997; Richardson and Andersen, 1995), to our knowledge there is no overview of challenges and best practices in collaborative modelling. Such overview would help us to find opportunities for research and for the design of new supporting tools and methods to empower simulation experts and modelling annalists to support effective and efficient collaborative modelling. In this paper, we provide an overview on collaborative modelling studies that brings together the experiences and findings from literature to identify the main challenges and lessons learned in the field. This will lead us to create a first and preliminary research agenda for new and innovative
collaborative modelling support systems and methods. The paper first defines collaborative modelling. Next, we describe the different schools in collaborative modelling. Third, we discuss the research method for the literature review, followed by the results in which we describe critical challenges and solutions for successful collaborative modelling. From this overview, we will derive an initial overview of functionalities and techniques to support collaborative modelling. We end with conclusions and suggestions for further research.
2
Collaborative modelling
For the purpose of the research presented in this paper, we define collaborative modelling as: The joint creation of a shared graphical representation of a system. In this definition, we focus on graphical modelling of systems as opposed to physical modelling of objects or artefacts such as in architecture and industrial design. Graphical models are usually created in a conceptual design phase either for analysis or for design. They are used to communicate a representation of a system to understand or change it. Conceptual models are used in an early phase of analysis and design and therefore are initially associated with a sketching activity. However, when they are used as a basis for design or structural analysis, they need to meet various requirements with respect to precision and rigor. They may also need to be translated to computer models to the effects of the model. For this purpose, modelling languages have been developed to capture conceptual models as computer models to enable easy manipulation and automatic syntactic verifications. Using computer models also makes it easier to make changes in a model, especially when changes in one component result in changes in other components and relations between components. Further, we focus on joint creation to indicate our interest in stakeholder participation in the modelling effort as opposed to modelling by external professionals or analysts only. Joint creation requires the exchange of perspectives among the participants. The model is a way to elicit, highlight, and communicate different perspectives and assumptions among group members. To create a shared representation as opposed to an individual representation, a shared understanding of the elements and relations in the model needs to be created. Shared understanding can be defined as “the overlap of understanding and concepts among group members” (Mulder et al., 2002, p.36). We build on this definition for the collaborative modelling domain where we define shared understanding as the extent to which specific knowledge
250
M. Renger et al.
among group members of concepts representing system elements and their relations overlaps. To create overlap in knowledge, participants need not only to share information about model elements and relations, but also to create shared meaning with respect to these elements and their relations. Creating shared meaning is often studied from a ‘sensemaking’ perspective. Sensemaking is described by Weick as involving “the ongoing retrospective development of plausible images that rationalise what people are doing” (Weick, 1995, p.409). Sensemaking usually requires some development of shared meaning of concepts, labels, and terms. It also includes the development of a common understanding of context and the perspective of different stakeholders with respect to the model. Collaborative modelling as a field of both practice and research has developed over the last decades. Within the field of system dynamics, modellers started to involve client groups in the modelling process since the late 1970s (Vennix et al., 1992). Since that time, various other modelling schools have adopted the notion of collaborative modelling and found approaches to involve stakeholders in their own modelling efforts, see e.g., Andersen and Richardson (1997), Dean et al. (2000) and Shaw et al. (2003). As a result, various research groups performed field studies, gained experience, and eventually developed sophisticated methods for modelling efforts that have high levels of stakeholder participation. However, different modelling languages are associated with different methods and approaches for analysis and design. To accommodate different stakeholder groups in these methods, new approaches had to be developed leading to different schools and eventually different patterns in collaborative modelling. Here, we describe the most important schools in collaborative modelling.
2.1 Problem Structuring Methods Problem Structuring Methods (PSMs) refer to a broad variety of methods and tools that have developed mainly in the UK to cope with complexity, uncertainty and conflict (Rosenhead, 1993). These methods have the following similarities in approach: use of a model as a transitional object, increasing the overall productivity of group processes, attention to the facilitation of effective group processes, and significance of facilitation skills in enabling effective model building (Eden and Ackermann, 2006). Especially, the first of these is characteristic for PSMs: models are seen as instrumental to strategic decision-making and problem solving in complex settings, so the approach typically focuses on the overall decision-making process, which often includes simulation for scenario explorations. The most important PSMs are: •
Soft Systems Methodology (SSM)
•
Strategic Choice (SC)
•
Strategic Options Development and Analysis (SODA) and JOintly Understanding Reflecting and Negotiating strategY (JOURNEY Making).
In PSMs, a group’s shared understanding is created by switching between the views of individual participants and the entire group, and focus on the differences in view to resolve them (Ackermann and Eden, 2005; Ackermann et al., 2005).
2.2 Group Model Building Group Model Building is considered a special case of PSMs for hard modelling, and has been developed by researchers of the University at Albany in New York, and the University of Nijmegen in the Netherlands. According to Andersen et al., Group Model Building refers to “a bundle of techniques used to construct system dynamics models working directly with client groups on key strategic decisions.” (Andersen et al., 2007, p.1)
Group Model Building always has a system dynamics approach, and usually extends the conceptual model to simulation models to explore diverse strategic options. Within this field, three types of cognitive tasks are distinguished: divergent thinking, convergent thinking, and evaluation (Rouwette et al., 2000). Best results are found if the divergent thinking task is carried without too much interaction among group members, e.g., in the so-called nominal group setting. The convergent thinking task is believed to require the input of the group as a whole. The flexible outlines of the method, the so-called scripts, are presented in Andersen and Richardson (1997). On the basis of these scripts is the more applied and specific approach called Mediated Modelling, which is developed mainly for complex problems of ecological nature. As the name suggests, Mediated Modelling focuses more on conflict handling tasks. A mediated modeller should, therefore, represent the role of mediator as well as the roles of facilitator and modeller (van den Belt, 2004). Typically, because of the complex nature of ecological management problems, Mediated Modelling involves large groups of stakeholders (up to 100) in lengthy projects that may take more than one year. Shared Vision Modelling provides a similar approach to handle water resource management problems (Lund and Palmer, 1997).
2.3 Enterprise analysis Collaborative modelling research at the University of Arizona has a relatively stronger focus on the development of software tools as well as facilitation techniques for the support of collaborative modelling efforts (Morton et al., 2003; Dean et al., 1994). This approach concentrates more on collaboratively built models as a goal in itself rather than as a transitional object. The models to be built are often of the IDEF0 type, and many techniques are
Challenges in collaborative modelling: a literature review and research agenda especially meant to deal with IDEF0-standard models (Dennis et al., 1994, 1999). Typically, collaborative modelling sessions start with the creation of an initial model with the plenary group of stakeholders. The whole group is subsequently split up into subgroups to work out the details of the model parts that correspond to the subgroups’ expert domain.
2.4 Other approaches Apart from the above-described schools of thought, we also found the terms Participatory Modelling, which is loosely used for collaborative modelling across different schools, and Companion Modelling (ComMod), which uses multi-agent systems or role playing games, to elicit information needed to construct the model (Gurung et al., 2006). Since stakeholders are not directly involved in the companion model building process, the model is not jointly built, and we do not consider Companion Modelling a form of collaborative modelling.
3
Framework of analysis
Fjermestad and Hiltz (Fjermestad, 1998) consolidated various previously developed frameworks into a general framework that was used for a comparative analysis of studies on groups using a Group Support Systems (GSSs) for a variety of tasks. Another well-known descriptive model of GSS research is the model of Nunamaker et al. (1991, 1997). Although collaborative modelling may not always involve GSS, both frameworks offer a basis to study technology-supported collaborative efforts. On the basis of these models, we developed our collaborative modelling framework (see Figure 1) that distinguishes between task factors and group factors, which together describe the context in which the collaborative modelling effort takes place. The next construct in our framework concerns the intervention factors, which describe the intervention to structure and coordinate the collaborative modelling effort. Finally, the framework includes outcome factors that describe the effects and results of a collaborative modelling effort. Figure 1
Factors in the research framework (see online version for colours)
The remainder of this section defines the factors in our framework in more detail and explains their relevance
251
to achieving a comprehensive understanding of the collaborative modelling research literature.
3.1 Group factors The group is the key resource in the collaborative effort. Without the input from the group, the model cannot be created. However, due to different stakes and perspectives, a group can also thwart the success of the modelling effort. Group size: The number of participants that were involved in the modelling effort. Group size is of particular importance in collaboration. When group size increases, productivity tends to decrease, and conflict tends to increase (Steiner, 1972). In group model building, it was also found that a larger group poses limitations on effectiveness (Rouwette et al., 2000). Group composition: The type of participants and their skills and stakes. The group composition reflects the stakes and skills in the process. When important skills are not present, the group will lack knowledge required to accomplish the modelling task. When important stakes are not represented, the results of the modelling process may not be supported after the session. Dean et al. (1994) note that the presence of both process designers and process owners is important in process analysis. The following group composition characteristics will be determined for each of the studies included in our analysis: •
the culture and country of the participants
•
the type of participants or subjects, i.e., whether they are students or professionals and what their educational background is
•
the stakes and reasons for involvement
•
the modelling expertise of the participants, i.e., to what extent they were already skilled modellers, and whether they received any modelling training in advance.
3.2 Task factors The modelling effort, especially when performed in a collaborative setting, should serve a particular purpose. The purpose can be to answer questions with respect to the system or process to be modelled, but it can also be a more tacit goal of creating shared understanding about the system and the problems or conflicts that occur. The domain of the modelling effort provides an indication of the characteristics of tasks and scopes in which a collaborative modelling effort is useful. This can help to identify the scope in which collaborative modelling is applicable and useful. Further, to explore the productivity of particular collaborative modelling approaches, it is useful to record the time spend on the collaborative modelling effort. There are three task factors included in our analysis: •
Model goal: the desired outcome of the modelling effort
252
M. Renger et al.
•
Domain: the type of context or sector in which the modelling effort takes place
•
Session time: the total time of the collaborative effort to create the model.
3.3 Intervention factors Interventions concern the rules, tools and instructions provided to the group to guide and support their collaborative modelling effort. The modelling language used determines (and limits) what can be represented and how it is represented. For example, a process model does not enable the representation of roles and an object model does not enable representation of sequential relations. Different model types will also result in different complexity and different perspectives on the system. As collaborative modelling efforts are geared towards enhancing understanding of a system among all involved stakeholders, it is interesting to understand which perspectives and model types are used in collaborative modelling efforts. Besides the modelling language, the use of roles is an important intervention factor. Richardson and Andersen (1995) described five essential roles that should be present in a group model building session: the facilitator, modeller/reflector, process coach, recorder and gatekeeper. Roles can be allocated to different people in the group, which can effect the workload of those participants and therewith the effectiveness and the efficiency of the team. Therefore, we are interested in the roles that were defined for the session and the people they were assigned to. To create a model, the group needs to perform several activities. To support the group in their modelling effort, an intervention can structure and sequence these activities to create a logic and efficient process to create the model. Some of these approaches are formalised in a modelling method. In a modelling method, the effort of individual participants can be coordinated with the use of rules and instructions. Key purposes of these rules and instructions include, for example, to support parallel work, to ensure objectivity through anonymity, to create change awareness, and to control access to capabilities that enable modification of the model. Access control is critical, as it determines the ability of participants to actively manipulate the model. Nunamaker et al. (1991) describe two styles of participation: interactive, where all can modify, and chauffeured, where participants can only modify through a recorder. To understand the interventions made in collaborative modelling, we will document the following factors: •
Model type: the type of model that is to be build, determined by the formal grammar of the modelling language
•
Roles: the assignment of particular responsibilities within the modelling task to specific individuals, serving as a basis to coordinate the collaborative modelling effort
•
Modelling approach: the structure and sequence of activities used by the group to create the model.
3.4 Outcome factors Outcome factors describe the qualities or perceptions on the results of the collaborative task. Using these factors, we can evaluate the success of the collaborative modelling effort. One of the key purposes of a modelling effort is to gain insight and overview of the system that is represented. Different stakeholders might have different perspectives on the system. The outcomes relevant for a specific modelling effort depend on the goal of the modelling effort; for instance, when the goal of a modelling effort is to demarcate a system boundary, quality criteria are different than when a conceptual model as a basis for simulation is build. The outcome factors that we recorded are: •
Effectiveness: Quality or validity of the resulting model compared with the intended quality (in ’t Veld, 1987).
•
Efficiency: The difference in real resource expense compared with the intended resource expense (in ’t Veld, 1987).
•
Satisfaction: Affective positive arousal towards the process and the results (Briggs et al., 2006b).
•
Consensus: A state in which all stakeholders are willing to commit to a proposal (Briggs et al., 2006a).
•
Shared understanding: The extent to which specific knowledge among group members of concepts and relations in a system overlaps (Mulder et al., 2002).
•
Model complexity: The number of concepts and relations that compose the model (Wood, 1986). Metrics for complexity are mostly quantitative, but can also be subjective, for example describing difficulty to understand. For some modelling languages, formal complexity metrics are defined (Erickson and Siau, 2007).
4
Method
Using the framework, we analysed papers in which the object of study consisted of one or more collaborative modelling efforts. We selected published articles that described a case study in which the deliverable was a graphical conceptual model of a system, per our definition of collaborative modelling. We did not discriminate among research methods or approaches (Creswell, 1994; Morton et al., 2003; Trauth and Jessup, 2000). We searched for papers in which the modelling approach was the central topic. When modelling cases were found in a domain-focused context, we included it, but we did not search domain-specific journals for modelling cases. To identify papers, we searched in various research databases such as Google Scholar, Elsevier’s Scopus, IEEE Explorer, the ACM Portal and Science Direct. Search
Challenges in collaborative modelling: a literature review and research agenda keywords included collaborative modelling/modelling, participatory modelling, group model building, shared vision modelling and mediated modelling. Further, we searched for cases of collaborative modelling within the context of the related subjects such as: facilitation, G(D)SS, Collaboration Engineering, (information) systems and software engineering, business process modelling, collaborative design, and collaborative learning. In the papers that we found, we searched through the references for additional sources, and we look at papers that cited the papers we found. Some approaches typically focus on a larger process in which collaborative modelling is embedded. For example, in many cases of Group Model Building and Mediated Modelling a conceptual graphical model is built, quantified, and simulated with specific simulation software to explore possible future scenarios. Although simulations and scenario explorations can be very useful, we do not consider these phases part of collaborative modelling effort. In the framework, we therefore concentrated on the building of conceptual models only. For each paper, we identified the individual cases and collected and captured information relevant to the factors in the framework, and the challenges and lessons learned in different collaborative modelling approaches. Some papers reported only one case, other reported several. One paper reported 23 different groups working in collaborative settings to build separate models. These were reported as 23 separate cases. For each case, we filled out the framework where possible. For some cases, some parts of the framework had to be left blank. Furthermore, for each factor, we included any available information, either quantitative or qualitative. A complete list of papers included in the study is provided in Appendix. After completing the framework for each case, we searched for commonalities and patterns in styles and methods used to enable and support collaborative modelling. The results are presented below.
5.1.2 Group composition
5
5.3 Intervention
Results
In total, we captured 71 cases in 22 papers. They covered different collaborative modelling efforts that fit our definition. All cases were conducted synchronously in a face-to-face setting. Below, we summarise the results for each factor in the framework.
5.1 Group factors
253
Sixty six cases reported the composition of the group. In 38 cases, the group consisted of representatives of various military services. In three cases, only domain experts were involved. In 12 cases, the group included management representatives as well. In one case, the group consisted of management representatives only. In eight cases, the group consisted of various public servants and citizens, and in four cases the group consisted of students. Fifty nine cases reported the country in which the modelling effort took place: 46 modelling efforts were conducted in North America, 12 in Europe and 1 in Asia. With respect to the modelling experience of the participants, only four cases explicitly mentioned the lack of modelling experience, and two cases mentioned modelling experienced participants in the group. In 12 cases, participants received a small training, and in 27 cases participants received several days of training in advance.
5.2 Task 5.2.1 Domain The domains in which the modelling efforts were conducted include military, police, public, healthcare, engineering, the environment, financial services, and logistics.
5.2.2 Model goal Most modelling efforts were initiated to handle escalating problems, for future planning, or for a better understanding of the current situation.
5.2.3 Session time The average time taken for the collaborative modelling sessions was 10.8 days per case. We note that the session time varies considerably per case, ranging from 15 min to 100 days.
5.3.1 Model types The most occurring types of models were IDEF0 (36 cases), causal maps (11 cases), task structures (nine cases), and data models (4). Further, the joint creation of an organisational structure and mental model was reported. Other papers did not report the model type.
5.1.1 Group size
5.3.2 Roles
From the case studies that reported the group size (57), we found 16 cases with small groups (1–6 group members), 27 cases with medium-size groups (7–15 group members), and 14 cases with large groups (16 or more participants). The average group size was 13.8 participants per case.
In 55 cases, the roles defined were reported. 52 cases mentioned the presence of a facilitator. In three of these cases, the facilitator role was combined with the role of modeller. In 13 of the 52 cases, a separate modeller was present at the sessions.
254
M. Renger et al.
5.3.3 Modelling participation We found 28 traditional, fully chauffeured sessions, 25 partially chauffeured sessions, and nine interactive sessions.
5.3.4 Modelling approach We identified three major schools of approaches that were used in the cases. These approaches are further explained in Section 6.
5.4 Outcome We found both qualitative and quantitative studies. Six cases reported effectiveness results, while five cases reported efficiency results. In three cases, satisfaction was assessed, in six cases consensus was evaluated, and in five cases shared understanding was evaluated. Finally, model complexity was assessed in 51 cases in a quantitative way and in four cases qualitative metrics were used to express model complexity.
6
Challenges
When comparing the different collaborative modelling studies, we found challenges and lessons learned on the following topics: •
the roles and group composition
•
the interactive process, collaboration and participation
•
the modelling approach, activities and modelling rules to support the modelling effort
•
the model quality, both from an objective and a subjective perspective.
6.1 Roles and group composition Collaborative modelling requires expertise in two distinct areas: facilitation of the group process and expertise in modelling, and the modelling semantics. Such expertise is generally not available in organisations and therefore outsourced. On the other hand, participants can also fulfil different roles in the collaborative modelling process to coordinate tasks and responsibilities in the modelling effort.
6.2 Facilitation roles Richardson and Andersen (1995) have described five essential roles that should be present in a Group Model Building Session: the facilitator, the modeller/reflector, the process coach, the recorder and the gatekeeper. Roles can be allocated to different people in the group, which can affect the workload of those participants and therewith the effectiveness and the efficiency of collaboration support (Kolfschoten et al., 2008). Furthermore, some roles can be combined, or even (partly) assigned to group members. Having an outside facilitator is considered very useful,
especially if technology is used (de Vreede et al., 2002; Dennis et al., 2001). Vennix et al. (1992) note that facilitated groups get less frustrated, have strongly improved group performance, less social–hierarchical domination in discussions and focus on a broader spectrum of approaches to the problem. In traditional modelling approaches, the input of stakeholders is processed into a model by the analyst/modeller. But, also in more collaborative settings the role of modeller is mentioned in the literature. However, a modeller/reflector will not only support the process of collaborative modelling, but also interfere with the content to help groups to understand the system or process under discussion. There is a discussion among scholars about the effect and ethics involved in content interference by outside facilitators (Griffith et al., 1998). Also, there is no consensus among scholars about whether the roles of facilitator and modeller should be represented by separate persons. Especially when the task is complex, a large cognitive load is imposed on a person that serves both (Richardson and Andersen, 1995). Separated roles of modeller and facilitator are found to save time and increase model quality (den Hengst and de Vreede, 2004; Richardson and Andersen, 1995; Rouwette et al., 2000). A facilitator and modeller need to work together seamlessly and be careful not to create conflict between each other. Van den Belt (2004) suggests that these roles are, therefore, inseparately intertwined, and a combined role of facilitator and modeller is equally or even more efficient because it allows a stronger focus on conflict resolving tasks. Moreover, a larger supporting team is more expensive. A third possibility would be to assign the modeller role to the group of participants as a whole, which would increase the participation of group members, but would also pose great challenges for the facilitation. Little explicit research is found with the use of this approach. The recorder, also known as chauffeur, provides the technical support to the group by processing the input of the group directly into a modelling tool and by operating any additional technology such as GSSs (Kolfschoten et al., 2008; Richardson and Andersen, 1995). The role of the chauffeur is closely related to the role of modeller, but the chauffeur functions more as a scribe, whereas the modeller also interprets and reflects on the group’s input. Both roles can be executed by one person. The process coach focuses solely on the dynamics of individuals and subgroups and serves as an assistant to the facilitator to decrease cognitive load. The role of the process coach is to detect conflict, uneasiness, dissatisfaction, a lack of motivation and other signs of the group that require action from the facilitator. It is the task of the process coach to not only identify these needs, but also suggest remedies (Richardson and Andersen, 1995). Little literature is found that explicitly mentions this role. Finally, the gatekeeper is the medium between the facilitation and the participation roles. This role is a twofold representative: for the participants, he represents the supporting team and vice versa. Usually, this is a person
Challenges in collaborative modelling: a literature review and research agenda within the organisation who initiates and carries responsibility for the project. The gatekeeper can help the facilitation team in preparation tasks and in assigning participation roles (Richardson and Andersen, 1995).
6.3 Participation roles One of the main reasons for failure of business process re-engineering projects is the involvement of the wrong people at the wrong time for the wrong reasons (den Hengst and de Vreede, 2004). Within all schools of thought, the importance of selecting the right participants is acknowledged (Andersen and Richardson, 1997; de Vreede et al., 2003). There are several factors that should be kept in mind when composing the group. Critical choices should be made with respect to the involvement of experts (professionals or people with experience) and the involvement of stakeholders. Sometimes both experts and stakeholders are involved to achieve both model quality as well as support for the resulting representation of the system or process. With respect to expert involvement, a trade-off emerges between quality of the model and shared understanding. First of all, the richness of the expertise in the group should be considered to produce a complete model that covers the scope of the system. However, when experts have non-overlapping expertise, it might become more difficult to create shared understanding. A similar trade-off occurs when inviting stakeholders. When critical stakeholders are not invited, the group can have insufficient decision power, and there can be a lack of support for solutions and decision by non-invited or insufficiently heard stakeholders (den Hengst and de Vreede, 2004; Maghnouji et al., 2001). Dean et al. (1994) note that the presence of both process designers and process owners is important in process analysis. In other methods, user involvement might be critical. On the other hand, more stakeholders can result in more conflict, which will require more consensus building activities. For the same reason, it is found more difficult to maintain motivation among participants in a freestanding project with non-professional interests (van den Belt, 2004). In general, in a large group, participants have less average speaking time in ‘live’ discussions (Rouwette et al., 2000). Further, a large group produces a lot of information, which in its turn puts a strain on the cognitive load of individual participants with the danger of information overload (Dennis et al., 1994, 1999). Although there are means to overcome the challenges of large groups, e.g. by working in parallel by the use of GSSs, group discussions are found to be in the heart of Group Model Building processes (Rouwette et al., 2000). In some cases, different stakeholders or experts are involved at different steps in the modelling process to reduce the burden on the costly time of professionals. However, stakeholders who are involved later can put earlier-made decisions back on the agenda, or even reject these (den Hengst and de Vreede, 2004; Maghnouji et al., 2001).
255
Simultaneously involved participants are found to increase pace and buy-in (den Hengst and de Vreede, 2004).
6.4 Collaboration and participation As mentioned above, the presence of a modeller or chauffeur has a large impact on the group processes. When using a scribe/modeller role, individual group members have less direct access and power to influence the model. The rules for scribe/chauffeur and modeller are different. In a chauffeured setting, changes made to the model will have to be agreed upon by the group before the chauffeur effectuates the change. Another rule, more associated to the modeller role, is that the modeller can freely interpret the group discussion, and based on this, change or extend the model (Richardson and Andersen, 1995). Another effect of the presence of the chauffeur or modeller is that participants have less access and ability to modify the model, and therewith to explain their perception to the group. This lack of access can lead to less interactivity because of more indirect communication among group members via the chauffeur (Rouwette et al., 2000). This can decrease the feeling of data ownership or group contribution to the model. Also, some cases in which a modeller carried out the steps were identified as time-consuming (den Hengst and de Vreede, 2004). In some cases, a modeller interpreted and made changes to the model in between sessions, which resulted in feelings that the model no longer captured the group’s original intent (Dean et al., 2000). A problem with inability to change the model is that, on the one hand, participants are asked to ‘translate’ their perceptions and ideas to the modelling language, but, on the other hand, they do not get the opportunity to verify this translation, or to express their perception in the common modelling language. This lack of ability to express and the lack of feedback can cause a feeling of not being understood, and not being able to express a vision or perspective. Experiments with a modeller making changes simultaneously with the group were less successful because the modeller could not keep up with the pace of the group. Another possibility is that the group can directly make changes to the model, while the facilitator or a separate modeller is present to give modelling guidance to the group (Dean et al., 1994, 2000). In most cases of unchauffeured modelling, group members were given a modelling training in advance of the session, which is acknowledged to be critical (Dennis et al., 1994). A critical enabler of full group participation is the ability to work in parallel. In all cases where the model was built in parallel, the group divided into subgroups, and subgroups were assigned parts of the model corresponding to the subgroups’ expertise. Dennis et al. (1994) found that parallel-built models are built ten times as fast as models built with the plenary group. While in general facilitation with GSSs individual parallel work is common, little literature is found on individual parallel model building.
256
M. Renger et al.
In both cases, a key challenge lies in change management and change awareness among parallel working individuals or groups. Andersen and Richardson (1997) found that convergent thinking requires the input of the group as a whole (Vennix et al., 1992). Group processes of collaborative modelling efforts can be strongly affected by social factors that are apparent among group members, e.g., organisation-hierarchical factors, and conformism. Conformism is the phenomenon that persons tend to align opinions to the group opinion, especially when speaking in front of the group, which is also called groupthink (Janis, 1972). Groupthink is a negative form of convergence, where confirmation is not rational but based on social pressure. These factors can be effectively dealt with by the use of anonymity in the electronic modelling tool (Rouwette et al., 2000). A downside of anonymity is that changes cannot be attributed to experts or stakeholders.
6.5 Modelling approach One of the main challenges of collaborative modelling is to design the process for the modelling effort, i.e., a sequence of modelling steps (Andersen and Richardson, 1997; Vennix et al., 1992). In Andersen and Richardson (1997), Andersen and Richardson plea for a flexible approach, where the structure of each modelling effort is adapted to the context and may even be adapted during the sessions. Dean et al. examine modelling approaches from a less flexible perspective (Dean et al., 2000). A key question in the design of the collaborative modelling process is whether to start from scratch or with a preliminary model that is created by the supporting team based on interviews or documents (Vennix, 1996). Vennix (1996) reports that the use of a preliminary model is most suitable for cases where time is costly or where the supporting team is less experienced, because it can increase efficiency and encourage a lively discussion right from the start (Andersen and Richardson, 1997). On the other hand, there is a danger of perceived lack of ownership of the preliminary model, which could result in low commitment, putting the process on the agenda, or rejection of the model (Eden and Ackermann, 2004; Vennix, 1996). The process may be thwarted if the preliminary model is based on unknown assumptions or outdated (den Hengst and de Vreede, 2004). When starting from scratch, several approaches are available. In most approaches, the first step consists of a brainstorm or ‘gathering’ to elicit the relevant concepts (Dean et al., 2000; Rouwette et al., 2000; Shaw et al., 2003). In approaches for Enterprise Analysis (Dean et al., 2000), an initial model is often created during the beginning of the modelling effort, often after a brainstorm about the most relevant high-level concepts. This initial model acts as a starting point for the further development, but differs from the preliminary models often used in Group Model Building (Andersen and Richardson, 1997; Vennix et al., 1992) in that it is built in-session with the plenary group rather than by the supporting team in advance. Furthermore,
although both starting models can change considerably during the overall process (and it is stressed that they will), initial models (built with the group) will probably determine the structure of the final model more than preliminary models (built by the support team), because preliminary models aim to provoke an initial discussion to elicit conflicting assumptions and perspectives (Dean et al., 2000; Vennix, 1996). The use of a preliminary model is extended in the so-called prototyping strategy, where for each step in the modelling process an analyst prepares the model and participants subsequently criticise and change the model (Lund and Palmer, 1997). den Hengst and de Vreede (2004) write that this approach produces better results than when participants carry out each step, or when an analyst carries out each step. In the subsequent convergence phase, a different emphasis emerges among modelling methods. PSMs have a stronger focus on eliciting the relations between individual mental maps, as the model is considered a means to achieve consensus and shared understanding (Eden and Ackermann, 2006; Shaw et al., 2003). The approaches for Enterprise Analysis focus more on the structure of the model itself, and are therefore more based on the grammar, and focus on correctness of individual modelling techniques, i.e., IDEF0-standards (Dean et al., 2000).
6.6 Model quality The importance of the model quality can differ for each case. For example, if the goal is to learn to improve collaboration and teamwork the modelling process is more important than its output (Ackermann et al., 2005), or when the goal is to learn the modelling method syntactic quality is critical (den Hengst, 2005). Quality is a container concept for ‘meeting criteria’. Depending on the goal of the collaborative modelling effort criteria for quality can be determined. In modelling, quality has two key classes of criteria: syntactic and a semantic quality. Syntactic quality concerns the correctness of the model according to the grammar of the modelling language, and therewith its explanatory power. Semantic quality concerns the correctness of the model in terms of content, and whether it represents the system it describes. In collaboration, quality is focused on process (e.g., participation, progress) and outcome (e.g., efficiency, effectiveness, complexity, shared understanding), and can be objective (e.g., time spend, quality according to experts) and subjective (e.g., satisfaction, usefulness).
6.7 Syntactic quality The syntactic quality of a model can be measured according to predefined model type-specific syntactic rules. These rules are most prescribed for the IDEF0 format, and most attention for syntax is found in the literature of the Enterprise Analysis School. Apart from IDEF0-specific rules, model quality aspects that are important for all modelling types are the low amount of homonyms (some concepts are included in others) and synonyms (overlapping
Challenges in collaborative modelling: a literature review and research agenda concepts), and the interconnectedness of different parts of the model, the latter being more a semantic qualifier. These aspects are especially important when the model is built by participants in a parallel setting (Dean et al., 1994; Dennis et al., 1994). Dennis et al. found that, as would be expected, models that are built by an experienced modeller have better syntactical quality than models that are interactively built by participants (Dennis et al., 1994, 1999). Therefore, approaches for parallel modelling with a high level of participation have to incorporate ways to improve syntactic quality. There are several methods at hand to this aim: training, guidance, periodic review, change awareness, and technological support. An extensive training of several days might be desired for the syntactic quality, but is often unpractical and costly. Therefore, Dean et al. (2000) suggest a combination of a small training and guidance of an experienced modeller during the sessions. Model integration can also be improved through an explicit integration process step with the plenary group, whereas integration of participant-built models by an external integrator can cause feelings of loss of ownership (Dennis et al., 1994, 1999). Further, a modelling tool can have various change awareness functionalities, with which a subgroup can view the changes made by other subgroups or are automatically notified about these changes. Some case studies report good results where subgroups integrated their own model parts with others during the parallel process step (Dean et al., 2000; Ram and Ramesh, 1998), but there is little evidence that supports the claim that a support team can rely on the voluntary integration by participants. Technological support can be used to avoid the appearance of homonyms and synonyms: before defining a new input, a user has to go through a list of previously defined concepts. Also, built-in restrictions according to syntactic rules can improve syntactical quality, which proved to be very successful (Dean et al., 1994, 2000).
6.8 Semantic quality In practice, semantic model quality can be difficult to measure, so one has to rely on the subjective perceptions of participants or the support team (Vennix, 1996). We note that in few studies perceptions of model quality are measured from a participants’ perspective. Semantic model quality concerns the completeness and correctness of the model (den Hengst and de Vreede, 2004). The completeness of a model denotes to what extent the model covers all aspects of the system it represents. A high complexity can be an indication of completeness, at the same time, models are meant to offer insight in an aspect or part of a system and should therewith reduce complexity (Nunamaker et al., 1991). There are several ways to measure complexity, either quantitatively by the number of objects and relations in the model, or qualitatively through observation, interviewing or analysis of results. The correctness of a model denotes to what extent the aspects of a system are depicted adequately (den Hengst and de Vreede, 2004).
257
den Hengst and de Vreede (2004) write that stakeholder involvement can produce more complete and correct models. This may be due to more richness and diversity of expertise in the group. Also, the role of a modeller in the session may be of influence to the model quality. However, Dennis et al. (2000) found hardly any difference in semantic model quality between collaborative and analyst-built models (Dennis et al., 1994).
7
Tool support for collaborative modelling
Given the challenges above, we see a number of requirements for tools to support collaborative modelling. When offering technology support for collaborative modelling, a digital visualisation of the model is created. This digital picture can be built and manipulated in an interactive and participative process. Functional requirements are discussed with respect to the user interface, manipulation rights, versioning and integration of models and coupling to simulation tools.
7.1 User interface Collaborative modelling poses high cognitive load on both the participants and the facilitators. Therefore, tools to support collaborative modelling should be as intuitive as possible. A natural user interface for modelling is a whiteboard. The natural whiteboard interface can be combined with advantageous features of electronic tools like GSS or Computer-Aided Software Engineering (CASE) tools by using an Interactive Whiteboard (IWB) (Damm et al., 2000). An IWB (also known as electronic whiteboard) is an interface device, which has at least the following functionalities: •
a large display, connected to a computer that is accessible for a group
•
the possibility to manipulate content on the display by the use of pens, fingers or other devices (touch screen).
Further: •
with the appropriate software, pens with different colours can be used to write text and lines that are recognised and transformed into digital text and straight connection lines
•
a wiper is available that can be used to erase content
•
if the IWB is connected to a beamer, one can reach larger groups.
An IWB offering this functionality is a Smartboard in combination with Smart-Ideas software or MS Visio (Smart Technologies Inc. web page, 2008). The use of IWBs is intuitive to manipulate the model, but might not be the most efficient way to support the divergent phase of a modelling effort. When collecting input from the group to create an initial set of elements and
258
M. Renger et al.
relations that will be included in the model, it is useful to enable parallel work. This can be achieved using GSSs. In such system, each participant can type in elements, indicate relations and vote on the need or characteristics of elements and relations (Aytes, 1995; Dean et al., 1997; Nunamaker et al., 1997). When the group engages in the phase where discussion leads to incremental changes and manipulations of the model, a whiteboard interface is more intuitive.
7.2 Manipulation rights In both the divergent phase and in the modelling phase, it is useful to have a highly flexible control on manipulation rights of the model. Some approaches to collaborative modelling work chauffeured, others allow for parallel manipulation of the model and yet others alternate between these modes of interaction. Further, the facilitators and modellers need to avoid chaotic interaction with the model and tug-of-war. Therefore, it is useful to enable a flexible control of manipulation rights, and a clear indication of manipulation rights. In a GSS system, the facilitator often has the option to give and take access rights; a flexible and intuitive interface for the facilitator to manipulate this is useful. For IWBs, the use of styluses can function to give access rights. However, if only one stylus is available, a participant can hold on to the stylus and therewith control the modelling effort, so the facilitator does not have a flexible way to change access rights. A solution to this would be to have multiple styluses that can be switched on and off by the facilitator and to indicate activity with a light or colour.
7.3 Versioning and integration As discussed in the challenges section, different approaches have different starting points. Some collaborative modelling approaches start with an existing model and extend this with the group, others work with sub-models that are later integrated, and yet others start from scratch and work in an iterative way to alter the model based on discussion. In each approach, it is important to enable saving of different versions of the model, to enable integration of different sub-models and to be able to go back or undo certain modelling steps. Changes in a graphical model can be captured in different ways as discussed in Tam and Greenberg (2004).
7.4 Coupling to simulation tools When creating a digital model, elements and relations can be associated with simulation building blocks and their relations (Valentin et al., 2003; Verbraeck et al., 2002). Further, research studies have indicated that it is possible to create an interface between conceptual digital modelling and actual simulation tools allowing groups to actually run simulation experiments when finishing their conceptual model with a single button (Davis, 2007). Besides creating the right interface, such a coupling requires the use of
default parameters, or further coupling of model parameters to feed data into the simulation model.
8
Discussion and conclusions
Increased participation in modelling processes is believed to produce higher-quality models of complex systems and lead to a better buy-in for the final model. However, participant involvement in collaborative modelling introduces a number of challenges. To address these challenges, we need to gain a deeper understanding of the different collaborative modelling techniques and methods used. A key difference in the approaches found is whether to build an integrated model from scratch or to let the group start from different models and work to integrate them. We also found some hybrid approaches in which iterative cycles were made from individual models to integration. From the results and the schools, we can distil a number of interesting opportunities for further research. With respect to the group size (average was 13.8), we believe that large group sizes are rather challenging by themselves. In management and collaboration literature, a group size of seven is considered the maximum span of control for a leader (Yassine et al., 2005). The effect of the group size on quality of the model is an interesting area of study. Also, the use of domain experts or stakeholders can have significant impact on the ease with which the group creates shared understanding and finds consensus. These relationships would also offer interesting research opportunities. The task in collaborative modelling efforts occurs often in complex domains with various stakeholders and presents the need to involve expertise from different fields that might be non-overlapping, which can result in misunderstandings. Collaborative modelling efforts take a significant amount of time, and are therewith costly. This reveals a need for methods and techniques that increase efficiency without sacrificing quality. With respect to the interventions, we found that while some characteristics of the approach and the type of model are often reported, the process, especially the detailed techniques, methods and interventions were less frequently discussed. Next, we found a couple of key trade-offs that have to be considered for successful collaborative modelling efforts. A first trade-off can be found in the choices with respect to group composition. On the one hand involving stakeholders and experts can improve correctness, buy-in and completeness; on the other hand, it can lead to conflict and misunderstanding due to different perspectives and non-overlapping expertise. In smaller groups, model building efficiency will be higher, participation will increase and it is generally easier to create shared understanding. A second important trade-off was found with respect to the level of participation. If participants are empowered to make changes to the model directly, they will have a feeling of ownership and are more likely to accept the final model and decisions derived from it. However, critical
Challenges in collaborative modelling: a literature review and research agenda stakeholders and domain experts are not necessarily skilled modellers. To achieve syntactical quality of the model, it is therefore useful to involve a chauffeur or modeller. The trade-off between quality and participation has to be evaluated in light of the scope and complexity of the system that is to be represented. Further research has to be done to evaluate whether the role of modeller can be performed by participants themselves. A third critical challenge is the choice of a starting point for the modelling task. The use of a preliminary model, created by an expert or analyst, outside the group process, can speed up the process and raise critical discussion topics, but can also cause detachment and even rejection of the process and the resulting model. A final challenge can be found when collaborative modelling effort is performed in parallel, which can also improve modelling efficiency. When separate (sub) models are created in parallel, a challenge lies in the convergence and integration of these models. To support integration of sub-models or changes created in parallel, strict rules are required to ensure syntactical quality and shared understanding. We note that some collaborative modelling studies focused on the effect of technology on the outcomes of the collaborative modelling process. The results of these studies suggest that GSS, IWBs and other collaboration technologies can increase the efficiency of collaborative modelling sessions. Further research on tools support to enable access control, model integration and versioning in a digital environment would be a valuable resource to both researchers and practitioners. Also, the coupling between conceptual modelling tools and simulation tools offers an interesting and promising opportunity to increase the efficiency of simulation while enabling increased participation of experts in the simulation effort. Finally, it appears that the outcomes of collaborative modelling efforts are only studied in a limited way. Many articles focus on the qualities and complexity of the resulting model according to experts. However, very few articles describe the participants’ evaluation of the resulting model, its use or the efficiency of the effort. Also, it would be especially interesting to assess shared understanding that results from a modelling effort and its correlation with the complexity of the resulting model. This paper presented a literature review of collaborative modelling cases. We developed a framework, collected data about the cases according to this framework and reported the results. Although we do not claim that the review is exhaustive, it does offer an interesting panorama of the richness of methods and approaches. The opportunities for synergy and learning from these cases are obvious. A critical step to advance research in this relevant and important domain would be to further identify, codify, and
259
document best practices and methods for collaborative modelling (Andersen and Richardson, 1997), and to collect both quantitative and qualitative data about the effects of specific interventions in collaborative modelling efforts. This will help to understand and compare different approaches and techniques and will ultimately improve the effectiveness and efficiency of collaborative modelling both in business and education.
References Ackermann, F. and Eden, C. (2005) ‘Using causal mapping with group support systems to elicit an understanding of failure in complex projects: some implications for organizational research’, Group Decision and Negotiation, Vol. 14, pp.355–376. Ackermann, F., Franco, L.A., Gallupe, B. and Parent, M. (2005) ‘GSS for multi-organizational collaboration: reflections on process and content’, Group Decision and Negotiation, Vol. 14, pp.307–331. Andersen, D.F. and Richardson, G.P. (1997) ‘Scripts for group model building’, System Dynamics Review, Vol. 13, No. 2, pp.107–129. Andersen, D.F., Vennix, J.A.M., Richardson, G.P. and Rouwette, E.A.J.A. (2007) ‘Group model building: problem structuring, policy simulation and decision support’, Journal of the Operational Research Society, Vol. 58, pp.691–694. Aytes, K. (1995) ‘Comparing collaborative drawing tools and whiteboards: an analysis of the group process’, Computer Supported Cooperative Work, Kluwer Academic Publishers, Vol. 4, pp.51–71. Barjis, J. (2008) ‘The importance of business process modeling in software systems design’, Sci. Comput. Program., Vol. 71, No. 1, pp.73–87. Boehm, B., Gruenbacher, P. and Briggs, R.O. (2001) ‘Developing groupware for requirements negotiation: lessons learned’, IEEE Software, Vol. 18, No. 3, pp.46–55. Briggs, R.O., Kolfschoten, G.L. and de Vreede, G.J. (2006a) ‘Instrumentality theory of consensus’, Paper presented at the First HICSS Symposium on Case and Field Studies of Collaboration, Kauai, Hawaii, USA, p.10. Briggs, R.O., Reinig, B.A. and de Vreede, G.J. (2006b) ‘Meeting satisfaction for technology-supported groups: an empirical validation of a goal-attainment model’, Small Group Research, Vol. 37, No. 6, pp.585–611. Brooks, R.J. (2006) ‘Some thoughts on conceptual modelling: performance, complexity and simplification’, Paper presented at the OR Society Simulation Workshop, Warwick, UK. Creswell, J.W. (1994) Research Design: Qualitative and Quantitative Approaches, Sage Publications, Inc., Thousand Oaks, California. Damm, C.H., Hansen, K.M. and Thomsen, M. (2000) ‘Tool support for cooperative object-oriented design: gesture based modeling on an electronic whiteboard’, Paper presented at the SIGCHI Conference on Human Factors in Computing Systems, The Hague, The Netherlands, pp.518–525. Davis, R. (2007) ‘Magic paper: sketch-understanding research’, Computer, Vol. 40, No. 9, pp.34–41.
260
M. Renger et al.
de Vreede, G.J., Boonstra, J. and Niederman, F. (2002) ‘What is effective gss facilitation? A qualitative inquiry into participants’ perceptions’, Paper presented at the 35th Annual Hawaii International Conference on System Sciences (HICSS’02), Vol. 1, Waikoloa, Maui, Hawaii, USA, pp.616–627. de Vreede, G.J., Davison, R.M. and Briggs, R.O. (2003) ‘How a silver bullet may lose its shine – learning from failures with group support systems’, Commun. ACM, Vol. 46, No. 8, pp.96–101. Dean, D.L., Lee, J.D., Orwig, R.E. and Vogel, D.R. (1994) ‘Technological support for group process modeling’, Journal of Management Information Systems, M. E. Sharpe, Inc., Vol. 11, pp.43–63. Dean, D.L., Lee, J.D., Pendergast, M.O., Hickey, A.M. and Nunamaker Jr., J.F. (1997) ‘Enabling the effective involvement of multiple users: methods and tools for collaborative software engineering’, Journal of Management Information Systems, M.E. Sharpe, Inc., Vol. 14, pp.179–222. Dean, D.L., Orwig, R.E. and Vogel, D.R. (2000) ‘Facilitation methods for collaborative modeling tools’, Group Decision and Negotiation, Vol. 9, pp.109–127. den Hengst, M. (2005) ‘Collaborative modeling of processes: what facilitation support does a group need?’, Paper presented at the Americas Conference on Information Systems, Omaha, NE, USA, pp.73–80. den Hengst, M. and de Vreede, G.J. (2004) ‘Collaborate business process engineering: a decade of lessons from the field’, Journal of Management Information Systems, Vol. 20, pp.85–113. Dennis, A.R., Hayes, G.S. and Daniels Jr., R.M. (1994) ‘Re-engineering business process modeling’, Paper presented at the 27th Annual Hawaii International Conference on System Sciences, Maui, Hawaii, USA, pp.244–252. Dennis, A.R., Hayes, G.S. and Daniels Jr., R.M. (1999) ‘Business process modeling with group support systems’, Journal of Management Information Systems, M. E. Sharpe, Inc., Vol. 15, pp.115–142. Dennis, A.R., Wixom, B.H. and Vandenberg, R.J. (2001) ‘Understanding fit and appropriation effects in group support systems via meta-analysis’, Management Information Systems Quarterly, Vol. 25, No. 2, pp.167–183. Eden, C. and Ackermann, F. (2004) ‘Cognitive mapping expert views for policy analysis in the public sector’, European Journal of Operational Research, Vol. 127, No. 3, pp.615–630. Eden, C. and Ackermann, F. (2006) ‘Where next for problem structuring methods’, Journal of the Operational Research Society, Vol. 57, pp.766–768. Erickson, J. and Siau, K. (2007) ‘Theoretical and practical complexity of modeling methods’, Communications of the ACM, Vol. 50, No. 8, pp.47–51. Fjermestad, J. (1998) ‘An integrated framework for group support systems’, Journal of Organizational Computing and Electronic Commerce, Vol. 8, pp.83–107. Frost and Sullivan (2007) Meetings Around the World: The Impact of Collaboration on Business Performance, www.frost.com, http://newscenter.verizon.com/kit/collaboration/MAW_WP. pdf
Fruhling, A. and Vreede, G.J.De (2005) ‘Collaborative usability testing to facilitate stakeholder involvement’, in Biffl, S., Aurum, A., Boehm, B., Erdogmus, H. and Grünbacher, P. (Eds.): Value Based Software Engineering, Springer-Verlag, Berlin, pp.201–223. Griffith, T.L., Fuller, M.A. and Northcraft, G.B. (1998) ‘Facilitator influence in group support systems’, Information Systems Research, Vol. 9, No. 1, pp.20–36. Gurung, T.R., Bousquet, F. and Trébuil, G. (2006) ‘Companion modeling, conflict resolution, and institution building: sharing irrigation water in the Lingmuteychu Watershed, Bhutan’, Ecology and Society, Vol. 11, No. 2, p.36, http://www.ecologyandsociety.org/vol11/iss2/art36/ Hill, R.C. and Levenhagen, M. (1995) ‘Methaphors and mental models: sensemaking and sensegiving in innovative and entrepreneurial activities’, Journal of Management, Vol. 21, No. 6, pp.1057–1074. in ’t Veld, J. (1987) Analyse van Organisatie Problemen, Stenfert Kroese, Leiden. Janis, I.L. (1972) Victims of Groupthink: A Psychological Study of Foreign-Policy Decisions and Fiascoes, Houghton Mifflin Company, Boston. Kolfschoten, G.L., Niederman, F., de Vreede, G.J. and Briggs, R.O. (2008) ‘Roles in collaboration support and the effect on sustained collaboration support’, Paper presented at the HICSS, Waikoloa, Maui, Hawaii, USA, p.9. Lund, J.R. and Palmer, R.N. (1997) ‘Water resource system modeling for conflict resolution’, Water Resources Update, Vol. 108, pp.70–82. Maghnouji, R., de Vreede, G., Verbraeck, A. and Sol, H. (2001) ‘Collaborative simulation modeling: experiences and lessons learned’, Proceedings of the 34th Annual Hawaii International Conference on System Sciences (HICSS-34), Vol. 1, IEEE Computer Society, Waikoloa, Hawaii, USA Washington, p.1013. Morton, A., Ackermann, F. and Belton, V. (2003) ‘Technologydriven and model-driven approaches to group decision support: focus, research philosophy, and key concepts’, European Journal of Information Systems, Vol. 12, No. 2, pp.110–126. Mulder, I., Swaak, J. and Kessels, J. (2002) ‘Assessing learning and shared understanding in technology-mediated interaction’, Educational Technology and Society, Vol. 5, No. 1, pp.35–47. Nunamaker, J.F.J., Briggs, R.O., Mittleman, D.D., Vogel, D. and Balthazard, P.A. (1997) ‘Lessons from a dozen years of group support systems research: a discussion of lab and field findings’, Journal of Management Information Systems, Vol. 13, No. 3, pp.163–207. Nunamaker Jr., J.F., Dennis, A.R., Valacich, J.S., Vogel, D.R. and George, J.F. (1991) ‘Electronic meeting systems to support group work’, Communications of the ACM, Vol. 34, No. 7, pp.40–61. Ram, S. and Ramesh, V. (1998) ‘Collaborative conceptual schema design: a process model and prototype system’, ACM Transactions on Information Systems, ACM, Vol. 16, pp.347–371.
Challenges in collaborative modelling: a literature review and research agenda Richardson, G.P. and Andersen, D.F. (1995) ‘Teamwork in group model building’, System Dynamics Review, Vol. 11, pp.113–137. Robinson, S. (2008) ‘Conceptual modelling for simulation Part I: definition and requirements’, Journal of the Operational Research Society, Vol. 59, pp.278–290. Rosenhead, J. (1993) Rational Analysis for a Problematic World: Problem Structuring Methods for Complexity, Uncertainty and Conflict, John Wiley & Sons Ltd, Chichester. Rouwette, E.A.J.A., Vennix, J.A.M. and Thijssen, C.M. (2000) ‘Group model building: a decision room approach’, Simulation and Gaming, Vol. 31, No. 3, pp.359–379. Rouwette, E.A.J.A., Vennix, J.A.M. and van Mullekom, T. (2002) ‘Group model building effectiveness: a review of assessment studies’, System Dynamics Review, Vol. 18, No. 1, pp.5–45. Shaw, D., Ackermann, F. and Eden, C. (2003) ‘Approaches to sharing knowledge in group problem structuring’, Journal of the Operational Research Society, Vol. 54, pp.936–948(913). Smart Technologies Inc. (2008) web page (2008) from http://smarttech.com/, 22 July. Standish Group (1995) CHAOS Report: Application Project and Failure, The Research Advisory firm The Standish Group International, Inc. in the so-called CHAOS Survey Research Series, http://net.educause.edu/ir/library/pdf/NCP08083B.pdf, http://www.standishgroup.com/ Steiner, I.D. (1972) Group Process and Productivity, Academic Press, New York. Tam, J. and Greenberg, S. (2004) ‘A framework for asynchronous change awareness in collaboratively-constructed documents’, Groupware: Design, Implementation and Use, Lecture Notes in Computer Science, Springer, Berlin/Heidelberg, Vol. 3198, pp.67–83. Trauth, E.M. and Jessup, L.M. (2000) ‘Understanding computer-mediated discussions: positivist and interpretive analysis of group support system use’, Management Information Systems Quarterly, Vol. 24, No. 1, pp.43–79. Valentin, E.C., Verbraeck, A. and Sol, H.G. (2003) ‘Effect of simulation building blocks on simulation model development’, Paper presented at the International Conference of Technology, Policy and Innovation, Monterey, Mexico, pp.54–61. van den Belt, M. (2004) Mediated Modeling: A System Dynamics Approach to Environmental Consensus Building, Island Press, Washington, DC, USA. Vennix, J.A.M. (1996) Group Model Building: Facilitating Team Learning Using System Dynamics, John Wiley & Sons, Chichester, UK. Vennix, J.A.M., Andersen, D.F., Richardson, G.P. and Rohrbaugh, J. (1992) ‘Model-building for group decision support: Issues and alternatives in knowledge elicitation support’, European Journal of Operational Research, Vol. 59, pp.28–41. Verbraeck, A., Saanen, Y., Stojanovic, Z., Shishkov, B., Meijer, A., Valentin, E. and van der Meer, K. (2002) ‘What are building blocks?’, in Verbraeck, A. and Dahanayake, A. (Eds.): Building blocks for Effective Telematics Application Development and Evaluation, Delft University of Technology, Delft, pp.8–21. Weick, K.E. (1995) Sensemaking in Organizations, Sage Publications Inc., Thousand Oaks, CA.
261
Wood, R.E. (1986) ‘Task complexity: definition of the construct’, Organizational Behavior and Human Decision Processes, Vol. 37, pp.60–82. Yassine, A., Goldberg, D. and Yu, T.L. (2005) Simple Models of Hierarchical Organizations, Illinois Genetic Algorithms Laboratory (IlliGAL) Technical Report No. 2005015, Urbana, IL, USA.
Appendix: Papers Included in the Literature Review Ackermann, F., Alberto Franco, L., Gallupe, B. and Parent, M. (2005) ‘GSS for multi-organizational collaboration: reflections on process and content’, Group Decision and Negotion, Vol. 14, pp.307–331. Ackermann, F. and Eden, C. (205) ‘Using causal mapping with group support systems to elicit an understanding of failure in complex projects: some implications for organizational research’, Group Decision and Negotion, Vol. 14, pp.355–376. Adamides, E.D. and Karacapilidis, N. (2006) ‘A knowledge centred framework for collaborative business process modelling’, Business Process Management, Vol. 12, No. 5, pp.557–575. Akkermans, H.A. and Vennix, J.A.M. (1997) ‘Clients’ opinions on group model-building: an exploratory study’, System Dynamics Review, Vol. 13, No. 1, pp.3–31. Andersen, D.F. and Richardson, G.P. (1997) ‘Scripts for group model building’, System Dynamics Review, Vol. 13, pp.107–129. Andersen, D.F., Vennix, J.A.M., Richardson, G.P. and Rouwette, E.A.J.A. (2007) ‘Group model building: problem structuring, policy simulation and decision support’, Journal of the Operational Research Society, Vol. 58, pp.691–694. Aytes, K. (1995) ‘Comparing collaborative drawing tools and whiteboards: an analysis of the group process’, Computer Supported Cooperative Work, Kluwer Academic Publishers, Vol. 4, pp.51–71. Daniell, K.A., Ferrand, N. and Tsoukia, A. (2006) ‘Investigating participatory modelling processes for group decision aiding in water planning and management’, Group Decision and Negotiation, Proceedings Group Decision and Negotiation International Conference, Karlsruhe, Germany, pp.207–210. de Cesare, S. and Serrano, A. (2006) ‘Collaborative modeling using UML and business process simulation’, HICSS ’06: Proceedings of the 39th Annual Hawaii International Conference on System Sciences, IEEE Computer Society, Kauai, Hawaii, USA, Washington, DC, USA, pp.10–12. de Vreede, G.J. (1995) Facilitating Organizational Change, PhD Dissertation, Delft University of Technology, Delft, The Netherlands. de Vreede, G.J. (1997) ‘Group modeling for understanding’, Journal of Decision Systems, Vol. 6, No. 3, pp.197–220. Dean, D.L., Orwig, R., Lee, J. and Vogel, D. (1994) ‘Modeling with a group modeling tool: group support, model quality and validation’, Proceedings of the Twenty-Seventh Annual Hawaii International Conference on System Sciences, Maui, Hawaii, USA.
262
M. Renger et al.
Dean, D.L., Lee, J.D. and Nunamaker, J.J.F. (1997) ‘Group tools and methods to support data model development, standardization, and review’, Proceedings of the Thirtieth Annual Hawaii International Conference on System Sciences, Maui, Hawaii, USA, p.386. Dean, D.L., Lee, J.D., Orwig, R.E. and Vogel, D.R. (1994) ‘Technological support for group process modeling’, Journal of Management Information Systems, M.E. Sharpe, Inc., Vol. 11, pp.43–63. Dean, D.L., Lee, J.D., Pendergast, M.O., Hickey, A.M. and Nunamaker Jr., J.F. (1997) ‘Enabling the effective involvement of multiple users: methods and tools for collaborative software engineering’, Journal of Management Information Systems, M.E. Sharpe, Inc., Vol. 14, pp.179–222. Dean, D.L., Orwig, R.E. and Vogel, D.R. (1996) ‘Facilitation methods for use with ems tools to enable rapid development of high quality business process models’, Proceedings of the 29th Hawaii International Conference on System Sciences, IEEE Computer Society, Washington, DC, USA, Vol. 3, p.472 Dean, D.L., Orwig, R.E. and Vogel, D.R. (2000) ‘Facilitation methods for collaborative modeling tools’, Group Decision and Negotiation, Vol. 9, pp.109–127. den Hengst, M. and de Vreede, G.J. (2004) ‘Collaborate business process engineering: a decade of lessons from the field’, Journal of Management Information Systems, Vol. 20, pp.85–113. den Hengst, M., de Vreede, G.J. and Magnhnouji, R. (2006) ‘Using soft OR principles for collaborative simulation: a case study in the Dutch airline industry’, Journal of the Operational Research Society, Vol. 58, pp.669–682. Dennis, A.R., Hayes, G.S. and Daniels Jr., R.M. (1994) ‘Re-engineering business process modeling’, Proceedings of the Twenty-Seventh Annual Hawaii International Conference on System Sciences, Maui, Hawaii, USA, pp.224–252. Dennis, A.R., Hayes, G.S. and Daniels Jr., R.M. (1999) ‘Business process modeling with group support systems’, Journal of Management Information Systems, M.E. Sharpe, Inc., Vol. 15, pp.115–142. Eden, C. and Ackermann, F. (2004) ‘Cognitive mapping expert views for policy analysis in the public sector’, European Journal of Operational Research, Vol. 127, pp.615–630. Eden, C. and Ackermann, F. (2006) ‘Where next for problem structuring methods’, Journal of the Operational Research Society, Vol. 57, pp.766–768. Gurung, T.R., Bousquet, F. and Trébuil, G. (2006) ‘Companion modeling, conflict resolution, and institution building: sharing irrigation water in the Lingmuteychu watershed, Bhutan’, Ecology and Society, Vol. 11, No. 2, p.36, http://www.ecologyandsociety.org/vol11/iss2/art36/ Hayne, S.C. and Ram, S. (1995) ‘Group database design: addressing the view modeling problem’, Journal of Systems and Software, Elsevier Science Inc., Vol. 28, No. 2, February, pp.97–116. Jefferey, A.B., Maes, J.D. and Bratton-Jeffery, M.F. (2005) ‘Improving team decision-making performance with collaborative modeling’, Team Performance Management, Vol. 11, No. 1, pp.40–50.
Lee, J.D., Dean, D.L. and Vogel, D.R. (1997) ‘Tools and methods for group data modeling: a key enabler of enterprise modeling’, SIGGROUP Bulletin, ACM, Vol. 18, No. 2, pp.59–63. Lee, J.D., Zhang, D., Santanen, E., Zhou, L. and Hickey, A.M. (2000) ‘ColD SPA: a tool for collaborative process model development’, HICSS ‘00: Proceedings of the 33rd Hawaii International Conference on System Sciences, IEEE Computer Society, Maui, Hawaii, USA, Washington, DC, USA, Vol. 1, p.1004 De Lucia, A.D., Fasano, F., Scanniello, G. and Tortora, G. (2007) ‘Enhancing collaborative synchronous UML modelling with fine-grained versioning of software artefacts’, Journal of Visual Languages and Computing, Academic Press, Inc., Vol. 18, pp.492–503. Lund, J.R. and Palmer, R.N. (1997) ‘Water resource system modeling for conflict resolution’, Water Resources Update, Vol. 108, pp.70–82. Maghnouji, R., de Vreede, G.J., Verbraeck, A. and Sol, H. (2001) ‘Collaborative simulation modeling: experiences and lessons learned’, HICSS ‘01: Proceedings of the 34th Annual Hawaii International Conference on System Sciences (HICSS-34), IEEE Computer Society, Waikoloa, Maui, Hawaii, USA, Washington, DC, USA, Vol. 1, p.1013. Millward, S.M. (2006) ‘Do you know your STUFF? Training collaborative modelers’, Team Performance Management, Vol. 12, Nos. 7–8, pp.225–236. Orwig, R. and Dean, D.L. (2007) ‘A method for building a referent business activity model for evaluating information systems: results from a case study’, Communications of the AIS, Vol. 20, Article 53, pp.872–891. Pata, K. and Sarapuu, T. (2006) ‘A comparison of reasoning processes in a collaborative modelling environment: learning about genetics problems using virtual chat’, International Journal of Science Education, Vol. 28, No. 11, Article 22, pp.1347–1368. Purnomo, H., Yasmi, Y., Prabhu, R., Hakim, S., Jafar, A. and Suprihatin (2003) ‘Collaborative modelling to support forest man-agement: qualitative systems management at Lumut mountain, Indonesia’, Small Scale Forest Economics, Management and Policy, Vol. 2, No. 2, pp.259–275. Ram, S. and Ramesh, V. (1998) ‘Collaborative conceptual schema design: a process model and prototype system’, ACM Transactions on Information Systems, ACM, Vol. 16, pp.347–371. Richardson, G.P. and Andersen, D.F. (1995) ‘Teamwork in group model building’, System Dynamics Review, Vol. 11, No. 2, pp.113–137. Rouwette, E.A.J.A., Vennix, J.A.M. and Thijssen, C.M. (2000) ‘Group model building: a decision room approach’, Simulation and Gaming, Vol. 31, No. 3, pp.359–379. Rouwette, E.A.J.A., Vennix, J.A.M. and van Mullekom, T. (2002) ‘Group model building effectiveness: a review of assessment studies’, System Dynamics Review, Vol. 18, No. 1, pp.5–45. Samarasan, D. (1988) ‘Collaborative modeling and negotiation’, SIGOIS Bulletin, Proceedings of the ACM SIGOIS and IEEECS TC-OA 1988 Conference on Office Information Systems, ACM, Palo Alto, California, USA, Vol. 9, pp.9–21.
Challenges in collaborative modelling: a literature review and research agenda Shaw, D., Ackermann, F. and Eden, C. (2003) ‘Approaches to sharing knowledge in group problem structuring’, Journal of the Operational Research Society, Vol. 54, pp.936–948. van den Belt, M. (2004) Mediated Modeling: A System Dynamics Approach to Environmental Consensus Building, Island Press, Washington, DC, USA. Vennix, J.A.M. (1996) Group Model Building: Facilitating Team Learning Using System Dynamics, John Wiley & Sons, Chichester, UK. Vennix, J.A.M. (1999) ‘Group model-building: tackling messy problems’, System Dynamics Review, Vol. 15, No. 4, pp.379–401.
263
Vennix, J.A.M., Akkermans, H.A. and Rouwette, E.A.J.A. (1996) ‘Group model-building to facilitate organizational change: an exploratory study’, System Dynamics Review, Vol. 12, No. 1, pp.39–58. Vennix, J.A.M., Andersen, D.F. and Richardson, G.P. (1997) ‘Foreword: group model building, art, and science’, System Dynamics Review, Vol. 13, No. 2, pp.103–106. Vennix, J.A.M., Andersen, D.F., Richardson, G.P. and Rohrbaugh, J. (1992) ‘Model-building for group decision support: issues and alternatives in knowledge elicitation support’, European Journal of Operational Research, Vol. 59, No. 1, pp.28–41.
