Computer Supported Cooperative Work (2010) 19:175–199 DOI 10.1007/s10606-010-9108-9
© Springer 2010
The Dynamics of Material Artifacts in Collaborative Research Teams Deana D. Pennington Department of Biology, University of New Mexico, MSC03 2020, Albuquerque, NM 87131-0001, USA (Phone: +1-505-2772595; Fax: +1-505-2772541; E-mail:
[email protected]) Abstract. Boundary objects are material artifacts that mediate the relationship between two or more disparate perspectives. The concept of boundary objects has been demonstrably useful in a variety of research areas; however, the meaning and function of boundary objects is contested. At issue is the relationship between boundary objects that negotiate between perspectives and those that specify across perspectives. In this study the changing nature of boundary objects in cooperative work is related to the dynamics of evolving problem conceptualization, system design, and enactment within cooperative work settings. Design based research on material artifacts produced by an incipient cross-disciplinary research team during their efforts towards negotiating integrated conceptualizations and specifying shared research agendas is used to generate a more comprehensive model of boundary objects through the life of a project. Key words: eScience, eResearch, cross-disciplinary collaboration, boundary objects, material artifacts, design based research
1. Introduction Star and Griesemer (1989) proposed two central activities for translating between different viewpoints in cooperative scientific work: methods standardization and boundary object creation. The first is critical in that established guidelines form a managerial system to which diverse allies can contribute concurrently. The second arises after the first. Once guidelines are established, boundary objects can be constructed. They conceived of boundary objects as an “analytical concept of those scientific objects which both inhabit several intersecting social worlds and satisfy the informational requirements of each of them.” Boundary objects were the point of intersection between viewpoints, and the process of creating and managing them was critical for maintaining coherence of the heterogeneous work system. Star and Griesemer (ibid.) found four types of boundary objects in their study of a natural history museum, which they recognized was not exhaustive: 1) repositories, 2) ideal types, 3) coincident boundaries, and 4) standardized forms. Subsequent work has demonstrated both the utility of the boundary object conceptualization, and its shortcomings for explaining the full range of usage of artifacts in practice. Case studies have further articulated characteristics of these four types, and identified new types (Fujimura 1992; Perry and Sanderson 1998;
176
Deana D. Pennington
Roth and McGinn 1998; Bowker and Star 1999). Boundary objects as theoretical constructs have become common in social science studies in general, and in CSCW studies in particular. The critical role of material artifacts in coordinating work practices is undisputed. However, Lee (2007), based on an ethnographic study of incipient collaborative work, implicated an impoverished view of boundary objects as a barrier to more comprehensive understanding of the role of material artifacts in collaborative work. She suggested material artifacts play an alternative role as ‘boundary negotiating objects’. Boundary negotiating artifacts are not constructed from standardized processes; rather, they assist in the establishment of standard processes. They are used to “record, organize, explore and share ideas; introduce concepts and techniques; create alliances; create a venue for the exchange of information; augment brokering activities; and create shared understanding” (p. 333). She suggested that boundary objects may be found primarily in fairly routine or fairly simple work projects, while boundary negotiating objects may be more prevalent in projects that are fairly non-routine and fairly complex, and that more studies of incipient collaborations might help resolve these issues. This paper presents a study of artifacts in incipient collaborative work in a cross-disciplinary science and technology research team. The goal is to develop a better understanding of the relationship between boundary negotiating objects (sensu Lee 2007) and boundary objects (sensu Star and Griesmer 1989). In this article the latter will be referred to as “boundary specifying objects” for clarity, and both types will be referred to as boundary objects (Figure 1). The questions of interest are “What is the relationship between boundary objects that negotiate and those that span viewpoints?”; “What is their relationship with methods standardization?”; and “How do each enable collaborative work?” The paper develops a fuller, dynamic system account of the role of different kinds of artifacts in different phases of collaborative work. This framework provides a basis for integrating the diverse empirical observations of this and prior work.
Figure 1. Terminology used in this paper. The set of all material artifacts contains a set of artifacts that constitute boundary objects—artifacts that bridge different viewpoints. Two classes of boundary objects have been recognized: 1) those that specify viewpoints and fully mediate their interaction, and 2) those that negotiate interaction between viewpoints. The question of interest in this article is, “What is the relationship between boundary negotiating and boundary specifying objects?”.
Collaborative Research Teams
177
2. Methods 2.1. Study context This study is based on a National Science Foundation Cyberinfrastructure (CI) Team project conducted from 2006 to 2009. The goal of the CI-Team program is development of cross-trained scientists, engineers, and computer scientists able to design, develop, and use emerging technologies. This CI-Team project (http:// scidesign.org) was designed to explore potential mechanisms for enabling collaborative research between scientists and computer scientists (eResearch or eScience). The overarching goal of such research is not just to support the work of scientists, but rather to collaboratively conceptualize and design innovative approaches in both science and technology (Lawrence 2006). Problem conceptualization must emerge during interactions between participants. The process for achieving this must include mechanisms for evolving individual ideas into collectively constructed agendas; advancing vague, ill-defined issues into well defined, tractable problems; and migrating independent researchers into unified cognitive systems. These must happen more or less simultaneously, as each is dependent on the others. A key factor known to influence the outcome of such collaborations is the development of common ground between participants (Olson and Olson 2000). Common ground is the mutual knowledge, beliefs and assumptions that form the basis for meaningful communication (Clark and Brennan 1991). The CI-Team project focused on building common conceptual ground that could lead to better integrated research efforts between diverse scientists researching broad-scale ecological problems, and computer scientists interested in developing distributed systems. Specifically, participants had to develop an understanding of each other’s research interests, which depended on effectively learning relevant vocabulary and concepts across disciplines. Based on that understanding, they had to formulate conceptual linkages between each other’s research interests, develop integrated conceptual frameworks, and conceive of innovative collaborative research. Participants in the group were recruited based on potential research contribution to the general problem area of climate change and vegetation impacts in the American Southwest, and included twenty five participants roughly equally divided between several scientific (ecology, geography, anthropology) and computer science (visualization, data mining, human factors, distributed computing) disciplines. Participants were a mixture of faculty, their students, and researchers. Common ground was built across these diverse perspectives using a set of group activities over the 3 year project period mediated by the principal investigator. All activities included creation of material artifacts. This article focuses on the changing nature of the material artifacts; however a brief description of the activities follows to depict the process that produced the artifacts.
178
Deana D. Pennington
Four consecutive group activities were conducted: 1) CI-Seminar, 2) CI-Vision Workshop, 3) CI-Strategy Workshop, and 4) CI-Design Workshop. The goal of the CI-Seminar was to expose the scientists to ongoing technical work in many existing eScience projects, reflect on how these emerging technologies could be relevant for their own research efforts, and develop relevant technical vocabulary. The CI-Vision Workshop brought the scientists and computer scientists together to initiate learning about each other’s research interests and begin to develop a shared research vision. The CI-Strategy Workshop further developed that vision by identifying strategic research directions. Lastly, multiple CI-Design Workshops targeted research proposal development in each strategic area. Although the overall, four step process was linear, each workshop consisted of complex interactions and iterative activities. Artifact creation was only one mechanism used to enable cross-disciplinary communication, but it was the most continuously used intervention throughout the project. A design-based research approach was used to study the role of artifacts in this incipient collaborative team. 2.2. Design based research (DBR) approach DBR is an extension of standard design and evaluation methodologies that explicitly maps interventions and their evaluation to extant theory, for the purpose of theory development (Figure 2). DBR was developed in classroom settings by education researchers in collaboration with teaching practitioners. For a complete review of DBR and its application in education, see the Special Issue of the Educational Psychologist devoted to DBR (Sandoval and Bell 2004). As Edelson
Figure 2. Logic model showing design based research approach and its application in this study. Theories, with or without formative evaluation, are used to design an intervention. During enactment the designed intervention interacts with exogenous factors to produce outcomes, some of which may have been expected based on theory and formative evaluation, others of which are unexpected. Summative evaluation takes place. Outcomes are explicitly mapped back to the theories that informed the design. The explicit link between scientific theory and summative evaluation is what sets this approach apart from standard evaluation practices.
Collaborative Research Teams
179
(2002) stated, DBR “explicitly exploits the design process as an opportunity to advance the researchers understanding.” Some CSCW projects already tacitly use the design process in this way. DBR formalizes the approach by making explicit the mapping between theories being tested, designed interventions and intervention outcomes. DBR is a synergistic methodology with case studies and controlled experimentation, which provide a wealth of valuable information about factors that must be considered in real situations and the processes they influence. However, they do not provide practical guidance about how the knowledge gained can be integrated in a real situation in order to achieve specific outcomes. As Sandoval (2004) states, “there is a tension between the desire for locally usable knowledge on the one hand and scientifically sound, generalizable knowledge on the other.” DBR integrates findings from case studies and controlled experiments into contrived activities in real settings with two goals: 1) development of theory-based conjectures that are used to manipulate group activities to produce targeted outcomes; and 2) refinement of theory based on evaluation of actual outcomes of those activities. Theory and practice are tightly coupled in DBR. DBR experiments with group interventions, recognizing that many factors in live settings cannot be controlled and that important, unanticipated outcomes may emerge as a result of the intervention. These unexpected outcomes provide evidence that can be used in refinement of the theory and concepts on which the original conjecture was based. Construction of material artifacts is known to be an important part of enabling group interaction and therefore was incorporated as part of the design of the CITeam intervention. The changing nature of the constructed artifacts, however, emerged during the process and was recognized as an important, unanticipated outcome. This paper focuses on that outcome and uses it to generate new ways of conceptualizing the role of material artifacts. At the same time, it illustrates the DBR approach and provides an example of its utility in CSCW contexts. 2.3. Material artifacts: theory, concepts and conjecture Frequently in DBR, theories and concepts from a variety of disciplinary perspectives are applied to generate the conjecture from which an intervention may be designed. There is a rich history of research on material artifacts across many disciplines. Two, in particular, informed the design of the CI-Team intervention: 1) material artifacts as boundary objects, and 2) material artifacts as learning scaffolds. Material artifacts as boundary objects. Voluminous literature exists on boundary objects as theoretical constructs and the application of boundary objects in a wide variety of settings. As mentioned in the introduction, boundary objects are conceived of as artifacts that capture and relate elements from more than one
180
Deana D. Pennington
perspective. In the original conceptualization of Star and Griesemer (1989), boundary objects align elements such that individuals can make use of them with no knowledge of any perspective other than their own. Conversely, Lee (2007) suggested that an alternative use of boundary objects is to negotiate and develop understanding between perspectives. Both perspectives, and others, have demonstrated the critical role of boundary objects in enabling information processing in a group, and are needed in incipient research teams. On the one hand, developing cross-disciplinary understanding is a critical prerequisite to developing integrated conceptual frameworks, from which co-created research ideas can emerge. On the other hand, cross-disciplinary efforts need to proceed as much as possible with only limited understanding of other disciplinary perspectives. Boundary negotiation must lead to boundary specification that everyone can use to align their research. Boundary negotiation and specification represent two different general kinds of cognitive processes: divergent and convergent thinking. Divergent and convergent thinking processes were initially identified by Guilford (1967) as two of six operational intellectual processes. Divergent thinking is the ability to generate a variety of solutions to a problem. Convergent thinking is the ability to deduce a single solution to a problem. In cross-disciplinary research, divergent thinking is needed to explore the problem space in search of potential conceptual connections. Convergent thinking is needed to evaluate those exploratory connections. Material artifacts can enable both kinds of thinking, but it is reasonable to expect that different kinds of material artifacts enable these different kinds of processes. Within CSCW, a substantial research agenda on knowledge cartography exists (Okada et al. 2008). These tools strive to map knowledge in order to better understand and reason about information. Okada et al. (2008) categorized these as concept mapping, argument and evidence mapping, issue mapping, web mapping, and thinking maps. These tools are similar in that they all enable explication of thoughts by iconic representation of terms and connections between terms. They differ in the kind of reasoning or audience that they target. For example, concept maps enable representation of the concepts a map creator considers relevant to a topic, and how he structures his knowledge about that topic. Conversely, argument and evidence maps enable explicit representation of the pros, cons, and evidence related to an issue. Either of these could be useful for divergent or convergent thinking activities, depending on how they are used. For instance, concept maps could be used to explore diverse perspectives on how to organize the problem space, a divergent thinking activity. Conversely, concept maps could be used to specify how different perspectives are going to be related in a particular project, a convergent thinking activity. Artifacts produced from either of these approaches, if used within a cooperative group to mediate perspectives, are boundary objects. The kind of boundary object that they represent (negotiating or specifying) depends on how they are used by the group.
Collaborative Research Teams
181
Material artifacts as learning scaffolds. In cross-disciplinary research teams a set of individuals representing diverse perspectives must learn each other’s mental models, learn how to fuse those differences into an integrated conceptual framework, and learn how to use that conceptual framework as a springboard to collaborative problem-solving (Pennington 2008). This depends on exchange of knowledge in ways that are conducive to making sense of a subject without requiring depth of understanding. A fairly limited understanding is sufficient. However, it is impossible to know in advance which aspects will be relevant, requiring an exploratory phase of high-level learning that helps frame problem conceptualization. Unfortunately, disciplinary concepts are rarely accessible in forms digestible by those from other disciplines lacking appropriate background (Jeffrey 2003). Some research has suggested that experts have specific cognitive issues that limit their ability to convey their knowledge to novices (Hinds and Pfeffer 2003). In particular, as depth of knowledge on a topic increases, experts conceive of the content in increasingly abstract and simplified ways and are unable to revert to more concrete, detailed descriptions required by novices (Hinds and Pfeffer ibid.). Research in the learning sciences has demonstrated that representations, technological tools, activities, and/or physical artifacts are used by more expert individuals in one-on-one learning situations as temporary supports (scaffolds) that improve novice learning (Davis and Miyake 2004). Once the learner has grasped the target concepts, the scaffold is no longer necessary— differing from the permanent artifacts described as boundary objects. While material artifacts are not the only scaffolding mechanism, they play a critical role in enabling the flow of information between individuals. There are two effects: 1) the effect on the recipient, for whom information is more easily grasped, and 2) the effect on the artifact creator, who must strive to construct an artifact that is easily grasped. In striving to construct an easily grasped artifact, an individual must organize his thoughts and conceptual frameworks in a way that he believes will make sense to the recipient. This requires active thought about what frameworks, other than his own, might make sense. The artifact creator reframes his own mental models in ways that he thinks might resonate with the recipient. In so doing, he is engaging in his own internal manipulation activity that partially incorporates the recipient’s perspective. Hence, artifact construction entails learning by both the creator and the recipient, and enables not just the flow of information but also the dynamic creation of new mental models that contain linkages between participants. Artifact construction dynamically affords collective learning (sensu Cook and Brown 1999). Given the conceptual frameworks provided by boundary objects and scaffolding accounts of material artifacts, the conjecture was made that material artifact construction could be a primary mechanism for enabling crossdisciplinary learning and discourse on eScience teams if used in structured, facilitated ways. Typically on these teams material artifacts are constructed in advance (such as presentation slides) or occasionally on the fly (such as a white
182
Deana D. Pennington
board illustration). The conjecture was that centering all group activities on the collective construction of specific kinds of material artifacts and facilitating discourse around those artifacts would better enable the emergence of integrated research ideas. Construction of material artifacts during discourse has the concomitant benefit of capturing the content being discussed, which can be subjected to content analysis and provide a mechanism for measuring the change in conceptual integration across disciplines. Therefore, if material artifacts were indeed playing a central role in the exchange of knowledge and ideas then their content should reflect the transformation from separate, disciplinary perspectives to more integrated cross-disciplinary frameworks through time. 2.4. Designing theory-based team interactions Based on the above concepts, an intervention was designed to enable crossdisciplinary discourse through the creation of material artifacts. Several key strategies were incorporated:
& & & &
&
&
All interactions between participants were centered around the creation of material artifacts. All material artifacts produced during the project were captured digitally and incorporated into a web accessible archive. The web archive both contains material artifacts and itself acts as a material artifact. The CI-Seminar incorporated numerous examples of ways emerging technologies are being used in science. These were made explicit through digitally recording of presentations, and incorporation of slide presentations into the web archive. Subsequent interactions between scientists and computer scientists iterated between 1) divergent thinking activities that supported exploration and learning across the problem space and 2) convergent thinking activities that strove to scope specific research areas of interest. Artifact creation was an integral part of both types of activities, and different types of artifacts were necessary for these two different kinds of activities. Divergent thinking activities, e.g. exploratory learning, were enabled by concept mapping, a commonly used scaffolding mechanism in classroom settings (Novak and Wurst 2005). Concept maps are diagrams that capture associations between concepts (Figure 3). The utility of concept maps as a mechanism for enabling interdisciplinary discussion has been demonstrated (Heemskerk et al. 2003; Jeffrey 2003). Convergent thinking activities, e.g. scoping activities, targeted making conceptual linkages explicit, identification of potential nexus points between researchers, and progressive narrowing of linked research interests. Use of a diverse array of material artifacts (wikis, issue visualization, and whiteboards) were explored.
Collaborative Research Teams
183
Figure 3. Example concept map from a portion of a map created by a scientist.
2.5. Enactment and data collection A wiki-based project website (http://scidesign.org) was designed using the open source content management platform Drupal (http://drupal.org). This was linked to the open source course management system Moodle (http://moodle.org). The website was used for participants to collectively edit shared documents of a public nature while Moodle was used to manage the CI-Seminar and to organize artifacts from topic specific research areas of interest. Moodle provides a wide variety of tools for group work, including document archives, links to shared resources, forums, wikis and chat. The CI-Seminar was held synchronously at three institutions in a videoconferencing environment (Macromedia Breeze; http://www.adobe.com): University of New Mexico, University of Arizona, and Northern Arizona University. Macromedia Breeze was selected because of its digital recording capabilities, lacking in available open source solutions such as AccessGrid (http://www. accessgrid.org). All scientists involved in the project attended the seminar; only students were required to complete assignments. Each session was conducted by remote speakers from across the US and recorded in its entirety. Each remote speaker provided an electronic copy of his presentation that was placed on the
184
Deana D. Pennington
website. Relevant papers were provided in Moodle prior to each session. Students were routinely assigned the task of writing about the content. They participated in a written online forum, responded to questions about technical developments in their own field, and wrote about prospective applications of technologies introduced in the seminar in their own fields. The CI-Vision Workshop was held shortly after the seminar concluded. Both scientists and IT experts were involved, and the meeting consisted of a divergent thinking activity designed to learn about each other’s research interests and explore possible connections followed by a convergent thinking activity to create a shared vision of integrated research. During this workshop concept maps were constructed by each participant regarding their research interests (Figure 3), after brief training on the approach and the open source CMap Tools software (http:// cmap.imhc.us). Each participant talked about their own research using their concept map as a scaffolding device. Participants were asked to keep a running log of potential pairwise linkages between themselves and others, a convergent activity. When every participant had presented and exchanged ideas, each individual posted their list of potential linkages on a shared wiki, from which a collective list of potentially interesting research topics was created. That list, and the activity of generating the list, was used to initiate discussion of a broad research vision that encompassed those linkages. The final outcome of this workshop was a written shared vision that was rather ill-defined, but which was collectively generated and incorporated the individual research interests of most participants. Artifacts produced by the CI-Vision Workshop included a repository of individual concept maps, a shared wiki that included individually identified linkages, and a short statement of the shared research vision. The CI-Strategy Workshop targeted further refinement of the shared vision. This workshop began with a restatement of the shared vision and discussion of potential nexus points, which in this case revolved around methods for data analysis and visualization. Participants were asked to draw concept maps of their view of data, analysis and visualization in the context of the shared vision, using seed terms that were suggested by one of the participants (a divergent thinking activity). Eight seed terms were given: distributed data, data analysis, data products, online modeling, modeling processes, decision support, visualization tools, and animation tools. Participants could elect to use any or none of these. Each concept map was collectively viewed and discussed, and an integrated conceptual workflow was generated (a convergent thinking activity). Dialogue and interactions around these concept maps led to identification of three separate cross-disciplinary research areas of interest; identified content for two new virtual seminars to develop better cross-disciplinary understanding in those areas; considered specific training that would enhance the ability of the scientists to envision usefulness of certain technical approaches; and creation of a novel educational approach for engaging computer science and science students together in a collaborative classroom setting. Material artifacts from this
Collaborative Research Teams
185
workshop included the concept maps, an integrated conceptual workflow, and wiki notes about all of the areas of interest identified above. Several months later, a CI-Design Workshop was held to design and develop a specific research proposal around one of the identified research areas. This meeting involved a smaller subset of participants (10). The meeting resulted in a cross-disciplinary research design that was collectively generated and engaged all of the participants. During the CI-Design meeting participants sometimes referred to shared artifacts from prior meetings, and created new ones as needed. These artifacts were primarily freeform drawings on whiteboard that were either copied into electronic drawing software or were photographed with a digital camera. Near the end of the meeting a draft proposal outline was collectively generated through one person editing a wiki and projection of the text onto a screen. The final draft was individually edited until an agreed upon draft emerged. 2.6. Data analysis Artifacts were classified as individually or group constructed. The number of individual and group produced artifacts was counted. Artifacts from the CISeminar were excluded because only the students created them. Each remaining material artifact was visually inspected and categorized as: 1) boundary negotiating object, or 2) boundary specifying object. This categorization was based on how the artifacts were used by the group, following the distinction identified by Lee (2007). Boundary negotiating objects were used to “record, organize, explore and share ideas; introduce concepts and techniques; create alliances; create a venue for the exchange of information; augment brokering activities; and create shared understanding.” Conversely, boundary specifying objects merged different perspectives in ways that removed the need to interact across disciplines. Artifacts collected were assessed for analytical compatibility. Concept maps from the CI-Vision and CI-Strategy Workshop were matched for comparative analysis of individually produced artifacts. The vision statement from the CI-Vision Workshop was matched to the CI-Design proposal for comparative analysis of group produced artifacts. Artifact content was assessed through semantic analysis as a surrogate measure of cross-disciplinary conceptual integration. Terms were extracted from each artifact and classified as 1) unique to science language, 2) unique to information technology language, or 3) shared by both. The percentage of terms in each of these three classes for each artifact was tabulated. Pairwise comparisons of individual concept maps collected during the CIVision and CI-Strategy Workshops were conducted using functionality provided by CMap Tools. Only concepts in each map (nodes) were compared; connections (links) were excluded because many participants did not specify connections between concepts, or used semantically-neutral phrases such as “leads to”. Matches were determined through four mechanisms: 1) full text match, 2) partial
186
Deana D. Pennington
text match, 3) keyword, and 4) synonym. Keyword and synonym matches were determined based on thesauri and other resources internal to the software. All potential matches identified by the software were manually checked; erroneous matches were excluded. Matches based on seed terms provided during the CIStrategy workshop were excluded. Four concept maps were discarded from the analysis because it was obvious that they had collaborated on parts—sections of their maps were duplicated. A fifth—the one that was the most comprehensive— was retained. Because the number of terms used in each concept map varied, counts for a given pair differed depending on which concept map was considered the source and which the target. Therefore counts of matches were made in both directions. Pairwise comparisons of terms used in the concept maps were classified as a comparison between 1) two scientists, 2) two information technologists, or 3) one of each. This classification was further generalized as 1) within discipline comparison and 2) between discipline comparison. Basic statistics for each of these were calculated: number of samples, mean count of terms in common, median count, variance and standard deviation. The data were not normally distributed, contained unequal numbers of samples, and the occurrence of values of zero prevented log transformation. Therefore the non-parametric statistical rank sum test was used to analyze differences between groupings. Differences between groups through time were compared. Artifacts were again visually checked to verify that the findings highlighted by the statistical analyses were valid, and to qualitatively assess the ways and degrees to which artifacts not analyzed diverged from those findings. 3. Results The total number of artifacts produced during the project period was 111. 20 of these were produced by the group and 91 by individuals (Figure 4). Ignoring the CI-Seminar artifacts which were all produced by individuals by design, the number of artifacts produced by individuals decreased in number through time from 34 during the CI-Vision Workshop to 0 during the CI-Design working meeting. The number of artifacts produced by the groups varied through time, peaking mid-project. Basic statistical analysis of terms used in concept maps indicates that the number of overlapping terms between paired concept maps was higher for all disciplinary combinations during the CI-Strategy Workshop than during the earlier CI-Vision Workshop (Table 1). During both the CI-Vision and CI-Strategy Workshops, the highest observed number of terms in common was between pairs of scientists (CI-Vision mean = 1.6, median = 1 term in common; CI-Strategy mean = 5.7, median = 5 terms in common). Pairs of information technology specialists had fewer terms in common than pairs of scientists (CI-Vision Workshop mean = 1.2, median = 1 term in common; CI-Strategy Workshop
Collaborative Research Teams
187
Figure 4. Counts of the number of material artifacts produced by individuals and by the group in four consecutive activities.
mean = 3.8, median = 4 terms in common). Interdisciplinary pairs had the lowest number of terms in common during the CI-Vision Workshop (mean = 0.3 and 0.6, median = 0 terms in common). During the CI-Strategy Workshop they had fewer terms in common that pairs of scientists but more terms in common than pairs of Table 1. Basic statistics from semantic analysis of concept maps created during two activities. Within discipline
Between discipline
Comparison of within and between disciplines
SCI/SCI
IT/IT
SCI/IT
IT/SCI
90
20
50
50
1.6
1.2
0.3
0.6
1
1
0
0
2.6
1.9
0.4
1.5
30
6
18
18
5.7
3.8
4.6
4.4
5
4
4
4
17.1
1.8
8.4
4.3
CI-Vision Workshop Sample size Mean number of terms n common Median number of terms in common Variance CI-Strategy Workshop Sample size Mean number of terms in common Median number of terms in common Variance
Significant difference z=−6.25 P
