Computer Support for Distributed Asynchronous Software Design ...

Proceedingsof the 28th Annual Hawaii International Conferenceon SystemSciences- I995

Computer Support for Distributed Asynchronous Software Design Teams: Experimental Results on Creativity and Quality Rosalie Ocker

S. Roxanne Hiltz

Rutgers University Newark, NJ

Murray Turoff

Jerry Fjermestad

New Jersey Institute of Technology Newark, NJ One approach is to provide support for design through the use of group support systems [16,33,10]. In this paper, we report on an experiment which investigated the usefulnessof computer-conferencingto support design groups working (1) on the early stagesof design (2) in a distributed asynchronous environment. Asynchronous refers to the dimension of time -- at different times -while distributed refers to the dimension of space -- at different maces. Thus, members of distributed asynchronousgroups “meet” or communicate with one anotherat different times, from different places. The distributed asynchronous form of group interaction has obvious benefits. Group members do not have to physically be in the same place to meet, nor must they communicate with one another at the same time. These two characteristics of distributed asynchronous communication greatly extend the definition of what constitutesa meeting. These loosenedconstraints can add value to the organization by increasing the means with which groups can accomplish work. However, there are also disadvantages. Points of reference for indexing communication by time, place, and talk sequenceare all missing [26]. As communication occurs over time, days can elapse between communication events, resulting in communication that can seemdisjointed. Another coordination problem is finding workable substitutesfor temporal sequenceas an ordering mechanism. In an asynchronousdiscussion, the norm is for many topics and subtopics to be active at once, and sometimes more than one person makes entries simultaneously,perhapson different topics. How does the group order and coordinate what can seem chaotic? One approach for dealing with these problems of coordination is to (1) provide structuring mechanismsfor groups to use to organize their communication and (2) sequence the problem solving processfollowed by groups to accomplish work. Where creative solutions to difficult problems are the desired outcome, some degree of structure can be a stimulus to triggering ideas [ e.g., brainstorming]. However, too much structure, or the wrong structure, can

Abstract Previous research has indicated that the creative task of deciding upon the initial specijcations for a sojtware SySfem can benefit from a structured process to guide group interaction in Face-to-Face meetings, and can benejlt from computer support. This 2x2 experiment is the Jrst to look at the joint effects of a distributed asynchronous computer conferencing mode of communication (CC, as compared to Face to Face), and a structured process, on sojtware design outcomes. Subjects were teams of graduate students in computer science and business, who designed an automated post of&e as a course assignment. Though the quality of solution produced by the CC groups was judged to be higher, it was not significantly so. However, the CC groups were considerably more creative. There were no main effects for the structured/unstructured factor, nor an-v signi$cant interactions.

1: Introduction As the businessworld increasesin complexity, so does the nature of problems which information systems must support. Basic systemsdealing with structured and routine problems are a wave of the past. Today’sproblems are unstructured, nonroutine, and even wicked [61, 591. Within systems development, it is recognized that the stages of requirements definition and high-level design are important, and even crucial to the development of effective software [7]. Collaborative designersand users work to achieve some consensus on the general characteristics of the new system in question [51]. Decisions made during these initial stages drive the decisions made throughout the remainder of the developmentprocess. To provide effective support, it has been increasingly suggested that the development of information systemsin general, and the definition of highlevel requirements and design, specifically, could benefit from the infusion of creative and innovative solutions [e.g., 6,18,65].

4

1060-3425/95$4.00@1995IEEE

Proceedings of the 28th Hawaii International Conference on System Sciences (HICSS '95) 1060-3425/95 $10.00 © 1995 IEEE

Proceedings of the 28th Annual Hawaii International Conference on System Sciences - I995

exhibited patterns of design activity interspersed with discussions of management (e.g., coordination) issues. Meeting participants clearly stated issues, alternatives, and criteria for evaluating alternatives, in addition to managing the design process. It was found that group design discussions,although observedto be chaotic, when content analyzed, were actually found to have a natural structure and a regular pattern of activity.

limit the creative process. In the distributed asynchronous environment, structure becomes an important component in the organization and comprehensionof the group view [66, 301. While synchronous groups can often vary the degreeand type of structure dynamically as needed,this is more difficult for distributed asynchronousgroups that are dependent upon both structure and process rules for coordination. Hence, understanding in detail the tradeoff between structure and creativity becomes critical for the design of an asynchronousprocess. In this paper, we report on results of the first experiment conducted to investigate the usefulnessof the distributed asynchronous mode of interaction and communication for groups working on the early stagesof design. We also investigate the usefulnessof a structured problem solving approach for accomplishing this activity. The dependentvariables we report on are (1) creativity of design solution and (2) quality of design solution. Distributed asynchronous groups use a computer conference (CC) to conduct all of their work. These groups are referred to as CC groups throughout the remainder of this paper.

2.2: Group Creativity and Quality Design is an inherently creative process [21]. The end-productis the creation of an artifact - something that previously did not exist. Creativity at the group level is not merely the summation of each group members’ individual creativity. Group creativity is influenced by groun comnosition (e.g., diversity), grouu characteristics (e.g., cohesiveness),and groun urocesses(e.g., problemsolving strategiesand social information processes)[7 11. Group composition and group characteristics: West [70] proposesthat an atmospherethat is perceivedas “interpersonally non-threatening”can enhance creativity [p. 3 111. Thus, cohesive groups are more likely to be considered safe by group members. However, West arguesthat group homogeneityis an important attribute of cohesivegroupsbut that it can lead to groupthink [32] and thereby inhibit creativity. Nystrtim [50] advocates that loosely joined, heterogeneousgroups are needed during the early stagesof work on a problem in order to facilitate the production of creative ideas. Communication within the group is viewed as vital to effective performance on researchteams [44,56,68,71]. The sharing of ideas is a stimulus which can increase not only the quantity but the novelty of ideas. Minority infhrence theory [45,373 offers one explanation of how group interaction processesresult in a creative group outcome. Minority influence is a construct that describes the situation where minority opinion holders win-over and thus influence the opinions of the majority opinion-holders, It has been argued that minority influence leads to greater levels of group member independenceand to more divergent and creative thinking [46,47,48,49]. Problems are considered from a wider variety of perspectiveswhich leads to the development of new, more creative and higher quality solutions [55,49,41]. Hoffman [3 1] argues that a diversity of viewpoints within a group is essential for creativity, but that these differences of opinion must be tolerated. Minority influence researchsuggeststhat a consistent and confident presentation of arguments by the minority is most likely to lead to attitude change and creativity. Group Processes - social information and problem solving: Groups can draw on the knowledge and resourcesof the various group members in order to

2: Background 2.1: Design Activity The early stages of design include the determination of high-level requirements and high-level design. Early design work is categorizedas ill-structured [6 11,due to incomplete and ambiguous specification of the problem, no predetermined solution path, and the need to integrate multiple knowledge domains. In order to reach a design solution, there is a need to structure the design problem. The ability to structure the problem is a function of the previous experience of the designer. There is empirical evidence showing that individual designersuse a structured, top-down approach to accomplish design, where aspectsof the overall system are designedfirst, and then the system is progressively decomposed into increasingly greater levels of detail (1,34,35]. It appears that the use of a structured, top-down approach to design is appropriate when the problem is well-structured. Deviations from a structured approach have been observed when the problem is in some way novel or complex to the designer [24]. For ill-structured problems, a method or structure which locks the designer into a strict order of activities may be inappropriate. Considering design from the perspective of the group, Olson et al. [52] characterize design discussionsas a form of argumentation where various issues are raised and alternative design possibilities are discussed. In their field study of groups working freely on the early stagesof design, Olson et al. [52] found that design discussions 5


Proceedings of the 28th Annual Hawaii International Conference on System Sciences - 1995

place. We combine aspects of IBIS and DSA to (1) sequencethe processof design activity and (2) to structure the argumentation surrounding the various issues of the design process.

positively impact task performance. However, social information, consisting of verbal and non-verbal cues, influences perceptual, attitudinal and behavioral outcomes [cf. 22,231. It has been found that perceptions of unstructured problem-solving tasks which require high levels of creativity are more susceptibleto social infhtence than are structured routine tasks which do not require high levels of creativity [4]. Finally, the problem solving approach used by the group necessarily affects group interaction and therefore group creativity. For example, the process of brainstorming was designed to limit the evaluation of ideas being generatedand thus to encouragethe quantity of ideas, thereby increasing the likelihood of generating creative ideas.

3: Research framework, experimental design and hypotheses Hackman and Morris [25] propose that a major portion of variation in measured group performance is controlled by three general summary variables: (a) the knowledge and skills of group members, (b) performance strategiesused by the group in carrying out the task and (c) the amount of coordination and effort exerted by group members on the task. They argue that it is possible to substantially affect the level of group effectiveness by controlling or influencing these three summary variables, and that the relative importance of each summary variable is contingent upon the task at hand. For a task requiring creativity, the development of new strategies and increased member skill are of paramount importance. However, for CC communication, coordination of member effort is also very important. Each summary variable is impacted by the type of interaction such that either processgains or process losses [64] occur (see table 1). These gains and losses in turn impact the performance effectiveness (outcome) of the group. Group effectiveness can be improved above the level expectedresulting from processlossesby altering the input factors. Figure 1 is a modified version of the Hackman and Morris model, and highlights the independentvariables used in this experiment.

distributed asynchronous 2.3: Coordination, communication and problem solving approach Coordination is the set of tasks and processesby which groups of actors carrying out activities manage interdependencies,in order for them to perform effectively as a group. Coordination can be viewed in terms of actors performing interdependent activities aimed at achieving goals [40]. Empirical findings indicate that actual design activities account for less than half of the meeting time of unsupported face-to-face design teams working on the early stages of design, while coordination activities required at least 20% of the meeting time [52]. Although the needfor coordination is inherent in all group problem solving activities, as previously described, distributed asynchronous groups have additional and unique coordination problems beyond those experiencedby faceto-face groups. To address coordination problems of distributed asynchronous communication associated with temporal sequencingand ordering of communication, we developed a rational problem solving approach incorporating highly structured communication. Given the successfuluse of a design rationale for accomplishing design [72] and for encoding actual design activity [52], we combined two specific design rationale approaches: 1) Issue Based Information System (IBIS) originally developedby Kunz and Rittel [38] for use in argumentation and 2) Design Space Analysis, developed by MacLean, Young, Bellotti, and Moran [39]. IBIS was developedfor the purpose of supporting argumentation by explicitly structuring communication for both pro and con ideas regarding the generation of various alternative solutions. IBIS has been implemented to document design meetings, as a historical record of the problem solving and decision making process [72]. Design Space Analysis (DSA) is used durinp the urocess of design, to actively build the structures of argumentation as design discussions take

Table 1 Summary of Proposed functions of group interaction 1. member knowledge and skills process losses - interaction is less-than-perfect meansfor applying member talents to the task potential gains - interaction is means for generationof new knowledge or skill by members 2. performancestrategiesfor carrying out the task process losses - interaction serves to implement less-than-perfectpreexisting strategies potential gains - interaction serves to reformulate strategic plans to increase their taskappropriateness 3. level and coordination of member effort processlosses- interaction is less-than-perfect means to coordinate member effort potential gains - interaction enhances coordination and member effort

6



Figure 1. Theoretical Framework

l

l

Independent Variables Mode of Communication - FtF -cc Problem Solving Approach -Structured -Unstructured

rj

Performance Effectiveness

71 l

process

Member Knowledge

0 Performance Strategies 4 Coordination of Member Effort

/-

l

-

l

Quality Creativity

Figure 2. Impact of Summary Variables on Quality and Creativity Reduced Social Information

Uninhibited+ Behavior

+

Equal Information Sharing Member Knowledge Decision Participation + Increased ----) Increased -----) Quality Increased

Loosely Joined Groups

-of

Diversity + Conflict + Consistent Opinion Behavior

Our experiment uses a 2 X 2 factorial design. Mode of communication compares groups communicating face-to-face(ft$ with those communicating in a Cc mode using a computer conferencing system. The problem solving approach compares groups who followed a highly structured and imposed process to unstructured groups having no imposed process. We use the model in figure 1 to frame our hypotheses,and draw on empirical research comparing computer-mediated communication versus unsupported face-to-face groups, along with the previously cited background research, in developing our hypotheses. We make the following speculations, which are depicted in figure 2. CC groups will feel less threatened than face-to-face groups due to reduced social cues [63]. Coupled with more equal participation [9,12,15,27,60,69], CC groups will experienceincreasedinformation sharing. This will positively impact the level and amount of member knowledge (summary variable), resulting in problem solutions of higher quality. Evidence of more uninhibited behavior in individuals communicating electronically [ 12, 36, 60, 63, 691 points to the likelihood that members of CC groups Additionally, will feel less pressure to conform. distributed groups are by their nature loosely joined, and therefore not likely to be as cohesive as their face-to-face counterparts [28]. These characteristics coupled with

New Performance Creativity Strategies -increased

more equal participation should lead to the sharing of more diverse opinions within the group [58]. Diverse opinions lead to divergent thought [54,19,20,67] and conflict [57]. Although these conditions will increase the probability for a creative solution, it is necessarythat the minority opinion holder(s) exhibit consistent behavior patterns over time in order to have influence over the majority opinion holders [45]. CC groups have an increased opportunity for communication, and more time/less pressure when developing and presenting their opinions. They must work harder and communicate longer [3,9,15,17,69]. Therefore, we speculate that minority opinion holders in the CC groups will be perceived as more consistent than their face-to-face counterparts, and thus be more likely to exert influence. In fact, it has been found that groups communicating electronically experienced higher amounts of minority influence than unsupported face-to-face groups [43, 51. Minority influence should increase the chances for the development of new performance strategies (summary variable) and therefore novel solutions to the problem, resulting in increased levels of creativity [46,47,48,49]. Thereforewe hypothesizethe following main effects: HI. H2.

CC groups will produce solutions oJhigher quality than face-to-face groups. CC groups will produce more creative solutions

than face-to-face groups.


Proceedings of the 28th Annual Hawaii International Conference on System Sciences -

Face-to-face communication and interaction is much richer than asynchronous communication [S]. Although the problem solving structure was chosendue to its capability to structure communication which includes a diversity of opinions and also for its fit with the activity of design, it has been argued that face-to-facedesign groups do not need this added coordination [52]. It is suggested that the activity of high-level design has its own inherent structure naturally embedded within the task, and that adhering to an IBIS derivative of argumentation would impede the productivity of face-to-facegroups working on design. However, we argue that the opposite is true for CC groups; coordination is difficult [17,62] and some structure [3,13] is required to easethe cognitive burden of distributed asynchronous groups. Due to the need for differing amounts of coordination (summary variable) between the face-to-face and CC groups, we predict that there will be an interaction effect such that: H3.

CC structured groups and face-to-face unstructured groups will produce solutions of higher quality than CC unstructured groups and face-to-face structured groups.

4: Method There are a total of 41 groups included in this study, distributed as follows: cc Face-to-face 10 11 Structured 10 10 Unstructured Task The Automated Post Office (APO) is the task used in this experiment. Groups were required to reach consensuson the initial requirements of the APO and to submit these requirements in a formal reuort at the end of the experiment; each group produceda single report. The report was to cover the functionality of the APO along with implementation considerations and was also to contain a description of the user interface design. This is a modification of the same task used by Olson et al. [1993], with an added emphasis on the design of the user interface. Olson et al. characterize this task as incorporating Planning, Creativity, Decision-Making, and Cognitive Conflict [42]. Subjects consisted of graduate students Subjects in the CIS and MIS majors at the New JerseyInstitute of Technology (NJIT) and MBA students from Rutgers University. For their participation, all subjects received course credit. The majority of subjects had coursework and/or job experiencedirectly relevant to systemsdesign. Groups were assigned to conditions so that each condition was balanced according to academic major. Group size ranged from 4 to 7 personsfor all groups with

I995

the exception of one group which contained only 3 members. Average group size was 5.2 people. Subjects were scheduledto meet basedon availability for scheduled sessions. Structured Approach The groups in the structured conditions followed a sequenceof steps contained in three main phases: (1) Generation of alternatives: Each group member individually develop his or her own design alternative of an automated post office. There is no group interaction during this phase, i.e., each person works alone. The outcomeof this phase is the individual design alternatives - one design alternative per group member. (2) Period of critical reflection and individual evaluation of alternatives: Each group member reviews other members’ design alternatives, making pro and con comments on each alternative. Pro comments are comments in support of various aspects of a design alternative; con comments are comments critiquing aspects of a design alternative. These comments are attachedto the alternative so that all members have access to them. The outcome of this phase is the individual evaluation of each design alternative. (3) Group evaluation of alternatives and consensus reaching: The goal of each group is to reach consensuson the contentsof the final design and write the formal report containing details of the design. Three steps are completedin this phase: Each group member individually reviews (4 other members’design alternatives and pro and con comments, noting aspects of each alternative he or she feel should be included in the formal design. The group develops a list containing (3) possiblealternatives. The group uses this list to debate and Cc) discussthe design alternatives to reach consensuson the formal report. Steps(b) and (c) are iterative and can be repeatedas many times as necessary,within the allotted time-frame. The outcome of this phase is the formal report describing the group’s design of the automatic post office. Technology and Facilitation The CC groups communicated using the EIES 2 computer conferencing systemdevelopedat NJIT. Each CC group communicated in its own conferenceset up on EIES 2. The conferences were minimally facilitated. The conference facilitator’s role was that of a technical assistant, helping groups with equipment problems and answering questions of a technical nature. Training The experiment lasted two weeks. All groups met face-to-face for training and used the same practice problem, called Entertainment for Dutch Visitors [53]. CC groups were trained on the communication



1995

groups produced two of the longer reports for this condition. All groups had a leader who volunteered for the role at the end of the training session, sometimes with encouragementfrom the facilitator. For the face-to-face groups, the leader was only responsible for ensuring that any materials generatedin sessionone were brought to the second session. For the CC groups, the leader was only responsible for ensuring that the formal report was submitted on EIES 2 by the end of two weeks. All groups completed a task and post-experiment questionnaire. For each face-to-face group, the facilitator completed an observation form at the end of the second session,rating the group on such aspectsas general group atmosphere and type of coordination used. It was also noted whether the group completed their work early, or had to rush to finish the assignment. Debriefing Face-to-facegroups were debriefed in a face-to-facesession. CC groups were either debriefed in a special on-line conferenceor in a face-to-facesession. All participants in the face-to-faceconditions were questioned regarding their adherenceto the rules for communication outsidethe two sessions. Measures of the Dependent Variables Overall quality of solution was measured by a panel of three expert judges, all of whom had academic and/or professional experience in systems design. All groups’ formal reports were printed using the same word processing package and aspects of each group’s mode of communication were masked. The judges rated each group on various aspectsof the design (e.g., functionality and interface), managerial issues,written presentationand overall quality of the analysis [53]. The level of creativity contained in each group’s design was also measured by the panel of judges. According to Amabile , a “product or responseis creative to the extent that appropriate observers independently agree it is creative” [2:359]. Additionally, there is no agreement as to the appropriate subcategoriesto use in order to rate creativity. Therefore, we did not provide the expert judges with explicit details. Rather, we instructed them to rate the creativity of each group using the general categoryof “Creativity of Solution.”

features of EIES 2 while the structured groups were trained on the steps in the approach. (Thus, the CC processgroups were trained on both the use of ElES 2 and the structured approach). Training was completed within two hours for CC groups, after which they departed to work on the APO task on-line. Training for face-to-face groups was completed within one hour. The face-to-face groups remained together to begin work on the APO task, For data collection purposes,all participants completed a background questionnaire. The CC structured groups followed a Procedures strict schedule. Phase 1 was to be completed within two days after training and phase two within five days after training. Phase three, steps (a) and (b) were to be completed within 11 days after training, while step (c) was to be completed within 14 days from the training date. The CC unstructured groups were not given any processor structure to follow. They were simply told that by the end of the two-week period, their group report had to be submitted on-line. All CC groups were explicitly instructed to communicate only within their respective computer conference. Face-to-facegroups met for two sessions,spaced exactly two weeks apart. In session one, members of the face-to-face structured groups had one hour to complete their individual design alternative (phase 1 of the imposed process). At the end of the session,the design alternatives were collected by the facilitator and copies were made. One copy of each alternative was then mailed to each group member on the day following sessionone. During the two-week interim period, each group member completed phase two of the structured approach. Group members brought pro and con comments back with them to session two, where they were read by each group member. In session two, groups had 2-l/2 hours to work on phase3 of the structured approach. The face-to-face unstructured groups worked on the APO task for one hour in session one. Group members were permitted to work independently on the APO task during the two-week interim betweensessions1 and 2; they were instructed not to communicate with fellow group members during this time. Session two consistedof the groups working on the APO task for 2-l/2 hours. The content and layout of the formal report was determined by each group. Most face-to-facegroups were provided with a word processor to use in the second session,at their discretion, in order to write their report. Two groups were unable to use a word processordue to technical problems. Their reports were handwritten and later transcribed using a word processor. This did not seem to negatively effect their performance, as both

5: Results Analysis of variance for an unbalanceddesign was performed to detect significant effects. Table 2 presentsthe descriptive statistics along with the ANOVA values for the dependentvariables. The panel of three expertjudges had an acceptablelevel of agreementwhen rating overall quality of solution (Chronbach’salpha = .74) and creativity of solution (Chronbach’salpha = .78). 9



TABLE 2 - Mean (Standard QUALITY Structured Unstructured Column Means ANOVA Source CC/FtF Structured/Unstruct. Interaction CREATIVITY Structured Unstructured Column Means ANOVA Source CC/FtF StructuredlUnstruct. Interaction

cc 7.33 7.27 7.30 DF 1 1 1 cc 6.46 6.60 6.53 DF 1 1 1

(1.80) (1.56)

(1.91) (1.92)

FtF 6.03 6.80 6.41 Tvoe 8.01 1.26 1.79 FtF 5.06 4.63 4.85 Tvoe 29.10 0.22 0.8.

Deviation)

& ANOVA

for Dependent

1995

Variables

Row Means 6.68 7.03

(1.70) (0.88) 3 SS

(1.831 (0.961 3 SS

Intercorrelations among expert-rated dependentvariables were computed. The level of association betweenoverall quality and creativity (R=.68) indicates that they are related but distinct dimensions in the judges’minds. In discussing results, significance levels of .05 or better will be considered “statistically significant.” Levels between .lO and .05 indicate findings that suggest a relationship may exist. Since this is the first experiment of its type, such findings are worthy of note for further study. Qualitv: Overall quality was consistently higher for the CC groups; however, the significance level was only .07. There was no main effect for the structured problem solving factor, nor any interaction effects. For the CC groups, overall quality was almost exactly the samefor the structured and unstructured groups. For the face-to-face groups, though the means for unstructured groups are higher, as predicted, the difference is not significant. When quality ratings are examined according to their components, the only components for which the differences are large and consistent enough to approach statistical significance are those concerned with the system “design” and “interface.” Aspects of the groups’ reports dealing with managerial issues (personnel requirements, cost/benefit ratios, etc.) or with the quality of the written presentation itself show practically no differences. Creativitv: The CC groups were considerably more creative than the face-to-face groups (p=.OO3). There the differences associated with were no structured/unstructuredfactor, nor any interactions.

E 3.42 0.54 0.76 Row Means 5.76 5.62

e .07 .47 .39

R-SO =.12 L

E 10.00 0.08 0.28

!? .003 .78 .60

R-SQ =.22 d

6:

Summary, Discussion and Conclusions

In sum, the results are: Hl, not supported: The quality of solution produced by the CC groups was not judged to be significantly better than that of the face-to-facegroups (p=.O7). H2, supported: The solutions produced by the CC groups were judged to be significantly more creative than those producedby the face-to-facegroups @=.003). H3, not supported: There was no interaction effect betweenmode of communication and the structure of the problem solving approach. The main objective of this study was to explore the effects of distributed asynchronous communication and problem solving on software design teams working on a creativity task. As hypothesized, the CC groups significantly outperformed the face-to-face groups, as rated by judges on the dimension of creativity. “Supporting creative group problem solving requires that the group be provided an environment and tools which help minimize the blocks to creativity of members, enhances creative traits and skills of individuals, and enablesthem to proceedthrough creative problem solving processes”[ 14:290]. Evidently, the act of communicating asynchronously in a dispersed setting over a period of time enhancescreativity. Whv was there no interaction between mode of communication and the nroblem solving approach on measures of aualitv? The structured problem solving approach was largely based on the IBIS method which was developed to support the process of argumentation and conflict resolution resulting during the analysis of illstructured, complex problems. The imposed process was

10



hypothesized to benefit the CC groups by providing the added coordination which is missing in this form of communication. We can offer several speculations why this did not occur. Concerning the design of the APO, a strong metaphor is available in the form of automatic teller machines. This might have reduced the need for coordination - instead of the problem being ill-structured, perhaps it was only semi-structured or even wellstructured. If this was the case, then groups already had a good idea of how to proceed to the solution, so that the needfor coordination would be greatly reduced. A second possibility is that IBIS-related methods are simply not the appropriate means for structuring and coordinating distributed asynchronous computer conferences, as comparedto face-to-facemeetings. Is there a relationship between distributed asvnchronous communication and aualitv? The results were not significant (p=.O7). However, the findings suggest that there is a positive relationship between distributed asynchronous communication and quality of solution, as groups in both CC conditions were rated higher than their face-to-facecounterparts. Aspects of our study can be compared to those of Olson et al. [53] who found that synchronous groups supported by a collaborative writing tool performing design work produced reports of higher quality than unsupported groups. Although the use of a joint editor in a “decision room” may seem completely different than the use of a computerized conferencing system in a distributed mode, in fact, there are some similarities in terms of the effects on group process and thus on group outcome. Both computer systems allowed work in parallel, and then the combining of written ideas. Perhapsthe aspectsof groups working in parallel and producing a written memory, by helping to reduce the number of lost ideas, positively impacts quality. Perhapsthe increasedconnectivity of the CC groups also positively impacts quality. Maybe if the CC groups had not experienceddifficulty trying to get into a busy, over-loaded system (see Limitations of Study, next section), quality would have been significantly better.

I995

problems will be resolved in the near future, and replications of the distributed asynchronous conditions with improved system features and access might well result in significantly better quality. Methodological limitations: By its very nature, it is impossible to assure full control of the behavior of distributed asynchronous groups. The group members could ignore the instructions limiting their communication. However, direct questioning of face-toface groups during debriefing, and daily observationof the CC groups, did not indicate any evidence of circumventing the experimental manipulation. A design limitation was that we did not explore the relative effectivenessof different modes of designating a leader or motivating and defining the leaders’role. Because the “leaders”in this experiment had only a very limited role, we do not think it would make much difference whether they volunteeredor were electedor appointed. Future Research: Additional communication conditions would strengthen our ability to draw conclusions. For instance, face-to-face groups were restricted from communicating with one another outside of the two meetings. Allowing these groups to communicate via the telephone would afford them the opportunity to stay connected throughout the experiment. A second, more equivocal design task is worthy of exploration. Also, we plan to add a synchronous computer conferencing condition. With the inclusion of this condition, we could more definitively conclude whether distributed asynchronous communication fosters creativity, rather than computer-mediated communication in general. At this point, we have a firm foundation from which to further investigate the relationship between mode of communication and creativity for systems design tasks. REFERENCES 1.

Adelson, B. and Soloway, E. 1985. The role of domain

experience in software design. IEEE Transactions on Engineering, 11:135l-1 360. 2. Amabile, T. M. 1983. The social psychology of creativity: A componential conceptualization. Journal of Personality and Sofiare

Limitations of studv: Svstem Limitations: One limitation of this study has to do with shortcomings of EIES 2 at the time of this study (the 1993-94 academic year) The current default text editor on ElES 2 is a line editor, rather than a full screen editor which can be very frustrating to use. Also, the interface was command driven, with no graphic representation. The structure imbedded in EIES is hierarchical, and as a result, can be difficult to maneuver around in. Additionally, access to EIES 2 can be very difficult in the evening during the peak hours of 7-1 I p.m. Numerous complaints were received about subjects’ inability to connect onto the system. Hopefully, these

SocialPsychology,

45(2):357-376.

3. Arunachalam, V. 1994. Computer-mediated communication and structure interaction in transfer pricing negotiation. Journal of Information Systems.

4. Bateman,, T.S., Griffin, R.W., L Rubinstein, D. 1987. Social information processing and group-induced shifts in responseto task design. Group & Organization Studies, 12: 88-108. 5. Clapper, D.L., McLean, E.R., Watson, R.T. 1991. An experimental investigation of the effect of a group decision support system on normative influence in small groups. Proceedings of the Twevth ICIS, 273-282. 11



6. Couger, J.D., Higgins, L.F., &McIntyre, SC. 1993. (Un)structured creativity in information systems organizations.MIS Quarterly, December375397. 7. Curtis, B., Krasner, H., & Iscoe, N. 1988. A field study of the software design process for large systems, Communicationsofthe ACM, 31:1268-1287. 8. Daft, R.L. and Lengel, R.H. 1986. Organizational information requirements, media richness and structural design, Managemenf Science, vol. 32(5):554-571, 9. Daly, B. 1993. The intluence of face-to-face versus computer-mediated communication channels on collective induction. Accounting, Management & Information Technology, 3( 1): l-22. 10. Daniels, Jr., R.M., Dennis, A.R., Hayes, G., Nunamaker, J.F., Valacich, J. 199I. Enterprise analyzer: electronic support for group requirements elicitation. Proceedings 0s the Twenty-fourth Hawaii International System Sciences, vol. 3,43-52.

2 1. Glass, R. and Vessey, I. 1994. Software tasks: Intellectual, clerical.. .or creative? Proceedings of the Twenty-Seventh Hawaii International Conference on System Sciences, vol. 4:377-382. 22. Griffin, R.W. 1983. Objective and social sources of information in task redesign: A field experiment. Administrative Science Quarterly, 28: 184-200. 23. Griffin, R.W., Bateman, T.S., Wayne, S.J., k Head, T.S. 1987. Objective and social factors as determinants of task perceptions and responses:An integrative perspective and empirical investigation. Academy of Management Journal, 30:501-523. 24. Guindon, R. 1990. Designing the design process: Exploiting Interaction, opportunistic thoughts. Human-Computer 5:305-344. 25. Hackman, J. R.L Morris, C. G. 1975. Group tasks, group interaction processes,and group performance effectiveness: A review and proposed integration. In L. Berkowitz (Ed.),

Conference on

11. Dennis, A.R. and Valacich, J.S. 1993. Computer brainstorms: More heads are better than one. Journal of AppliedPsychology, 78(4):531-537. 12. Dubrovsky, V.J., Kiesler, S., and Sethna, B.N. 1991. The equalization phenomena: Status effects in computermediated and face-to-face decision making groups. Journal of Human-Computer Interaction, 6:119-146. 13. Dufner, D., Hiltz, S., and Turoff, M. 1994. A preliminary analysis of the effects of the use of voting tools and sequential processing. Proceedings of the Twenty-Seventh Hawaii International Conference on System Sciences, vol. 4:114-123. 14. Duncan, N. and Paradice, D. 1992. Creativity in GDSS: An integration of creativity, group dynamics, and problem solving theory. Proceedings of the Twenty-F$th Hawaii International Conference on System Sciences, vol. 4:277287. 15. Eveland, J.D. & Bikson, T.K. 1989. Work group structures and computer support: A field experiment. ACM

Advances in experimental social psychology, 8~47-99. New

York: Academic Press. 26. Hiltz, S., Dufner, D., Holmes, M., and Poole, S. 1991. Distributed group support systems: Social dynamics and design dilemmas. Journal of Organizational Computing, 2(1):135-159. 27. Hiltz, S., Johnson,K., Aronovitch, C. and Turoff, M. 1980. Face-to-face vs. computerized conferences: A controlled experiment. Research Report Number 12, volume 1, New JerseyInstitute of Technology, Newark, NJ. 28. Hiltz, S., Johnson, K., & Turoff, M. 1986. Experiments in group decision making, 1: Communications process and outcome in face-to-face vs. computerized conferences. Human Communication Research, 13(2):225-252. 29. Hiltz, S. Turoff, M., & Johnson, K. 1988. Experiments in group decision making, 3: Disinhibititon, deindividuation, and group process in pen name and real name computer conferences.Decision Support Systems, 5: 1- 16. 30. Hiltz, S. and Turoff, M. The Network Nation, Cambridge: MIT Press, 1993. 3 1. Hotfman, 1961. Conditions for creative problem solving.

Transactions on Ofice Information Systems, 6,4:354-379.

16. Finholt, T., Sproull, L., & Kiesler, S. 1990. Communication and performance in ad hoc groups. In J. Galegher, R. Kraut, & C. Egido (Eds.), Intellectual teamwork: Social and technologicalfoundation of cooperative work. Hillsdale, NJ: Lawrence Erlbaum. 17. Galegher, J. and Kraut, R. 1990. Computer-mediated communication for intellectual teamwork: A field experiment in group writing. Proceedings of CSCW 90, November 1990,65-78. 18. Galletta, D.F., Sampler, J.L., & Teng, T.C. Strategies for integrating creativity principles into the system development process. Proceedings of the Twenty-F$th Hawaii International

Journal of Psychology, 521429-444. 32. Janis, I.L. 1982. Groupthink, 2nd ed. Boston: Houghton

MitIIin. 33. Jarvenpaa,S, Rao, R. & Huber, G. 1988. Computer support for meetings of medium-sized groups working on unstructured problems: A field experiment. MIS Quarterly, 12:645-665. 34. Jeffies, R., Turner, A., Polson, P., & Atwood, Me E. 198I. The processesinvolved in designing software. In J. R. Anderson (Ed.), Cognitive skills and their acquisition, Hillsdale, NJ: Lawrence Erlbaum. 35. Kant, E., and Newell, A. 1984. Problem solving techniques for the design of algorithms. Information Processing ad Management, 28:97-l 18. 36. Kiesler, S., Zubrow, D., Moses, A. & Geller, V. 1985. Affect in computer-mediatedcommunication: An experiment in synchronous terminal-to-terminal discussion. Human Computer Interaction, 1:77-104.

Conference on System Sciences, vol. 4:268-

276. 19. Gallupe, R.B., Bastianutti, L.M., t Cooper, W.H. 1991. Unblocking brainstorms. Journal of Appfied Psychology, 76(1):137-142. 20. Gallupe, R.B., Dennis, A.R., Cooper, W.H., Valacich, J.S., Bastianutti, L.M., & Nunamaker, J.F. 1992. Electronic brainstorming and group size. Academy of Management Journal, 35:350-369. 12



I995

56. Payne, R. 1990. The effectiveness of research teams: a review, In M.A. West and J.L. Farr (Eds.) Innovation and Creativity at Work, Chichester: Wiley & Sons. 57. Poole, M.S., Holmes, M., & DeSanctis, G. 1991. Conflict managementin a computer-supportedmeeting environment.

37 King N. and Anderson , N., 1990. Innovation in working groups. In M.A. West and J.L. Farr (Eds.) Innovation and Creativity at Work, Chichester:Wiley and Sons, 38 Kunz, W. and Rittel, H. 1970. Issues as elements of information systems. Working paper no. 131, Institute of Urban and Regional Development, llniversity of California at Berkeley. 39. MacLean, A., Young, R., Bellotti, V. and Moran, T, 1991. Questions, options, and criteria: elements of design space analysis. Human-Computer Interaction, 6(4):201-250. 40. Malone, T. and Crowston, K. 1990. What is coordination theory and how can it help design cooperativework systems? CSCW 90 Proceedings, pp. 357-370. 41. Maass, A. and Clark, R.D. 1983. Internalization versus compliance: differential processesunderlying minority influence and conformity. European Journal of Social Psychology, 13:197-215. 42. McGrath, J. 1984. Groups: Interaction and perfbnnance, Prentice Hall, Englewood Cliffs, NJ. 43. McLeod, P. L. and Elston, SE. 1994. Minority opinion and private opinion change: moderating effects of GSS use and anonymity. Paper presented at Academy of Management Meeting. 44. Meadows, I.S.G. 1980. Organic structure and innovation in small work groups. Human Relations, 33:369-382. 45. Moscovici, S. & Nemeth, C. 1974. Social Influence II: Minority influence. In C. Nemeth (Ed ) Social Psychology: Classic and contempora y integrations. pp.217-249. Chicago: Rand McNally. 46. Nemeth, C.J. 1986. Differential contributions of majority and minority influence. Psychological Review, 94: 23-32. 47. Nemeth, C.J. & Chiles, C. 1988. Modeling courage:the role of dissent in fostering independence.European Journal of Social Psychology, 18:275-280. 48. Nemeth, C.J. & Kwan, J. 1987. Minority influence, divergent thinking and the detection of correct solutions. Journal ofApplied Social Psychology, 17:788-799. 49. Nemeth, C. J. and Wachtler, J. 1983. Creative problem solving as a result of majority vs. minority influence.

Management Science, 37(8):926-953.

58. Rao, VS. & Jarvenpaa, S.L. 1991. Computer support of groups: Theory-based models for GDSS research. Management Science, 37: 1347-l 362. 59. Rittel, H. 1972. On the planning crisis: systems analysis of the first and second generations. Institute fur Grundlagen der Planung LA., Universitat Stuttgart; Stuttgart, Germany. 60. Siegel, J., Dubrovsky, V., Kiesler, S., & McGuire, T. 1986. Group processes in computer-mediated communication. Organizational

Behavior

& Human Decision

Processes,

37:157-187. 61. Simon, H.A. 1973. The structure of ill-structured problems. Artificial Intelligence, 4: 145-l 80. 62. Smith J. Y. and Vanecek, M. T. 1990. Dispersed group decision making using nonsimultaneous computer conferencing:A report of research.Journal ofManagement Information Systems, 7(2):71-92. 63. Sproull, L. & Kiesler, S. 1986. Reducing social context cues: Electronic mail in organizational communication. Management Science, 32( 11): 1492-l 5 12. New 64. Steiner, I. V. 1972. Group Process and Productiviv, York: Academic Press. 65. Telem, M. 1988. Information requirements specification brainstorming collective decision-making approach. Information Processing and Management, 24(5):549-566.

66. Turoff, M. Computer-mediated-communicationrequirements for group support 1991. Journal of Organizational Computing, 1(1):85-l 13. 67. Valacich, J., Paranka, D., George, J. & Nunamker, J. 1993. Communication concurrency and the new media: A new dimension for media richness. Communication Research, 20(2):249. 68. Visart, N. 1979.Communication between and within researchunits. In F.M. Andrews, (Ed.), Scientific Productivity, Cambridge University Press, pp. 223-25 1, 69. Weisband,S. 1992. Group discussion and first advocacy effects in computer-mediated and face-to-face decision making groups. Organizational Behavior and Human

European Journal ofSocial Psycholog): 13: 45-55. 50. Nystrom, H. 1979. Creativity and innovation. New York:

Wiley. 51. Olson, G.M., and Olson, J.S. 1991. IJser-centereddesign of collaboration technology.Journal of (3rganizational Computing, I:6 l-83. 52. Olson, G.M., Olson, J.S., Carter, M., and Storrosten, M. 1992. Small group design meetings: An analysis of collaboration. Human Computer interaction, 7(4):347-374. 53. Olson, J.S., Olson, G.M., Storrosten, M. and Carter, M. 1993. Groupwork close up: A comparison of the group design process with and without a simple group editor. ACM Transactions on Ofice Infbrmation Systems, 11(4):32 l-348 54. Osbom, A.F. 1953. Applied Imagination, New York: Scribner’s. 55. Parmeter, S.M. and Gaber, J.D. 1971.Creative scientists rate creativity factors. Research Management, November, 65-70.

Decision Processes, 53,352-380.

70. West, M. A. 1990. The social psychology of innovation in groups. In M.A. West and J.L. Farr (Eds.)Znnovanonand Creativity at Work, Wiley and Sons: Chichester. 71. Woodman, R., Sawyer, J. & Griffin, R. 1993. Toward a theory of organizational creativity. Academy of A4anagement Review, 18(2):292-32 1.

72. Yakemovic, K.C. Burgess and Conklin, E.J. 1990. Report on a development project use of an issue-based information system. CSCW 90 Proceedings, pp. 105-I 18.

13
