Information & Management 41 (2004) 399–411
Integrating nominal group technique and joint application development for improved systems requirements determination Evan W. Duggana,*, Cherian S. Thachenkaryb a
Information Systems, Statistics, and Management Science, Culverhouse College of Business Administration, University of Alabama, 367 Alston Hall, Tuscaloosa, AL 35487-0226, USA b Department of Decision Sciences, College of Business Administration, Georgia State University, University Plaza, Atlanta, GA 30303-3083, USA Received 24 January 2002; received in revised form 29 January 2003; accepted 27 June 2003
Abstract Joint application development (JAD) is a facilitated group technique that can be used in systems requirements determination (SRD); it was designed to encourage team rapport and achieve synergy by leveraging the combined knowledge of participants. JAD has been reported to cure several problems of conventional SRD techniques and shortened development schedules. However, its freely interacting meeting structure may curtail effectiveness by encouraging adverse group-related actions that challenge even the best facilitators. In this study, we integrated JAD and nominal group technique (NGT), a popular group structure that has been used to reduce the effects of negative group dynamics on task-oriented objectives. We examined this integrated structure in a laboratory experiment to determine whether it could alleviate the problems that JAD has experienced during SRD. The results suggest that the integrated approach outperformed JAD in our test environment; it was as efficient as JAD alone and it appeared to contribute to the reduction of the need for great facilitation skills in group decision-making. # 2003 Elsevier B.V. All rights reserved. Keywords: Communication medium; Group support methodology; Human factors; Information systems development; JAD; Meeting facilitation; Nominal group technique; Systems requirements determination
1. Introduction While many organizations have developed successful information systems (ISs), several have expended large amounts on software products that failed to deliver expected benefits. For many successfully deployed systems, maintenance activities have consumed *
Corresponding author. Tel.: þ1-205-348-7688; fax: þ1-205-348-0560. E-mail address:
[email protected] (E.W. Duggan).
a disproportionate share of systems development resources. A large part of this problem can be traced to miscommunication about required systems features and misunderstanding among system users and builders during the systems requirements determination (SRD) [4,39]. For this reason, SRD has been acknowledged as a critical determinant of system development success or failure [16,22,55]. Yet low quality systems requirements are common [17,30,37,46]. The facilitated group process called joint application development (JAD) was instituted, primarily, as
0378-7206/$ – see front matter # 2003 Elsevier B.V. All rights reserved. doi:10.1016/S0378-7206(03)00080-6
400
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
an alternative to the conventional interviewing technique for determining systems requirements [29]. A successful JAD process may reduce the communication barriers to effective requirements elicitation and analysis and eventually help to improve the quality of the final system [20]. However, JAD groups have experienced problems usually associated with the freely interacting meeting structure [24]. In this arrangement, communication takes place between group members with minimal control being applied by the meeting structure [74]; hence social and emotional dynamics often obstruct the process [43]. The nominal group technique (NGT), in which individuals work in the presence of others without significant interaction [10], has been used to cure many of the problems that freely interacting groups encounter; they include ineffective idea generation, unequal participation in discussions, groupthink, domination by more powerful participants, and ineffective conflict resolution [11]. In our study, we examined the effects of integrating NGT and JAD (during the JAD workshop) on the communication problems that occur during user–developer interactions in SRD. We contend that, while JAD has contributed to an improvement in the quality of requirements generated by the conventional (interviewing) technique, its intended benefits are often reduced by the problems faced by freely interacting groups [50]. An integration of JAD with NGT (NJAD) is expected to improve the generation of requirements as well as consensus and agreement on them and restore the group process loss with JAD alone. Our study is an extension of previous research that was geared toward improving JAD’s performance. However, unlike some of those studies, it does not require the acquisition of group support systems (GSSs) technology.
2. Background, research model and hypotheses 2.1. JAD—strengths and weaknesses JAD is superior to the interviewing technique and takes significantly less time [7]. During a JAD session a trained facilitator helps system developers and users pool their knowledge to establish system requirements. The facilitator stimulates effective user–developer interactions to generate ideas about
system features and helps to speed up decisionmaking [23]. JAD and its many derivatives have become increasingly popular in systems development and other organizational, decision-making contexts [5,41]. It is considered best practice for fostering user commitment [72]. JAD is often used with rapid application development (RAD) [67], an iterative and incremental approach for accelerating ISs delivery, and the dynamic systems development method (DSDM), the RAD-based technique used extensively in the UK [9,12]. It is considered a risk-reducing investment in ISs development [48]. JAD, however, relies heavily on facilitation to guide the meeting toward the attainment of its objectives [1]. Effective JAD facilitation is essential to encourage group productivity, resolve conflict, and minimize problems that are usually associated with freely interacting groups [73]. Kettelhut [44] identified several potential group problems that have affected JAD sessions. These overlap with similar difficulties that Nunamaker et al. [60] claimed contributed to process loss, resulting in the failure of groups to realize their potential. The JAD literature reminds facilitators of the importance of ‘‘chilling the dominator,’’ minimizing disruptive group behavior, and resolving conflict [6]. It contains several prescriptions for encouraging constructive interactions that are necessary for effective outcomes [68]. These suggestions typically do not include recommendations for how they should be accommodated in JAD; the facilitator is expected to employ a variety of techniques to implement them. 2.2. Incorporating NGT The NGT originated by Delbecq et al. [27] for conducting potentially problematic group meetings, embodies a set of procedures for conducting group sessions. It is a popular technique and one of the most successful processes for structuring group meetings [31]. NGT consists of the following five steps [53]: 1. Participants independently and silently generate a list of ideas. 2. The facilitator records one idea at a time from group members in a round-robin format until all participants have completed their list.
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
3. Participants discuss each idea for clarification only, without critical evaluation or lobbying. 4. Participants independently rate and rank the ideas. 5. The group decides the priority ordering of the alternatives based on voting and mathematical pooling of the individual rankings. The partitioning of activities into creative thinking and idea-generation, evaluation, and decision-making prevents negative group dynamics from inhibiting the free flow of information, views, and opinions; this contributes to more and better ideas [28,56]. These benefits help to produce solutions that reflect the combined judgment of the group and may lead to synergy (process gain) [36]. Because of its communication structure, NGT requires less facilitator intervention than JAD [78]. Poole and DeSanctis’ [63] adaptive structuration theory (AST) claims that meeting outcomes reflect both the impact of the structuring mechanisms and the manner in which the structures are used as intended (or modified). The role of facilitation is primarily to inspire faithful adoption of the meeting structure [14]. Although Lederer [49] proposed the use of NGT alone for determining systems requirements, it may not be entirely applicable to all aspects of SRD.
Phase I
Phase II
Phase III
401
JAD involves five stages; only the workshop is affected by NGT intervention. Furthermore, many desirable features of JAD [18] should be preserved. Our concept, therefore, does not suggest the replacement of JAD by NGT but their integration during the JAD workshop to influence SRD process effectiveness. Fig. 1 illustrates how NGT may be interspersed into the JAD workshop at appropriate intervals to stimulate ideas about major systems components, identify the decisions that these components support and the sources of information needed to make those decisions. It may also be used to bring about consensus on competing alternatives, and to categorize and prioritize system features. Conventional JAD approaches are still required for activities such as discussing assumptions, agreeing on guidelines and ground rules, accommodating knowledge sharing by domain experts, constructing models, and rationalizing and synthesizing the ideas generated. NGT is easy-to-use and is easily adaptable to the problem domain [21]. However, facilitators must develop a sense for the appropriate timing of each NGT intervention. For example, it does not enhance group process goals such as building team rapport [66], nor is it particularly effective for synthesizing ideas [15,52]. Similarly, some facilitators may
Phase IV: The Workshop
Phase V
Open Interactions… P R O J E C T
D E F I N I T I O N
R E S E A R C H
P R E P A R A T I O N
F W O O R R K T S H H E O P
NGT Open Interactions… NGT Open Interactions… ….. …..
Fig. 1. JAD/NGT integration.
F I N A L
D O C U M E N T
402
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
misapply the participation objective of NGT and aim for equal participation instead of participation commensurate with knowledge and expertise.
Group Task
2.3. Summary of past research Purvis and Sambamurthy [64] confirmed that users perceived that JAD contributed to superior, designer– user interactions in comparison to traditional techniques. Other studies that have investigated the effects of supporting JAD with electronic meeting systems during SRD [19,51] have been largely inconclusive. Davidson [25] found that organizational adaptations and inappropriate uses of JAD have curtailed its effectiveness, while Dean et al. [26] concluded that JAD’s (single-user) methods restrict participation and productivity. Researchers have validated the claim of NGTs effectiveness in a variety of decision-making environments. These include claims of its superiority over freely interacting techniques used for problem finding and problem solving [75]; the confirmation of its effectiveness with heterogeneous groups, and for solving complex problems [69]; and its usefulness in generating diverse ideas for the solution of multidimensional problems [33]. Gresham reported [47] that participants were highly satisfied with NGT process, and it has been used successfully in combination with other techniques [70,71,79]. In the largest sample size used in a group study to date, Watson et al. [76] investigated structured support for group discussions under conditions of no support, pencil and paper support similar to that provided by NGT, and with a group decision support system (GDSS). They found that while GDSS support produced superior results when compared to no support, there was no significant difference in the results obtained from groups using a GDSS and those using pencil and paper support. 2.4. The research model In our study, we have borrowed extensively from frameworks used in group studies—particularly from GSS, which can be viewed as an alternative meeting arrangement. We agree that group decision-making is similar to the transformation that occurs in work systems, which involves input, process, and output
Process
Outcome
Context Communication Structure
Fig. 2. Transformation model for group interaction.
[42]. The model adapted form Nunamaker et al. [59] for testing group decision effectiveness (Fig. 2) denotes this transformation. We also borrowed from two other frameworks [61,62] used to test structured support for group deliberations. The former specifically addressed software requirements development teams. The model indicates that when groups interact, several input factors (for example, group attributes, the nature of the task, contextual characteristics, and the adopted communication structure) act simultaneously on the deliberation process. These interactions determine the nature of the process conditions achieved. Inputs may either contribute to effective group interactions (in which process gain occurs) or poor deliberations (which result in process loss); the outcome of the process experience is the aggregate of process gains and losses. This relationship is represented in Huber’s equation for the effectiveness of decision groups [40]: Actual effectiveness ¼ potential effectiveness þ process gain process loss (1) Process gain results from effective communication, the ability of group members to be stimulated by the ideas of others to create new and useful ideas themselves, and group learning, which may result in synergy. Process loss is caused by non-task factors that reduce decision-making capability; these include excessive socialization, free riding, groupthink, conformance pressure, domination, and conflict avoidance. The research model (Fig. 3) incorporates the following input (independent) variables that affect group
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
403
Input Group Factors Dynamics Size Effort
Process Output
Task Nature Complexity
Facilitation Skill Level Control of Structure
Interaction - Effectiveness - Efficiency
- User Satisfaction
Outcome - Quality - User Satisfaction
Structure JAD NGT
Fig. 3. The research model.
process and the meeting outcome. Group communication structure and facilitation were manipulated during the study and the others controlled: Group communication structure (JAD or NJAD)— the procedure that governs the pattern of interaction—is the major mechanism under review. It is the vehicle through which the other input factors are focused on the meeting objectives. Group facilitation is also an important contextual characteristic. The facilitator (either through competence or assistance permitted by the meeting protocol) is the guardian of faithful adoption of the communication structure. Group interaction and decision quality are improved when facilitators successfully use meeting procedure as intended. Other group characteristics (dynamics, composition, and size) also bear directly on the process. Similarly, the level of effort the group devotes to the task, the extent of the group’s engagement (reflected in the degree of socialization and free riding), and its knowledge and experience are important characteristics that may determine process loss or gain. The structure and complexity of the task are also influential factors.
The process and outcome variables of interest are also indicated. Process effectiveness is a well-used gauge of successful group interaction. It is reflected in the quality of communication, idea generation, conflict resolution, and consensus building that the group achieves. Efficiency is also an indicator of process ‘‘goodness.’’ It is apparent in the rate of flow, quality, and relevance of the ideas generated during the meeting. The effects of group dynamics, facilitator behavior, and the structuring mechanism for the meeting influence process efficiency. The effectiveness of the process helps to determine the quality of the outcome and participants’ satisfaction with it. Group sessions may be considered successful when the participants develop consensus about the outcome of interest and commitment to, and support for, the agreement (i.e., they are satisfied with it). The outcome criteria also include the decision quality—the extent to which the meeting produces complete, correct, and feasible decisions. 2.5. Research hypotheses Six hypotheses were constructed to evaluate the relationships of our research model. The first five expressed the expectation that NJAD would
404
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
Table 1 List of hypotheses Hypothesis Hypothesis Hypothesis Hypothesis Hypothesis Hypothesis
1 2 3 4 5 6
Groups using NJAD will experience greater process effectiveness than those using JAD alone Groups using NJAD will experience the same degree of process efficiency as those using JAD alone Groups using NJAD will experience a higher level of satisfaction with the process than those using JAD Groups using NJAD will identify more with the decision than those using JAD Groups using NJAD will produce higher quality systems requirements than those using JAD alone Novice facilitators will experience greater improvement in process effectiveness as they move from using JAD to NJAD
outperform JAD in creating more desirable process conditions, which would help to generate higher quality requirements and greater satisfaction with the outcome. The sixth related to the impact of the communication structure on facilitation; whether NJAD would reduce JAD’s critical dependence on facilitation excellence (Table 1). The effectiveness of JAD or NJAD sessions may be reflected in the extent to which the input factors generate favorable process conditions for effective communication. Increased effectiveness results from a higher level of participation and conflict resolution and a reduction in the degree of destructive dominance. NJAD was expected to outperform JAD in creating these conditions by channeling group efforts more toward the attainment of its mission (Hypothesis 1). Process efficiency, which is one of JAD’s strengths [45], describes the extent to which the input factors permit the productive use of available meeting time. In comparison to freely interacting techniques, NJAD could increase process time because of the increased rigor of the NGT procedure [58], but it was expected to provide greater process effectiveness. We predicted that the additional NJAD overhead would be balanced by the gain in process effectiveness (Hypothesis 2). Group behavior that contributes to process loss is sometimes a source of disharmony [38]. On the other hand, there is a direct relationship between effective interactions and participants’ affinity with the process [13]. We proposed that NJAD, which promotes more effective interactions, would also contribute to a higher level of satisfaction than JAD (Hypothesis 3), and to a greater level of satisfaction with the final decision about the requirements and the participants’ identification with them (Hypothesis 4). The quality of systems requirements is defined as the degree to which they identify and incorporate desirable features for overall system success.
High-quality specifications are internally consistent, accurate, complete, unambiguous, relevant, and feasible [32,65,77]. NJAD was expected to contribute to more effective idea-generation and decision-making than JAD, and this would result in higher quality systems requirements (Hypothesis 5). The reason excellent facilitation is such a crucial determinant of JAD’s success is that the JAD facilitator is largely responsible for inducing effective group behavior [2]. NGT, however, provides such assistance within its protocol. This helps to reduce the facilitator’s burden and reduces the need for excellent facilitation to obtain effective process outcomes. Because of the technique, novice facilitators should reduce the performance gap between themselves and their more experienced counterparts (Hypothesis 6).
3. Research method In the completely randomized design we used for this study, communication structure (JAD and NJAD) was crossed with two levels of facilitation (expert and novice). Table 2 depicts the 2 2 factorial design with the four experimental conditions. In the experiment, we varied the impact of the communication structure and the effects of facilitation to determine their influence on the deliberations and on the resulting decisions, and controlled the effects of other potentially influential variables. Table 2 The 2 2 factorial design Group structure
JAD NJAD
Facilitator skill level Level I (high)
Level II (low)
6 groups ( 6 members) 6 groups 6
6 groups 6 6 groups 6
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
3.1. Subjects One-hundred-and-seventy-four persons volunteered for the experiments, which took place in three US cities. The volunteers included 12 active JAD facilitators, 18 note takers, who did not participate in the discussion, and 144 role-players. Four organizations– three facilitators’ associations and a training company– supplied the facilitators. Role-playing volunteers and scribes came from four enterprises, two educational institutions, and the general business community. The diverse group of role-players included non-facilitator members of the volunteer organizations, information technology workers, graduate and undergraduate business students, and business professionals. The participants had an interest in the outcome of the experiments and were promised copies of the results. 3.2. The task The simulated case selected for the experiment was designed to demonstrate the processes and problems of systems analysis and provide an appreciation for the human factors involved [54]. The groups were required to determine high-level systems requirements for an integrated order processing, inventory management, accounts payable and receivable, and distribution management system to support a chain of owned and franchised deli-style sandwich shops.
405
Facilitators were extensively debriefed for approximately 1 h before each experiment, which consisted of at least two group sessions. On the day of the session, the facilitators were given the freedom to select the order in which they applied the two meeting structures. Immediately before the start of a session, which lasted for approximately 2 h, participants completed the pre-session survey, reviewed the rules for the session, and read the case. During the (audio) taped session, they produced the solution set of system features. A scribe (selected from the available volunteers) supported one or two sessions and documented the final solutions. Immediately after the session, the participants completed the post-session questionnaire and the facilitators recorded their observations. An independent volunteer that was both treatmentand hypothesis-blind later coded the data into the database and typed all the hand-written requirements. A second volunteer compared the original and typedup versions of the requirements and verified their accuracy. The typed documents and the rating guidelines were then sent to the three judges. 3.4. Instruments Four questionnaires were used to collect data (Table 3). They were all adapted from instruments developed by researchers in previous studies. 3.5. Variables and measures
3.3. Procedures Twenty-four, six-member groups played user and developer roles in generating the requirements for the case. The participants were randomly assigned to a group, and groups were randomly assigned to one of the four experimental conditions. Each group was supported by a scribe and directed by a facilitator, who was already familiar with the details of the case. The participating organizations classified their volunteer facilitators into the two skill categories. Twelve facilitators (six from each category) were randomly pre-selected to conduct the 24 sessions. Each facilitator conducted two sessions—one with each meeting structure. Facilitators’ packets containing the description of the task, sample questionnaires, data collection instructions, and a script to conduct the NJAD sessions were delivered to them beforehand.
The input variables from our research model were either manipulated in the experiment (group structure and facilitation skill level) or controlled experimentally (e.g. all groups were of the same size and used the same task) or statistically (group experience, level of effort, and experience level of the group). The dependent variables and their operationalization are shown in Table 4. 3.6. Analytical procedures The analysis included a pre-examination of the data to gauge the reliability of the measurement instruments, establish construct validity, and determine whether there was a need for co-variation analyses to control influential variables that were not controlled experimentally. Multiple analysis of
406
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
Table 3 Instruments used in the study Instrument
Completed by
Comments
Source
Pre-session questionnaire (PI1) Post-session questionnaire (PI2) Facilitators survey (FI)
Participants
[3]
Facilitator
Rating sheet (RI)
Expert judges
Objective background information on participants’ profile (nine questions: six on a seven-point Likert scale, three single-answer) Four multi-item scales to capture perceptual data about the process and outcome (24 items, seven-point Likert scale) Facilitator report on session: mix of objective and perceptual data (five items) 10 items including an eight-item quality rating sheet (scores of 0–5)
Participants
[3,34,35] [30] [8]
Table 4 Operationalization of constructs Construct
Measure
Operationalization
Process effectiveness
Participation
Self-reported
Number of non-participants or minimal contributors (FI) Two-item scale from PI1 Three-item facilitator instrument Two-item scale (PI1) Number of unresolved items at the end (reported by the facilitator) Two-item scale (PI1) Seven-item scale (PI1)
Process efficiency
No. of unique features divided by meeting duration
Meeting duration (FI) Number of unique features determined by expert judges
Satisfaction with the process
Self-reported
Four-item scale (PFI)
Satisfaction with the outcome
Self-reported
Five-item scale (PFI)
Requirements definition quality
Quality points
Three judges’ scores (on accuracy, precision, completeness, conciseness, relevance, creativity, consistency, and feasibility)
Domination Conflict resolution
variance (MANOVA) and analysis of variance (ANOVA) were the main statistical procedures used to test the hypotheses. Factor analysis was conducted to establish construct validity and reliability analysis to investigate the internal consistency of the measurement scales. The resulting factor loadings suggested a high degree of construct validity. Other statistical techniques used in support of the main procedures included discriminant analysis to assess facilitator classification accuracy and Kendall’s coefficient of concordance for testing agreement among raters.
4. Summary of results The tests conducted on the control variables indicated no need for co-variation analyses. No
significant difference among the groups was found for the level of effort expended for either group structure (P ¼ 0:093) or facilitation (P ¼ 0:254). Similar results were obtained for group experience in a professional business environment and with SRD (P ¼ 0:758 and 0.290 for group structure and facilitation, respectively). Cronbach’s reliability test was used to revalidate the instruments and produced the following Alpha ratings: the post-session instrument, 0.91; attitude to group work (PI2), 0.79; destructive dominance (FI), 0.83; and the expert judges’ rating instrument 0.98. Kendall’s coefficient of concordance indicated reasonable evidence of congruence among the expert judges (at a significance level of 0.058), and that they applied similar standards of assessment. The two-group discriminant analysis to determine whether facilitators were accurately categorized also
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
407
Table 5 Summary of results of the tests of hypotheses Hypothesis Hypothesis Hypothesis Hypothesis Hypothesis Hypothesis Hypothesis
1: 2: 3: 4: 5: 6:
Process effectiveness Process efficiency Satisfaction with process Satisfaction with outcome SRD quality Difference in process effectiveness
Statistical technique
Significant
Hypothesis supported
ANOVA MANOVA ANOVA ANOVA ANOVA MANOVA
Yes ðP < 0:003Þ No ðP < 0:101Þ Yes ðP < 0:034Þ Yes ðP < 0:017Þ Yes ðP < 0:011Þ Yes ðP < 0:032Þ
Yes Yes Yes Yes Yes Yes
failed to reject the equality hypothesis (P ¼ 0:092). Based on participants’ evaluation of the competence of the facilitators, the discriminant analysis indicated classification accuracy of 66.7%. The empirical tests corroborated our prediction that NJAD would outperform JAD with respect to the quality of the requirements, and confirmed our expectation that the integrated approach would be at least as efficient as JAD. The results also supported our contention that participants using NJAD would be more satisfied with the process and identify more with the requirements than those using JAD. The findings also indicated that the integrated structure helped to reduce the need for highly competent facilitators to obtain successful JAD outcomes. Table 5 summarizes these results.
5. Discussion of the results The main premise of our theory that the integration of JAD and NGT would contribute to greater process effectiveness than JAD alone was corroborated in the experiment. This was an important result for the consistency of the model because the expectations for user satisfaction with the process and outcome and the quality of the requirements (which were also supported by the findings) were all based on the confirmation of this proposition. The empirical support for the proposition that NJAD was as least as efficient (time conserving) as JAD is important for ISs practitioners. It reduces the risk of lengthy NJAD meetings jeopardizing the release of busy managers and key personnel whose participation are required for effective results. It should be reassuring to practitioners if NJAD, despite its more rigorous structure, could preserve the benefit
of reduced SRD cycle time, which is one of JAD’s acknowledged strengths. In systems development, satisfaction with the SRD process may be necessary for achieving psychological identification with, and ownership of the project, which increases the probability of acceptance of the final system. Satisfaction with the process that generates the requirements may also be a prerequisite for users’ satisfaction and identification with the requirements generated by the process. Dissatisfied users are often obstacles to system acceptance and deployment. Newman and Robey [57] maintained that systems success is not always determined by technical validity but also by the ‘‘affective and behavioral responses of users.’’ The results support our proposition that NJAD would produce higher quality requirements than JAD. This is promising for practitioners who have struggled with poor systems requirements. SRD, though not an end in itself, is a critical determinant of system success. High-quality requirements have a ripple effect that contributes to better results in later development phases and eventually to technically sound, and maintainable systems. We believed that excellent facilitation was required for excellent JAD outcomes but we also proposed the reciprocal relationship (for NJAD) where the communication structure assisted facilitators in reducing problematic group behaviors. Our final result indicated that NJAD reduced the performance gap between competent and unskilled group leaders; it appeared to reduce JAD’s critical dependence on excellent facilitation for high-quality results. This finding could pave the way for applying NJAD in those cases where the unavailability of competent facilitators would otherwise have precluded the use of JAD.
408
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
These results do not signal the end of JAD; our proposal advocates that JAD be integrated with NGT. Also, the results are only applicable to user–developer communication problems and their effects on the quality and acceptance of the resulting requirements. Several other activities (e.g., project management, systems design, testing, and implementation) have their independent influences on the delivered system.
6. Conclusions Our research can be considered a follow up to studies that (1) found no significant difference in results produced by groups using GSS and NGT and (2) examined the effectiveness of using JAD in a GSS environment for SRD without formally investigating the effects of facilitation. Anson et al. [3] examined the impact of facilitation on meeting outcomes, with and without GSS, but did not include either JAD or SRD. Our study considered group structure and facilitation effects simultaneously with SRD. Two limitations should be mentioned. The unrealistic setting and the restrictive experimental conditions necessary to establish internal validity reduce our ability to generalize the results. It is not clear whether their full effects can be reproduced in a natural SRD setting characterized by emotional issues, political turf wars, vested interests, and especially, the differences in the rank and knowledge of participants. These are the basis of the difficult problems that facilitators are likely to encounter. Another limitation was the difficulty in specifying suitable criteria for classifying the facilitators. The measures of facilitation experience and the number of sessions actually conducted were used, but organizations that participated applied other (undisclosed) means of classification. However, the likelihood of misclassification was reduced, because the skills required in this experiment were at opposite ends of the facilitator competence scale. In any event, the discriminant analysis provided evidence that the classification was reasonable. Despite the increasing sophistication of information technology, systems builders continue to grapple with poor systems development processes and low-quality systems. NJAD may be able to complement other
techniques (e.g., prototyping) that have been used to increase the quality of user participation in SRD and, consequently, the effectiveness and efficiency of the process and the quality of the results. These benefits should help to reduce systems failures and, eventually, the direct and opportunity costs of the usually high allocation of IS resources to systems operations and maintenance. The results of our study promise some assistance to this effort. The proposed integrated structure appears to counteract the problems that inhibited JAD’s effectiveness and preserve its benefits. These effects may help to enhance the deliverables from other phases of SDLC that depend on systems requirements, and improve overall system quality. Ultimately, higher quality systems will allow organizations to reassign resources usually allocated to systems maintenance for other useful organizational activities. Because NJAD helps to offset the effects of facilitation skills and reduces the reliance on facilitator competence for outstanding process outcomes, this removes one of the potential dilemmas when a skillful facilitator is unavailable: not to use JAD at all or to make do with an unskilled facilitator. Superb facilitation, though desirable, is not a critical success factor under NJAD; this makes the technique more widely available to solve pervasive SRD problems.
References [1] D.C. Andrews, JAD: a crucial dimension for rapid applications development, Journal of Systems Management 42 (3) (1991) 23–27, 31. [2] D.C. Andrews, N.S. Leventhal, Fusion: Integrating IE, CASE, and JAD, Prentice-Hall, Englewood Cliffs, NJ, 1993. [3] R. Anson, R. Bostrom, B. Wynne, An experiment assessing group support system and facilitator effects on meeting outcomes, Management Science 41 (2), 1995, pp. 189–208. [4] P. Antunes, On the design of group decision processes for electronic meeting rooms, CLEI Electronic Journal (special issue) 2 (1) 1999. [5] P.M. Asaro, Transforming society by transforming technology: the science and politics of participatory design, Accounting Management and Information Technologies 10 (4), 2000, pp. 257–290. [6] J.H. August, Joint application design: the group session approach to systems design, Prentice-Hall, Englewood Cliffs, NJ, 1991. [7] S.A. Becker, E. Carmel, A.R. Hevner, Integrating joint application development (JAD) into cleanroom development
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23] [24]
with ICASE, in: Proceedings of the Hawaii International Conference on Systems Sciences, Hawaii, 1993, pp. 13–21. J.E. Bailey, S.W. Pearson, Development of a tool for measuring and analyzing computer user satisfaction, Management Science 29 (5), 1983, pp. 530–545. P.D.M. Barrow, P.J. Mayhew, Investigating principles of stakeholder evaluation in a modern IS development approach, Journal of Systems and Software 52 (2–3), 2000, pp. 95–103. J.M. Bartunek, J.K. Murninghan, The nominal group technique: expanding the basis procedure and underlying assumptions, Group and Organization Studies 9 (3), 1984, pp. 417–432. M.G. Beruvides, Group decision support systems and consensus building issues electronic media, Computers in Industrial Engineering 29 (1–4), 1995, pp. 601–605. P. Beynon-Davies, H. Mackay, D. Tudhope, Its lots of bits of paper and ticks and post-it notes and things: a case study of a rapid application development project, Information Systems Journal 10 (3), 2000, pp. 195–216. Borovits, S. Ellis, O. Yeheskel, Group processes and the development of information systems, Information and Management 19 (2), 1990, pp. 65–72. R.B Bostrom, R. Anson, V.K. Clawson, Group facilitation and group support systems, in: L.M. Jessup, J.S. Valacich (Eds.), Group Support Systems: New Perspectives, Macmillan, New York, 1993, pp. 146–168. B. Broome, The role of facilitated group process in community-based planning and design, in: L.R. Frey (Ed.), Innovations in Group Facilitation: Applications in Natural Settings, Hampton Press, Inc., New Jersey, 1994, pp. 27–52. G.J. Browne, V. Ramesh, Improving information requirements determination: a cognitive perspective, Information and Management 39 (8), 2002, pp. 625–635. T.A. Byrd, K.L. Cossick, R.W. Zmud, A synthesis of research on requirements analysis and knowledge acquisition techniques, MIS Quarterly 16 (3), 1992, pp. 117–138. E. Carmel, J.F. George, J.F. Nunamaker, Supporting joint application development (JAD) with electronic meeting systems: a field study, in: Proceedings of the International Conference on Information Systems, vol. 12, USA, 1992, pp. 223–232. E. Carmel, J.F. George, J.F. Nunamaker, Examining the process of electronic-JAD, Journal of End User Computing 7 (1), 1995, pp. 13–22. E. Carmel, R.D. Whitaker, J.F. George, PD and joint application design: a transatlantic comparison, Communications of the ACM 36 (6), 1993, pp. 40–48. R.J. Chapman, The effectiveness of working group risk identification and assessment techniques, International Journal of Project Management 16 (6), 1998, pp. 333–343. K.E. Cheng, A requirements definition and assessment framework for SDL tools, Computer Networks and ISDN Systems 28, 1996, pp. 1703–1715. A. Crawford, Advancing Business Concepts in a JAD Workshop Setting, Prentice-Hall, Englewood Cliffs, NJ, 1994. E.J. Davidson, An exploratory study of joint application design (JAD) in information systems delivery, in: Proceedings
[25]
[26]
[27]
[28]
[29]
[30] [31]
[32]
[33]
[34]
[35]
[36]
[37]
[38]
[39]
[40] [41]
[42]
409
of the XIV International Conference on Information Systems, vol. 12, Nos. 5–8, 1993, pp. 271–278. E.J. Davidson, Joint application design (JAD) in practice, Journal of Systems and Software 45 (3), 1999, pp. 215–223. D.L. Dean, J.D. Lee, M.O. Pendergast, A.M. Hickey, J.F. Nunamaker, Enabling the effective involvement of multiple users: methods and tools for effective software engineering, Journal of Management Information Systems, 14 (3) (1997– 1998) 179–222. A.L. Delbecq, A.H. Van de Ven, D.H. Gustafson, Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes, Scott. Foresman, Glenview, IL, 1975. A.L. Delbecq, A.H. Van de Ven, D.H. Gustafson, Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes, Greenbriar, Middleton, WI, 1986. A.R. Dennis, G.S. Hayes, R.M. Daniels Jr., Business process modeling with group support systems, Journal of Management Information Systems 15 (4), 1999, pp. 115–142. M. Dieckmann, Making new technology investments pay off, Managing Office Technology 41 (7), 1996, pp. 14–16. K.L. Dowling, R.D. St. Louis, Asynchronous implementation of the nominal group technique: Is it effective? Decision Support Systems 29 (3), 2000, pp. 229–248. B. Fazlollahi, M.R. Tanniru, Selecting a requirement determination methodology—contingency approach revisited, Information and Management 21 (5), 1991, pp. 291–303. S. Frankel, NGT þ MDS: an adaptation of the nominal group technique for ill-structured problems, Journal of Applied Behavioral Science 23 (4), 1987, pp. 543–551. D.S. Gouran, C. Brown, D.R. Henry, Behavioral correlates of perceptions of quality in decision-making discussions, Communication Monographs 45, 1978, pp. 51–63. S.G. Green, T.D. Taber, The effects of three social decision schemes on decision group process, Organizational Behavior and Human Performance 25, 1980, pp. 97–106. T.R. Henrich, T.J. Greene, Using the nominal group technique to elicit roadblocks to MRP II implementation, Computers and Industrial Engineering 21 (1–4), 1991, pp. 335–338. R.A. Hirschheim, User experience with and assessment of participative systems design, MIS Quarterly 9 (4), 1985, pp. 295–304. E.S. Ho, Y.J. Lai, S.I. Chang, An integrated group decisionmaking approach to quality function deployment, IIE Transactions 31 (6), 1999, pp. 553–567. K. Holtzblatt, H.R. Beyer, Requirements gathering: the human factor, Communications of the ACM 38 (5), 1995, pp. 30–32. G.P. Huber, Managerial Decision Making, Scott. Foresman, Glenview, IL, 1980. R.B. Jackson, D.W. Embley, Using joint application design to develop readable formal specifications, Information and Software Technology 38 (10), 1996, pp. 615–631. J. Jiang, G. Klein, Software development risks to project effectiveness, Journal of Systems and Software 52 (1), 2000, pp. 3–10.
410
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
[43] M.C. Kettelhut, Avoiding group-induced errors in systems development, Journal of Systems Management 42 (3), 1991, pp. 13–17. [44] M.C. Kettelhut, JAD methodology and group dynamics, Information Systems Management 10 (1), 1993, pp. 46–53. [45] M.C. Kettelhut, Using JAD for strategic initiatives, Information Systems Management 14 (3), 1997, pp. 29–36. [46] R. Kling, The organizational context of user centered designs, MIS Quarterly 1 (4), 1977, pp. 1–52. [47] L.J. Korhonen, Nominal group technique, in: M.W. Galbraith (Ed.), Adult Learning Methods, Krieger Publishing Company, Florida, 1980, pp. 247–259. [48] R.L. Kumar, Managing risks in IT projects: an options perspective, Information and management 40 (1), 2001, pp. 63–74. [49] A.L. Lederer, Information requirements analysis, Journal of Systems Management 32 (12), 1981, pp. 15–19. [50] N.S. Leventhal, Using groupware tools to automate joint application development, Journal of Systems Management 46 (5), 1995, pp. 16–22. [51] Y.I. Liou, M. Chen, Using group support systems in joint application development for requirements specifications, Journal of Management Information Systems 8 (10), 1993– 1994, pp. 805–815. [52] A. MacPhail, Nominal group technique: a useful method for working with young people, British Educational Research Journal 27 (2), 2001, pp. 161–170. [53] J.G. Mahler, Structured decision making in public organizations, Public Administration Review 47 (4), 1987, pp. 336–342. [54] R. Marble, Casebook for systems analysis and design: FSS Inc., Mitchell-McGraw Hill, 1992. [55] M.L. Markus, Power, politics and MIS implementation, Communications of the ACM 7, 1983, pp. 430–614. [56] C.M. Moore, Group Techniques for Idea Building, Sage Publishing, Newbury Park, CA, 1987. [57] M. Newman, D. Robey, A social process model of user– analyst relationships, MIS Quarterly 16 (2), 1992, pp. 249–266. [58] F. Niederman, C.M. Beise, P.M. Beranek, Issues and concerns about computer-supported meetings: the facilitator’s perspective, MIS Quarterly 20 (1), 1996, pp. 1–22. [59] J.F. Nunamaker, A.R. Dennis, J.S. Valacich, D.R. Vogel, J.F. George, Group support systems research: experience from the lab and field, in: L.M. Jessup, J.S. Valacich (Eds.), Group Support Systems: New Perspectives, Macmillan, New York, 1993, pp. 125–145. [60] J.F. Nunamaker, A.R. Dennis, J.F. George, J.S. Valacich, D.R. Vogel, Electronic meeting systems to support group work: theory and practice at Arizona, Communications of the ACM 34 (7), 1991, pp. 40–61. [61] R. Ocker, S.R. Hiltz, M. Turoff, J. Fjermestad, The effects of distributed group support and process structuring on software requirements development teams: results on creativity and quality, Journal of Management Information Systems 12 (3) (1995–1996) 127–153.
[62] Pinsonneault, K.L. Kraemer, The impact of technological support on groups: an assessment of the empirical research, Decision Support Systems 5 (2) (1989) 197–216. [63] M.S. Poole, G. DeSanctis, Understanding the use of group decision support systems: the theory of adaptive structuration, in: C. Steinfield, J. Fulk (Eds.), Organization and Communication Technology, Sage Publications, Beverly Hills, 1990, pp. 173–179. [64] R. Purvis, V. Sambamurthy, An examination of designer and user perceptions of JAD and the traditional IS design methodology, Information and Management 32 (3), 1997, pp. 123–135. [65] S. Raghaven, G. Zelesnik, G. Ford, Lecture Notes in Requirements Elicitation, SEI-94-EM-010, Carnegie Mellon University, Pittsburgh, 1994. [66] E.B. Richmond, M.T. McKnelly, Alternative user survey and group process methods–nominal group technique applied to US depository libraries, Journal of Government Information 23 (2), 1996, pp. 137–149. [67] O. Rist, Reinventing E-Business, Network Computing 12 (20), 2001, pp. 41–47. [68] S.P. Schuman, What to look for in a group facilitator, Quality Progress 29 (6), 1996, pp. 69–72. [69] B.Y. Stephenson, L.K. Michaelsen, S.G. Franklin, An empirical test of the nominal group technique in state solar energy planning, Group and Organization Studies 79 (3), 1982, pp. 320–334. [70] Teltumbde, A framework for evaluating ERP projects, International Journal of Production Research 38 (16) (2000) 4507–4520. [71] J.B. Thomas, R.R. McDaniel Jr., M.J. Dooris, Strategic issue analysis: NGT þ decision analysis for resolving strategic issues, Journal of Applied Behavioral Science 25 (2), 1989, pp. 189–200. [72] L. Uden, Design and evaluation of human centered CIM systems, Computer Integrated Manufacturing Systems 8 (2), 1995, pp. 83–92. [73] J. van Maurik, Facilitating excellence: styles and processes of facilitation, Leadership and Organizational Development Journal 15 (8), 1994, pp. 30–34. [74] A.H. Van de Ven, A.L. Delbecq, Nominal versus interacting group processes for committee decision-making effectiveness, Academy of Management Journal 14, 1971, pp. 203–212. [75] A.H. Van de Ven, A.L. Delbecq, The effectiveness of Nominal, Delphi, and interacting group decision making processes, Academy of Management Journal 17 (4), 1974, pp. 605–621. [76] R.T. Watson, G. DeSantis, M.S. Poole, Using a GDSS to facilitate group consensus: some intended and unintended consequences, MIS Quarterly 12 (3), 1988, pp. 463–478. [77] J.C. Wetherbe, Executive information requirements: getting it right, MIS Quarterly 15 (1), 1991, pp. 51–65. [78] J. Wood, D. Silver, Joint Application Development, Wiley, New York, 1995. [79] N. Zuech, Identifying and ranking opportunities for machine vision in a facility, Industrial Engineering 24, 1992, pp. 42–61.
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411 Evan W. Duggan is an assistant professor of MIS in the Department of Information Systems, Statistics, and Management Science at the University of Alabama. He received his PhD and MBA from Georgia State University and BSc from the University of the West Indies, Jamaica. He has over 25 years of experience in IT with a major multinational organization. His research interests involve the management of ISs in corporations including IS success factors, human and group issues in systems development and implementation, and group support systems and facilitation. He has research publications (or articles forthcoming) in the International Journal of Industrial Engineering, Journal of International Technology and Information Management, Information Technology and Management, Journal of End User Computing, Information Resources Management Journal, and Human– Computer Interactions. He has taught MIS and Decision Sciences courses at the graduate and undergraduate levels in several US and international institutions, including executive MBA programs.
411
Cherian S. Thachenkary is an associate professor of Management in the J. Mack Robinson College of Business at Georgia State University in Atlanta, GA, USA. He obtained his PhD and MSc degrees in Management Sciences from the Faculty of Engineering at the University of Waterloo in Canada, and a BSc from the University of Toronto. He conducts teaching and research in the area of Management of Technology. His current focus is on the digital economy—the cost/benefits of high bandwidth networks and electronic services/applications. His research has been published in the Journal of Telemedicine and e-Health, OR/ MS Today, Computer Networks and ISDN Systems, IEEE Transactions on Engineering Management, Office: Technology and People, and Computerworld. He has served as a Senior Editor of Computer Networks and ISDN Systems, and as a Departmental Editor of DATABASE. He has also held faculty appointments at the University of Waterloo in Canada, The University of Waikato in New Zealand, and at Cairo University in Egypt.
