Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
UNDERSTANDING INTERNAL IS CUSTOMER MODELS OF FIRM PERFORMANCE TO IDENTIFY POTENTIAL HIGH-IMPACT PROJECTS Ken Peffers Texas Tech University Email:
[email protected]
Charles E. Gengler City University of New York Email:
[email protected]
Abstract
critical for the firm, while others may be essentially irrelevant to its overall performance. There is no shortage of ideas for new IS projects in most business organizations. With news of technological innovations appearing at a dramatic rate and with a continuous stream of design corrections, updates and new features requested for the existing capital base of systems, managers have no problem finding potential IS investments for the firm, many of which are seemingly worthwhile. To generate ideas for projects that will have the greatest impact on achieving the firm's goals is a more difficult problem [9, 17]. Both strategic and tactical level projects may be critical for the firm. Some important projects require imagining new systems that will change products, delivery channels, processes, and the structure of the organization, while others, less dramatically, call for training, user support, or incremental system improvements. While knowledge of the firm and its customers, processes, and products that is necessary to generate such ideas is richly distributed around the organization, attention to the mission of the whole organization may be dearer. Consequently, a deluge of system project proposals, mostly to support the incremental improvement of current activities, may hide and even camouflage important project ideas. Executives have expressed dismay about their inability to confidently identify and value IS project proposals [18, 23]. Complicating this is identification process is the supplier-customer relationship between the IS unit and other units in the firm. For these customers, the IS unit provides a variety of services, such as consulting, project development, system maintenance, and operations. Furthermore, the IS unit may be competing with other potential providers, including outside vendors and potential devolution of activities to functional areas, to provide these services. Consequently the CIO can hardly afford not to provide a credible avenue for consideration of "bottom up" project proposals. How can the IS group identify potential high-impact projects, using both top-down and bottom-up, information?
The CIO faces competing demands from internal customers for a variety of services, such as consulting, development, maintenance, and operations. S/he is also responsible for identifying new projects that will increase firm value. How can s/he identify potentially high-impact projects, in the face of a deluge of requests, many for politically motivated, suboptimal projects? We adapt a personal constructs theory based method, used in marketing research practice, to IS planning as the critical success chain (CSC) method. The CSC method uses knowledge distributed through the organization to develop socially constructed models of the relationships between new IS features, critical success factors (CSF), and firm goals. It uses these models to generate ideas for feasible high-impact IS projects. A case demonstrates CSC’s usefulness for (1) making practical use of knowledge distributed throughout the organization for IS planning and (2) providing rich models to better understand the needs of the IS unit's customers.
1. Introduction Today's CIO is faced with a number of conflicting demands for system development from internal customers in the firm. At the strategic level, the information systems (IS) unit should be developing and operating systems that will maximize the value of the firm. To accomplish this, it is important for the CIO to focus on the identification of potential development projects that are aligned with the business strategy of the firm and are focused on the firm's critical success factors (CSF). Such systems may create competitive advantage, add value to products, drastically reduce costs, open new channels for marketing, or change the economies of scale and scope in the firm's favor [8]. The IS unit is also faced with a steady stream of tactical and operational level requests for system changes and enhancements. Some of these requests are required to adapt processes to new environments, products, and markets. Others are intended to incrementally enhance quality, service, or productivity. Some of these requests are
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
1
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
CSF is a widely understood concept for identifying performance objectives to be satisfied by strategic IT investments [26]. The concept is simple and intuitively appealing: to identify needs for strategic systems, identify the small number of performance requirements on which the success of the firm depends [30]. CSF works best if implemented among a small number of key employees in the firm. It may be difficult, therefore, to apply CSF directly to capture strategic knowledge from a large number of employees because of the intensive interview on which it is based. As a result, knowledge distributed through the organization about potential high-impact IS projects may be difficult to collect and analyze. To address these issues, we have extended CSF to facilitate the empirical development of internal customer models of firm performance, through a process that might be described as 'both top-down and bottom-up.' These models, which we call critical success chains (CSC), are aggregated across members of the organization at several levels. We use them to identify potential high impact IS projects for the firm. The CSC method is based on methods successfully used by leading edge marketing research analysts to understand customers' models for how preferred product features provide value. This approach benefits managers involved in IS planning in several ways. First, it helps the IS manager make use of the knowledge of a diverse group of people spread across the organization about what IS attributes may be important to the firm and why. This helps to identify important strategic and tactical IS projects for the firm. Second, it helps managers understand internal customers' motivations for their preferences for IS features. This provides richer information to help managers to determine how to specify systems to meet customer needs. In this paper we make the case for the use of the CSC method for IS planning and we provide a descriptive procedure for its implementation. The remainder of the paper is organized as follows. First we briefly review the CSF concept, noting opportunities to extend it for IS planning. Second, we introduce PCT as a framework for studying CSF, extend the CSF concept to CSC, and describe the CSC method for IS planning. Third, we present a scenario in which we demonstrate the successful use of the CSC method for IS planning. Finally, we discuss the benefits of the CSC method, the contributions of this paper to practice, and potential future extensions of this research.
2. Critical Success Factors Rockart [29] proposed CSF to help CEOs specify their own needs for information to support important decisions
so that systems could be developed to meet those needs. Firm growth, diversification and international trade had resulted in a greater need for executive information [7], but executives were still the unhappy customers for reports designed as convenient byproducts of the firm’s transactions processing systems, which seldom provided much useful information for management decisionmaking. In addition, they were frustrated by MIS departments that were unresponsive to ad hoc information requests. CSF are “…for any business, the limited number of areas in which [satisfactory] results… will ensure successful competitive performance … [29].” For example, an auto rental firm CSF may be the ‘availability of cars to match customer reservations’ because this performance objective is essential for customer satisfaction. Conceptually, CSF are the intended performance consequences of systems and behaviors within the firm that are related most strongly to the achievement of desired firm objectives. They are unique to the firm, depending on the firm's product line and intended product positioning. CSF have been adapted for a broadly for use in IS planning [6, 4, 32], performance evaluation [1], and information requirements determination [5]. Many firms have used the CSF concept to focus attention on the development of new IS to achieve the firm's most important goals. CSF are intuitively appealing for executives because they help justify the development of strategically important new systems, the benefits of which might be hard to quantify. CSF is arguably the most important planning concept for senior, including IS, managers. If its functionality could be extended in two ways, however, it might be even more useful. First, CSF doesn’t lend itself to support the use of knowledge widely distributed among employees throughout the firm about how system features might affect firm performance. Secondly, it doesn’t provide rich support for understanding the motivation of customers for preferring IS features. The primary focus of CSF is on senior managers. This is a strength because a top-down method is considered necessary to keep a strategic perspective [32]. Bottom-up methods of generating proposals result in many more proposals than can be implemented, most of which support existing methods of operation [20] rather than strategic objectives. In order to develop a complete CSF model, however, researchers have recommended studying the views of personnel at various levels in the organization, in addition to those at the executive level, e.g., [29, 31, 26]. The use of such distributed knowledge is problematic for three reasons. First, many employees at lower organizational levels may be unable to “readily relate to the CSF concept [5].” CSF methods assume that firm members interviewed understand the CSF concept, which
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
2
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
may not always be true among employees at every level of the organization. Secondly, direct solicitation of CSF from employees distributed around the organization is likely to result in many non-strategic ideas. CSF doesn’t yet have a mechanism to sort out the important ideas. Thirdly, CSF is expensive, if widely applied through the organization. This is because CSF participation requires intensive interviews of each participant, which becomes expensive when applied to many participants. The CSF concept focuses directly on the performance factors that are most critical to the firm’s objectives. Consequently, it doesn’t provide a mechanism to help the analyst to understand the participant's motivation in preferring any particular system feature. The lack of a motivational context for CSF information may not be of concern where participation is limited to a small number of top-level executives, but is more important as a larger number of participants are involved in the planning process.
3. A Customer-oriented Method to Identify High-impact Projects. These two issues do not prevent CSF from being one of the most important tools for IS planning, but their relaxation might add considerably to the inherent appeal of CSF analysis. A data collection and analysis method that extended CSF to model the reasons why customers preferred IS features and to identify high-impact projects for the firm would be extremely valuable to the firm. Ideally, a method would (1) make use of knowledge distributed throughout the firm and, (2) help IS managers to understand customer motivation for preferring system features. There is considerable justification for treating IS users as customers. The metaphor of the internal customer to refer to employees and to downstream processes suggests important positive objectives for IS planning. IS services and functionality is usually intended to provide additional net value to the firm’s customers, either directly or by supporting downstream processes. Consequently, providing value to IS users provides value to customers. In addition, although IS users do not pay directly for the services of the IS unit, they may be able to opt for alternative sources for these services, either outside the organization or in a decentralized mode. In this section we introduce personal constructs theory (PCT), as the basis for a new CSF model and method. Next, we extend the CSF concept, introducing critical success chains (CSC), a PCT-based theory that incorporates CSF and is suitable for use in IS planning. Finally, we describe the CSC method for strategic IS
project planning, which addresses the objectives described above.
3. 1. Personal constructs theory (PCT) PCT is a social scientific theory, developed by George Kelly [19], in the course of his experiences working as a practicing clinical psychologist. It provides a framework that helps the practicing psychologist understand individual patient models for how the universe works. Each person, Kelly [19] noted, is an observer-scientist who develops his or her own unique models about how the world works by making observations, developing and testing hypothesis, and interpreting results. We do this in order to predict and control our individual environments. These personal models are based on many dimensions (constructs), which are unique to the individual and which describe the attributes and behavior of objects and events. The collective knowledge that we have about how the universe works results from communication of these interpretive and piecemeal constructs and their aggregation into construction systems. PCT is consistent with contemporary scientific views, in which there are "alternative constructions" of reality [25, p. 228], no one of which is exclusively valid. PCT has been used successfully to model the decision making processes in several applied social scientific domains, including education [11], mental health [33], advertising [16], leisure [21], and sales [15]. In addition, PCT has been used in research for knowledge engineering [3, 10, 12]. In general, practical methods based on PCT seek to elicit information about people's knowledge structures by observing how they differentiate among stimuli. One such methodology, called “laddering” [28], is used to model consumers' value structures related to preferences for product features. In the use of a laddering method, a participant is given a choice task, such as asking her which of three products she would most prefer to have. Then, the participant is asked why she prefers the one she chose. When the participant responds with a reason, the interviewer repeats a series of “Why is that important?” questions. Typically, participants respond with series of concrete reasons why they prefer one object to another. These “Why…” questions help uncover the expected consequences of choosing the preferred product and the personal values which are supported by these consequences. Other questions, such as "What was it about this product that made you think that it would have that effect?" are used to uncover the product features related to the reasons. The chains of features, consequences, and values uncovered through the laddering process, called 'ladders,' are content analyzed into consistent constructs across participants and used to
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
3
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
produce network models of how participants interrelate the constructs [14]. Such models can be used to design products that have potentially higher value for customers. When an IS manager uses CSF for IS planning, he or she implicitly uses a three-element model of consequence, similar to PCT. The manager assumes that, if systems with appropriate IS attributes are developed, the use of these systems will result in outcomes that are observable as changed CSF performance. Good CSF performance is, in turn, required to achieve firm goals. We have explicitly extended CSF to incorporate implicit these personal construct elements. This linkage of IS attributes to CSF performance to specific firm goals we call a critical success chain (CSC). The resulting model is shown graphically in figure 1. For example, for a rental car firm, IS that help manage the allocation of the firm's inventory of cars to locations (IS attributes) affect the firm's ability to have cars available when customers arrive with reservations (CSF performance). This availability affects IS Attributes
CSF Performance
Firm Goals
Figure 1. Critical success chain (CSC) [22]. customer satisfaction and the firm's market share (firm goals). This CSC, along with others for the firm, could be used to identify and evaluate the attributes of potential new strategic IS. The one to one mapping between PCT and CSC allows us to develop a method for data collection and analysis for CSC that is based upon applications of PCT that are already used successfully in other domains, including advertising and product feature development.
number of participants affects the diversity of views expressed as well as the cost of the study. No theoretical considerations dictate a minimum or maximum number, but the range of ideas represented in the results of the study is expected to be sensitive to the diversity of views represented in the sample. Consequently, it is expected to be important to solicit participation from representative members of senior management, middle management, and professional or 'journeyman' level line employees, as suggested by Boynton and Zmud [5]. The broader the scope of the study, the larger the group of participants required to provide representative views. Obtain a diverse set of project ideas for use as stimuli in the interviewing process. This can be accomplished by soliciting project ideas from study participants. Step 2, participant interviews: Interview the participants to collect data about their personal constructs. 1. Prestudy Preparation Determine scope & participants. Collect project idea stimuli. 2. Participant Interviews Elicit personal constructs from org. members.
3. Analysis Aggregate personal constructs into CSC models.
4. Technical Workshops Elicit feasible strategic IS from technical experts.
3.2. CSC method for IS planning To use the CSC for IS planning, we have developed a four-step procedure, called the CSC method, represented in Figure 2. In steps one through three, analysts work with participants to develop graphical models of CSC for the organization. In step 4 they work with IS development personnel to generate feasible ideas for IS projects that have the potential to affect performance in terms of the CSC. Here we briefly describe the steps in the process. Step 1, prestudy preparation: Determine the organizational scope of the study, in terms of lines of business or business units. Limiting the scope of a study to a single line of business, a narrow product line, or within a single facility or division, focuses the study and limits its complexity. The more diverse the products and processes that are included in the study, the more complex the resulting analysis. Select the study participants. The
Figure 2. The critical success chain (CSC) method for strategic IS project idea generation in four steps. Present each participant with a subset of the project ideas collected in step 1 and ask the participant to rank order them in terms of their importance to the organization. For the higher ranked stimuli, ask the participant "Why would this project be important to the organization?" to elicit expected performance impacts. Ask series of "why is that important to the organization?" questions to collect data on associated concepts ending with the organization's goals. Ask series of "what about this system that makes you think it would do that?" questions to elicit associated concepts
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
4
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
ending with specific attributes of the system. Record the responses as linked chains. Step 3, analysis: Interpret the responses in a qualitative hierarchical clustering process to classify each of the responses into consistent constructs across participants, so that uniquely worded similar meanings are clustered together. Develop a matrix, XiYj, where the rows represent chains and the columns represent constructs and xiyj equals 1 if the jth chain contains the ith construct, 0 otherwise. Cluster the chains (rows in the matrix) to minimize the variation in the constructs among chains within clusters. Transform the clusters into network CSC models, representing the constructs as nodes linked by links from the chains. The area of each node is proportional to the number of participants mentioning the related construct. Simplify the network models by deleting redundant links, i.e., delete link A—C if A—B and B—C already exist. If the use of a large participant sample results in excessively complex models, delete links and nodes, the mention of which falls below a cutoff point, determined by the analyst. The resulting CSC models are aggregations of the constructs collected in the interviews. They represent socially constructed models of the firm's performance. Step 4, technical workshops: Select participants for project ideas workshops. The purpose of the workshops is to use technical expertise to produce feasible IS project ideas that support the intent of the CSC models. The participants should, as a group, possess expertise in a representative set of IS development domains, such as systems analysis and design, computer programming, communication, and database management. Conduct project idea workshops, explaining the CSC concept to participants and presenting the CSC models developed for the organization. The participants use their technical expertise to identify feasible IS project ideas that would address the performance factors identified in the CSC models. The ideas are developed to a 'back-of-theenvelop' standard, i.e., a level of specification that can be achieved by the participants at the meeting without resorting to research outside the meeting. The intent is to encourage participants to identify multiple ideas to minimal level of detail, rather than a small number of detailed proposals. For each system, the participants label the system, briefly describe its nature, its likely architecture, the resources required to develop it, its cost, likely sourcing, and magnitude of risk, and its expected impact on the organization. At the conclusion of step 4, analysts have developed the CSC models and project ideas. What happens next is outside the scope of the CSC method and depends on organizational culture and the nature of the sponsorship of the study in the organization. Typically the analysts would present the CSC models and project ideas to senior managers in an IT steering committee or other appropriate
forum to determine which of the ideas should be pursued with full feasibility studies.
4. An Illustrative Case Scenario This case scenario is based on the lessons learned in actual use of the CSC method by the authors for IS planning. The facts described are similar in many respects to CSC studies that the authors have conducted in two organizations and include results, which are based on and illustrative of the outcomes of these studies. Because of the confidentiality agreement with one of the organizations, no facts about the industry, the organization, or the systems described below are factual. The scenario should be regarded as illustrative, rather than evidentiary. Wing Fat Foods (WFF) is a grocery and sundry wholesaler that operates in three Eastern seaboard states, delivering perishable and nonperishable foodstuffs, as well as hardware, kitchen ware and household goods to restaurants, groceries, and similar businesses. WFF's focus is on Chinese and Southeast Asian foods, but other AsianPacific and Indian foods are also included in its product scope, as well as tropical fresh fruits, vegetables, and seafood. WFF prides itself in the delivered quality of its perishable foods, which are, it claims, fresher than those of competitors, and in flexible, on-demand delivery to customers. In an industry in which the level of information systems investments is relatively low, WFF hopes its IS investments will give it an operational edge over competitors, allowing it to continue to compete on quality and service first and price second. WFF's CIO was persuaded to use the CSC method to identify potential high-impact IS projects for the firm and engaged an outside analyst to conduct a CSC study. The CSC analyst obtained agreements to participate from 25 IS unit customers at WFF, including six senior managers, eleven middle managers (four regional managers, the comptroller, marketing manager, two shipping department managers, and three others), five "journeyman" level employees (from delivery, shipping and receiving, accounting, and elsewhere), and three WFF customers (two restaurant managers and a supermarket representative). To develop a list of viable IS projects to serve as stimuli, she asked each participant for an IS project idea when she scheduled the interview appointment with that person. Asking each participant to describe the functionality of an IS that would benefit WFF, she recorded the project name, a description of its purpose, the system inputs and outputs, who would use the system, and its expected benefits. Later she rewrote each of the descriptions into a standard 50 to 100 word format. Nearly every participant suggested a project idea. Typical of what might be expected from a "bottom-up" process, nearly
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
5
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
every project idea appeared to be intended to affect its authors' job duties. In consequence, many were likely to be organizationally suboptimal. Next, she conducted 25 to 50 minute interviews with each participant over a two-week period. In each interview she showed the participant descriptions of three information systems suggested by other participants and asked him or her to rank order them in terms of their importance to the organization. For the highest ranked system idea, she asked the participant "why is that system important to WFF?" When the participant responded with a reason, she asked, “Why is that important to the organization?” The questioning was repeated several times until the participant was unable to continue because he or she had reached an ultimate value or goal for the organization. Next, the participant was asked, with respect to the first reason given, "What was it about the system that makes you think it would do that?" This question was repeated until the participant could not continue, i.e., the participant had reached a concrete feature or attribute that he or she expected would be part of the project idea. The questions were repeated for the second most highly ranked of the three project ideas. The third ranked idea was ignored. The output from each interview was one or more chains, representing the participant's models of the relationships between system attributes or features, performance consequences, and organizational goals. In figure 3, for example, a participant thinks that a deliveryordered packing list (IS attribute) could result in quicker deliveries to customers (CSF performance), which could result in higher margins (firm goals) for the firm. Participants suggested 6-10 chains each, on average. Because every participant expressed his or her ideas using unique statements, she needed to cluster and relabel statements into consistent constructs across participants. She and another analyst independently and iteratively read through the concepts, combining and relabeling those that were most similar, stopping when additional combinations would appear to result in the loss of substantial information. They compared independently created constructs, approximately 80% in agreement, and reconciled differences by consensus. Next she mapped the constructs into a vector matrix of rows, representing the chains, and columns, representing the constructs, in which an element was coded as 1 if the construct appeared in the ladder and 0 otherwise. To aggregate the personally constructed CSC into socially constructed CSC for WFF, she clustered the chains (the matrix rows), using Ward’s method, to minimize the within cluster variance in the constructs contained in each cluster. A seven cluster solution was chosen on the basis of psuedo F and t tests, as well as the analyst's judgment about the understandability of the cluster solution.
She mapped the clustered chains into CSC maps, a separate map for each cluster. The constructs were represented by circles, the area of each of which was proportional to the number of participants in the sample that mentioned the construct, and the circles were linked by lines representing the links in the chains. Each map represents a specific firm CSC. Figure 4 shows one of the maps. It represents a Better margins We can charge more than competitors Fresher seafood Get to each customer quicker after order Less unloading time at each stop Load trucks in reverse delivery order Delivery-ordered packing list
Figure 3. Example chain collected from participant at Wing Fat Foods consensus model consisting of, from left to right, descriptions of desired system attributes, the resulting expected performance outcomes (CSF), and associated firm goals. This map shows a model in which better information and analysis for scheduling, routing and packing trucks (system attributes), would affect delivery speed and timeliness and product freshness (CSF). Changed CSF performance would affect the number of customers served, pricing, perceived quality, and margins (firm goals). The number in each circle represents the number of participants mentioning the construct. The CSC maps developed in the first part of the study were used as the starting point for systems professionals to develop strategic IS proposals. In a two-day group workshop, the analyst met with six professionals from the WFF IS unit. The participants were selected to represent a variety of technical specialties, such as systems analysis, DBMS, communications and logistics, who were involved in systems development projects. Also, two user representatives, one a marketing manager and the other a shipping dispatcher, also participated. During these sessions, each participant received a copy of each of the seven CSC maps. The figures were distributed one at a time, explained, and then discussed. Participants were asked to think about ideas for feasible IS to address each
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
6
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
PACK TRUCK IN REVERSE DELIVERY ORDER
DELIVERYORDERED PACKING LIST 8
EARLIER DELIVERIES
1
DRIVER ROUTING ANALYSIS
SYSTEM ATTRIBUTES Information and analysis for better scheuling, routing and packing
HIGHER MARGINS
PREMIUM PRICING 5
8
6
3
1 DELIVER MORE IN A DAY
CRITICAL SUCCESS FACTORS
Faster/earleir deliveries to customers
SERVE MORE CUSTOMERS
2
BUILD MARKET SHARE
2 FASTER DELIVERIES
REAL TIME TRAFFIC INFO
1
6
2
1
DELIVERY FLEXIBILITY
FRESHER PRODUCE/FISH
1
REPUTATION FOR QUALITY
ORGANIZATIONAL GOALS Better service and higher quality productsimpact reputation, market share, and margins
Figure 4. CSC network map for delivery effectiveness and speed at WFF. of the CSC. They were told that the objective of the meeting was to use their expertise, without resorting to outside information, to produce 'back-of-the-envelope' level proposals that briefly described feasible IS projects that addressed performance in terms of the models described by the CSC maps. The projects should be specified briefly in terms of a project name, description, the likely project architecture, resources required, cost, risk, and expected impact on the organization. The richness of the information in the models helped the participants in the technical workshops to focus on generating project ideas consistent with the models of firm performance embedded in the CSC. In addition, technical staff and users discussed details of desirable IS features for those CSC maps for which relevant users were in attendance. Workshop participants also had access to the original individual chains on which the maps were based. They made use of this access for one of the maps. The workshops resulted in the generation of 14 IS project ideas. An example of one of the ideas is shown in Table 1. It involves a modestly scaled idea for a decision support system for scheduling, routing, and preparing trucks for deliveries. Several of the proposals were for larger projects, however, most of the project ideas were small in scale. Three of the ideas involved support activities for existing systems, including two training process ideas and one that involved updated equipment and additional maintenance support.
5. Conclusory discussion. Clearly, information systems are potentially very important to firm performance [24, 8]. Consequently, it is critically important to identify IS projects with the maximum potential for positive impact on achievement of the firm's goals. Better information has been found to be a determinate of IS planning success [27]. The CSC method extends critical success factors to support the effective use of knowledge distributed around the organization for strategic IS planning. In addition, it provides a mechanism to support understanding of participant motivations for preferring system features. Using the CSC method, the WFF IS group was able to make use of knowledge from a diverse group of internal and external customers about the IS features that were expected to be important to the firm. Participants were not required to understand the method or the CSC concept. The method permitted “bottom-up” ideas to be used as stimuli in a process that produced information about what system features were critical to the firm and why. In addition, participation by a large number of employees and customers was considerably less expensive than it would have been using traditional CSF methodology. Twenty-five participants required an average of about 40 minutes each for interviews, compared with three to six hours each, as suggested by Rockart [30]. Clearly, the use of the CSC method made practical
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
7
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
participation by a larger number of employees and customers. The CSC method calls for an idea generation workshop, which in the case of WFF, required two full days of participation for six professionals. Of course, when IS planning uses traditional CSF, staff time must also be used to translate CSF into project ideas, however, this stage of the CSC method has no direct analogy in the CSF concept. Table 1. WFF dispatch decision support system. Title
Dispatch Decision Support System
Description
Provides semi-structured support for using data from order processing, inventory, geography, marketing to help dispatcher plan deliveries, to achieve marketing priorities, high delivery effectiveness and efficiency
Likely Architecture
Windows NT workstation, communication links to inventory, order processing, and procurement, and accounts receivable systems. General communication links. Custom written templates for optimization packages and custom developed 4GL applications.
Resources
Inside high-level champion, one full time outside contract application developer, 3-5 business & 1 technical staff (part time) for 4-6 months.
Cost
$100-200K.
Risk
Narrow focus, well-defined objectives limit risk. Risk limited to cost of system. Worst case scenario: little or no negative affect on operations.
Impact
Revenue, reputation, margin, and market share potential gains expected, but difficult to quantify. "Behind the scenes" system difficult for competitors to copy.
The CSC maps produced for WFF were far more informational than simple CSF statements. CSF are performance objectives, but CSC maps represent full models, socially constructed and specific to the firm, about the perceived connections between IS attributes, performance objectives, and firm goals. Researchers have recognized the importance of understanding, based on collection, interpretation, and communication of rich information, to IS planning, rather than reliance solely on the deterministic use of normative firm models [2]. IS group personnel in the idea generation workshops thought that the CSC maps helped them to understand their customers' motivation for preferring specific IS features. Involvement of the IS group's customers in the CSC method also helped WFF to identify important nonstrategic IS project ideas for the firm. With a "top-down" process, important projects that don't address the strategic needs of the firm, probably would not be addressed. In a "bottom-up" process, planners are inundated with project
ideas, most of which may not have a positive value for the firm. An unforeseen result of the case was that proposed IS solutions did not always require the development of major new IS applications. The richness of the information provided helped the workshop participants to consider relatively inexpensive solutions to customer needs, such as the development of enhanced training support.
5.1. Contributions This study introduces the CSC method to the IS executive and makes a case for its use in IS planning. For the CIO, IS users throughout the organization are internal customers, with whom IS planning is not the only, and perhaps not the most critical, relationship. The IS group may be expected to perform a range of consulting, maintenance, operational, and developmental activities for these customers. CSC is a planning methodology that treats IS users as customers. There is good reason to think that the CSC method may be successfully applied in practice and that IS executives would find the method attractive. It is very similar to a method used by one of the authors in a market research consulting practice. There it is used to understand how customers associate product features with expected consequences and personal values. The resulting “meansends-values maps” are used to plan advertising positions and to develop ideas for new product features. It has been successfully applied to develop product feature ideas and marketing strategies for a number of US 'Fortune 100' firms. In practice, the CSC method has several advantages for use in IS planning. It allows planners to make use of knowledge distributed throughout the organization about important potential IS projects. First, allows effective participation of many employees in the firm without requiring them to understand CSF concepts. Secondly, it converts self-serving, suboptimal ideas into models of important firm relationships. Finally, it is economical. By requiring short participant interviews, it makes practical data collection from many participants in the firm. The understanding gained by obtaining richer results and by involving more people in the planning process may be well worth the cost. As a heuristic, the authors’ marketing research customers often budgeted 1-5% of an advertising budget for such a study. If the study improved the advertising positioning even slightly, the payoff was excellent. By analogy, it might be worthwhile to spend $50,000 for a CSC study, if the IS capital budget is $1-5 million over 2-3 years. In marketing the method helps researchers to better understand the meaning of what customers are seeking by understanding the customers’ association of product features, consequences, and personal values. In IS
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
8
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
planning, it helps planners to better understand internal customer motivations for preferring IS features by mapping customer models of the relationships between IS attributes and firm performance. This knowledge richness helps technical development staff to propose IS projects that better target the firm's goals. In many cases, these project ideas may be relatively small and inexpensive. This paper provides the CIO with a guide and a template for use of the CSC method. Figure 2 provides an overall graphical roadmap for the CSC method and the four-step description of the method specifies the procedure. In addition, we have provided a case scenario that is a composite of successful implementations of the method, presented in an industrial setting that most readers can easily understand. Nothing would appear to prevent the CSC method from being combined with traditional CSF interviews for the top executives or with other planning methodologies, such as surveys. Indeed the results of a survey might be used as input into the CSC method.
5.2. Limitations The CSC method is intended to be applicable for use in practice. Consequently, we have attempted to completely specify here how the method may be applied. Arguably, the interviewing and analysis required to implement CSC may seem complex. One of the authors has written software to automate portions of the analysis; however, its interface has not been refined sufficiently to support use by casual users. Consequently, in many organizations, a CSC study might be most effectively conducted by an internal or external specialist consultant. The CSC method is a model generation and idea generation method, rather than a decision-making process. Decisions about which proposed ideas should be undertaken by the organization are beyond the scope of the method. Discrimination among the resulting outcomes requires the judgment of managers in the appropriate organizational venue, e.g., IS steering committee. The CSC models are unprioritized aggregate models. There is no attempt to prioritize them, except that, when large sample sizes result in extremely complex models, outlying concepts may be pruned to make the models understandable. It was not necessary to prune any such concepts in our studies. The results of the CSC method may depend, in part, on the project idea stimuli, the participant sample, and the workshop participation. The project idea stimuli should be representative of the range of ideas in the study organization. The method of stimulus collection presented here is economical and may be representative of the range of ideas present in the organization to the extent that the participant sample is representative. The participant
sample is expected to be representative of a variety of occupations and managerial levels in the organization, however, participants may not be represented in proportion to their numbers, knowledge, importance in the organization. For example, if an organization has one CFO, one executive chef, and 50 desk clerks, there is neither an attempt to represent them in proportion to numbers or to weight them by importance. The outcome of the idea generation workshops is a function of the professional judgment of the participants.
5.3. Future directions Future research can branch in several directions. One limitation of the CSC method, addressed by workshop participants, is that the CSC maps are at a fairly abstract level. Consequently, they are too simple to be useful for specifying detailed system requirements. The CSC method may be adapted for use in IS project requirements determination by restricting the scope of a study to the narrow domain of a single project idea. Other potential applications include business process reengineering and general strategic business planning. Several other options exist to experiment for potential improvements in the CSC methodology. It may be interesting to differentiate between models developed from participation by members at different organizational levels or to try to understand the extent to which the results depend on variation in the sample participant selection. In data collection, it may be better to offer participants a wider choice of stimuli. In our demonstration, we showed participants three system ideas of which we used two to develop chains. By offering a wider choice, say five choices, of which two are used, we would give participants more control over the subject of their contribution. This might reduce the potential for variance that results from the sample of stimuli. Finally, it is important that the results of a planning process gain wide support for implementation [13]. Does widespread participation in the planning process help the organization achieve important project "buy-in" when the time comes to design and implement the planned projects? We are not sure, but this is a question that seems well worth investigating.
6. References [1] Bergeron, F. and Begin, C. “The use of critical success factors in evaluation of information systems: a case study,” Journal of Management Information Systems, (5:4), 1989, pp. 111-124. [2] Boland, R.J. Jr., Tenkasi, R.V. and Te'eni, D. " Designing information technology to support distributed cognition," Organization Science (5:3), 1994, pp. 456-475.
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
9
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
[3] Boose, J. H., "Uses of repertory grid-centred knowledge acquisition tools for knowledge-based systems," International Journal of Man-Machine Studies. (29:3), September 1988, pp. 287-310.
Evaluation Quality," Journal of Management, (6:4), 1998, pp. 1-11.
Global
Information
[19] Kelly, G.A., The Psychology of Personal Constructs, W. W. Norton and Co., Inc., New York, NY, 1955.
[4] Bowman, B. J., Davis, G. B., and Wetherbe, J. C. "Three Stage Model of MIS Planning," Information & Management (6:1), 1983, pp. 11-25.
[20] Kendall, K.E. and Kendall, J.E. Systems Analysis and Design, 4th ed., Prentice Hall, Upper Saddle River, NJ, 1999.
[5] Boynton, A.C. and Zmud, R.W. “An assessment of critical success factors,” Sloan Management Review, (25:4), Summer 1994, pp. 17-27.
[21] Klenosky, D.B., Gengler, C.E., and Mulvey, M.S. "Understanding the Factors Influencing Ski Destination Choice: A Means-End Analytic Approach," Journal of Leisure Research (25:4), Fall 1993, pp362-379.
[6] Byers, C.R. and Blume, D. “SOS Tying critical success factors to systems development,” Information & Management (26:1), 1994, 51-61. [7] Daniel, R.D., “Management Information Crisis, HBR, (39:5), Sept-Oct 1961, pp. 111-121. [8] Dos Santos, B.L. and Peffers, K. "Rewards to Investors in Innovative Information Technology Applications: First Movers and Early Followers in ATMs," Organization Science (6:3), May-June 1995, pp. 241-259. [9] Earl, M.J. "Experiences in Strategic information systems planning," MIS Quarterly, (7:1), 1993, pp. 1-24. [10] Ford, K.M., Petry, F.E. and Chang, P.J. “An Approach to Knowledge Acquisition Based on the Structure of Personal Construct Systems,” IEEE Transactions on Knowledge and Data Engineering (3:1), 1991, pp. 78-89. [11] Fromm, M. "What students really learn: Students' personal constructions of learning items," International Journal of Personal Construct Psychology (6:2), Apr-Jun 1993, pp. 195208. [12] Gaines, B.R., and Shaw, M.L.G. “Knowledge Acquisition Tools Based on Personal Construct Psychology,” The Knowledge Engineering Review (8:1), pp. 49-56. [13] Galliers, R.D, Pattison, E.M, Reponen, T. "Strategic information systems planning workshops: Lessons from three cases," International Journal of Information Management (14:1), 1994, pp. 51-66. [14] Gengler, C.E., Klenosky, D.B. and Mulvey, M.S. "Improving the Graphic Representation of Means-End Results," International Journal of Research in Marketing (12:3), 1995, pp. 245-256. [15] Gengler, C.E., Howard, D.J. and Zolner, K. "A Personal Construct Analysis of Adaptive Selling and Sales Experience" Psychology and Marketing (12: 4), 1995, pp. 287-304. [16] Gengler, C.E. and Reynolds, T.J. "Consumer Understanding and Advertising Strategy: Analysis and Translation of Laddering Data," Journal of Advertising Research (35:4), 1995, pp. 19-33. [17] Grover, V., Teng, J.T.C, and Fiedler, K.D. "IS investment priorities in contemporary organizations," Communications of the ACM (42:2) 1998, pp. 40-48.
[22] Peffers, K. and Gengler, C.E., "Generating Strategic IS Project Ideas Using Empirically Constructed Critical Success Chains (CSC), working paper, Texas Tech University, 1999. Available at http://www.peffers.com [23] Peffers, K. and Saarinen, T. "Measuring the Business Value of IT Investments: Inferences From a Study of Senior Bank Executives," Journal of Organizational Computing and Electronic Commerce 1999. [24] Peffers, K. and Tuunainen, V. “Expectations and Impacts of a Global Information System: the Case of a Global bank from Hong Kong” Journal of Global Information Technology Management (1:4), 1998, pp. 17-37. [25] Pervin, L.A. Personality Theory and Research, 6th ed. John Wiley & Son, Inc., New York, NY, 1993, 563 pp. [26] Premkumar, G. and King, W. R. "Organizational characteristics and information systems planning: An empirical study," Information Systems Research (5:2), 1994a, pp. 75-109. [27] Premkumar, G. and King, W. R. "The evaluation of strategic information system planning," Information & Management (26:6), 1994b, pp. 327-340. [28] Reynolds, T.J., and Gutman, J. "Laddering Theory, Method, Analysis, and Interpretation," Journal of Advertising Research, (28:1), 1988, pp. 11-31. [29] Rockart, J.F., “Chief executives define their own data needs,” HBR, (52:2), 1979, pp. 81-93. [30] Rockart, J.F., “The changing role of the information systems executive: a critical success factors perspective,” Sloan Management Review, (24:1), Fall 1982, pp. 3-13. [31] Sambamurthy, V, Venkataraman, S, and Desanctis, G. "The design of information technology planning systems for varying organizational contexts," European Journal of Information Systems (2:1),1993, pp. 23-35. [32] Shank, M.E., Boynton, A.C., and Zmud, R.W. “Critical success factor analysis as a methodology for MIS planning,” MIS Quarterly (9:2), 1985, pp. 121-129. [33] Warren, W.G. "Personal construct theory and mental health," International Journal of Personal Construct Psychology (5:3), Jul-Sep 1992, pp. 223-237.
[18] Hallikainen, P., Heikkila, J., Peffers, K., Saarinen, T., and Wijnhoven, F. "Evaluating Information Technology Projects: Procedures, Follow-through, Decision-making and Perceived
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
10
