A Design Theory for Systems That Support Emergent Knowledge Processes By M. Lynne Markus Peter F. Drucker Graduate School of Management Claremont Graduate University 1021 North Dartmouth Avenue Claremont, CA 91711 Tel: 909-607-3151 Email:
[email protected] Ann Majchrzak School of Business Administration University of Southern California Los Angeles, CA 90089-1421 Tel: 213-740-4023 Email:
[email protected] and Les Gasser Graduate School of Library and Information Science University of Illinois at Urbana-Champaign 501 East Daniel St., Champaign, IL 61820 Voice: 217.265.5021 Email:
[email protected]
Revision for May conference April 17, 2000
Abstract This paper presents a design theory for systems to support what we call “emergent knowledge processes.” Emergent knowledge processes are business processes that involve intellectual activities, expert knowledge, and diverse people in unstructured and unpredictable combinations. Examples of emergent knowledge processes include the processes of organization design, new product development, and strategic planning. Information systems support for emergent knowledge processes is usually limited to routine decision-making or communication support, whereas more comprehensive support would consist of helping participants with the creative, problem-finding aspects of the processes. Drawing from the literatures on knowledge management and emergent processes, this paper identifies the characteristics of emergent knowledge processes and the characteristics of systems likely to provide effective and comprehensive IT support for the characteristics. Then, drawing on a case study of a successful system development effort for the process of organization design, we discuss the principles of good IS product design and development for emergent knowledge processes. These principles are quite different from those commonly used in system design today. Implications of this design theory for further research are discussed.
04/18/00
1
Introduction The holy grail of the IS field is effective IT support for people’s most demanding intellectual tasks, such as decision-making, planning, and designing. Since the earliest days of the IS field, it has been clear that the quest is daunting. IT has proved far more useful for structured processes than for unstructured; IT support is generally possible only for limited aspects of complex intellectual processes; and there has been only limited success in automating or replacing people’s higher cognitive functions. For example, Todd and Benbasat (2000) have concluded that IT has successfully supported only the problem solving tasks associated with decision-making. But problem finding— including the tasks of uncovering the underlying decision problem, gathering relevant information about it, and diagnosing it— is “fuzzy, difficult, and not amenable to technical support.” Similarly, Davenport, Jarvenpaa, and Beers (1996) have argued that “the abstract and unstructured inputs to and outputs from knowledge work processes … make the application of [information] technology more difficult.” Indeed, the most successful applications of information technology in the domain of knowledge management have been relatively “low tech” communication and document handling systems. Is this the best IS can do for unstructured knowledge processes? Or is it possible that we’ve been hindered by the lack of a “design theory” tailored to the unique characteristics of these processes? According to Walls, Widmeyer, and El Sawy (1992), design theories are prescriptive theories, based on scientific theory, technical information, and imagination, that explain how the process of designing an information system can be feasibly and effectively carried out for a particular set of requirements. Walls et al. showed that a design theory requires both a unique type of IT solution (IT product design) and a unique design strategy (IS development approach) to satisfy unique “metarequirements”. The features of the design and the development approach must correspond, and what unites them is a “kernel theory” of the type of business process one is trying to support. This notion of a design theory is very important for the IS field, because it provides a sound conceptual basis for ideas we teach in every basic IS course: good decision support systems have different features from good transaction processing systems (and communication systems, expert systems etc.), and the process of building good decision support systems is different from that of building good transaction processing systems. Walls and colleagues were concerned with a category of systems they called “vigilant EIS,” information systems that can enhance the effectiveness of executives in identifying and diagnosing emerging strategic problems and opportunities in turbulent
04/18/00
1
environments. In this paper, we are concerned with what we call “emergent knowledge processes,” that is, processes involving intellectual activities, expert knowledge, and diverse people in unstructured and unpredictable combinations. A good example of an emergent knowledge process is the process of organization design (also called sociotechnical redesign). Litterer and Jelinek (1983) studied organization design and concluded that the process is: •
synthetic, taking smaller elements and combining them into a larger whole
•
multivariate, focused on interrelationships, limits, and constraints
•
intuitive, inductive, incremental, non-linear
•
aimed at creating a workable solution, not the best solution
•
always a case of redesign, not of designing from scratch
•
invoked infrequently and only if there is no other means to handle a problem
•
a process of identifying solutions through examining the contingencies, limits, and tradeoffs of alternative organizational patterns measured against multiple criteria, with the end result typically some adaptation and combination of different patterns.
Similarly, socio-technical redesign has been described as both knowledge-based and emergent (Taylor and Felten, 1973). The fundamental principle of minimal critical specification, when applied to the socio-technical design process, means that the process should not specify any more detail than is minimally needed, focusing only on specifying outputs (for example, criteria to which plans must adhere) rather than the process by which the outputs are derived (Cherns, 1976; Cherns, 1986) Additionally, the principle of “incompletion” (Cherns, 1976; Cherns, 1986) means that redesign is continuous, so that new needs will unpredictably arise. Thus both the outcome of design and the design process itself must be flexible to embrace these changes. These characteristics of the organization design process make traditional notions of IS requirements analysis and structured system design difficult to implement. Because the requirements for an emergent knowledge process cannot be known in advance, one might conclude that the process is not amenable to comprehensive IT support, allowing only limited support for interpersonal communication and information storage. Yet, two of the authors 1 created a system, called TOP Modeler, that was successful in providing broad, not just partial, support for the process of organization design, including its most intellectual, knowledge-intensive aspects. Both the IT product and the IS development process were quite different from the prescriptive literature on IT products and IS development processes. But they fit each other well, and in retrospect it is clear that they fit an evolving kernel theory of emergent knowledge processes.
04/18/00
2
In this paper, we present the basic elements of a design theory of emergent knowledge processes. We focus on theory, first, then use our case study of the TOP Modeler system to illustrate the design theory. The paper is organized as follows: 1) a review of theory and research related to emergent knowledge processes, 2) the characteristics of emergent knowledge processes and the meta-requirements for systems to support them, 3) background on the case of TOP Modeler, 4) principles for IS product design and for the IS development process with illustrations from the case of TOP Modeler, 5) conclusions about the generality of our theory and implications for future research. While the process of organization design is in many ways unique, we believe that its essential characteristics as an emergent knowledge process can also be found in many other situations, including business intelligence and corporate strategic planning. Thus, we hope that the design theory we articulate in this paper will help other developers tackle situations that have not previously been viewed as amenable to IT support.
A Kernel Theory of Emergent Knowledge Processes The literatures that bear on emergent knowledge processes are many and divergent. There is a sizable body on research about knowledge workers and the organizational contexts of their work (Frenkel, et al., 1999). There is a growing popular and academic literature on “knowledge management” (Davenport and Prusak, 1997), “knowledge management projects” (Davenport, et al., 1998) and “knowledge management systems” (Alavi, 2000). Further, in the field of technology and innovation management, there is a considerable literature on knowledge creation and the development and management of core competencies (Leonard-Barton, 1995; Nonaka, 1991). Finally, there are scattered discussions of what we call “emergence” (Orlikowski, 1996; Truex, et al., 1999). Our very brief review abstracts those elements most relevant to emergent knowledge processes. Knowledge Processes One branch of knowledge management grew out of the IS field and the business process reengineering movement. (Davenport and Prusak, 1997) A central concept in this branch is that of “process,” defined as “a specific ordering of work activities across time and place, with a beginning, an end, and clearly identified inputs and outputs: a structure for action” (Davenport, et al., 1996). Process management is the set of tasks involved in “mapping” the process, streamlining it, and implementing changes to achieve higher levels of performance on metrics reflecting the whole rather than the specific activities within it.
04/18/00
3
Most process management has been attempted in the arena of operational and administrative processes. Many believe that process management has considerable potential when applied to knowledge work, but “there are also significant challenges in its application” (Davenport, et al., 1996). Knowledge processes are described as “untidy… the inputs and outputs of knowledge work—ideas, interruptions, inspirations, and so on are often less tangible and discrete. There are no predetermined task sequences that, if executed, guarantee the desired outcome. Knowledge workers may operate by an intuitive feel for how to accomplish their work or through accumulated experience” (Davenport, et al., 1996). This view of knowledge work processes is consistent with the conclusions of observers such as Suchman (1987), who documented unstructured processes even in clerical and administrative work, and Pava (1983), who argued that knowledge work might better be described in terms of diffuse deliberations than as a process in the traditional sense of an orderly sequence of activities. Even when the process element of knowledge work is relatively clear, it is often difficult to develop appropriate throughput and output measures; hence, knowledge work is often measured by inputs, making true process management infeasible (Davenport, et al., 1998). The logical result is that it is difficult to support knowledge processes with IT (Davenport, et al., 1996). When IT has been applied in knowledge management projects, it has mainly been interpersonal communication systems such as email and document repositories such as Lotus Notes. Workflow and expert systems have had much more limited application. Making matters even more difficult, knowledge workers can have a high degree of autonomy in how they do their work, and they can resist the imposition of standard routines and new technologies in their work. There is considerable variation in the performance of knowledge workers (Davenport, et al., 1998; Frenkel, et al., 1999). Knowledge As Process A second major stream of knowledge management literature has also used the word “process,” but very differently from the way that term is used by the “knowledge process” researchers cited above. Proponents of the “knowledge as process” view claim to focus on the flows of knowledge among people, rather than on knowledge as an object to be stored, accessed, and reused via knowledge management processes and knowledge repositories (Fahey and Prusak, 1998). In this stream, the primary focus is on individual and organizational learning and the social interactions involved in knowledge creation and creative recombinations of knowledge (particularly involving R&D and new product development). For example, Dorothy Leonard-Barton describes a cycle of core
04/18/00
4
competence development that involves learning from external sources, solving current problems, implementing and integrating solutions to those problems, and proactively experimenting (Leonard-Barton, 1995). Nonaka and Takeuchi (1991) describe a spiral of organizational knowledge creation involving individuals, small groups, and the organization in activities that include learning, socialization, making tacit knowledge explicit, and combining explicit knowledge in new ways. In this view, knowledge is created socially by a “community of practice” drawn together by the practitioners’ work (Brown and Duguid, 1998; Spender, 1996; Wenger, 1998). Thus, knowledge is said to be “situated” in practice, implying that knowledge evolves as people work together on the problems of a common domain of practice. It further means that knowledge is highly specific to the local context, and that useful knowledge is knowledge that supports action. Because of their shared framework of knowledge, communities of practice can become closed to new ideas. “New knowledge often requires new forms of evaluation” to avoid self-deluding social behavior (Brown and Duguid, 1998). Emergent Processes Finally, throughout the literatures on IS and knowledge management, there are scattered references to the property of “emergence.” For example, Orlikowski reminds us that however well planned an initial sequence of actions, the situation that emerges from action has new characteristics that call for additional responses (Orlikowski, 1996). Truex and colleagues (1999) refer to situations in which “every feature of social organization— culture, meaning, social relationships, decision processes and so on—are continually emergent, following no predefined pattern”. They argue that emergent processes pose special challenges for IS development, among other reasons because “complete and unambiguous specifications are ineffectual” (Truex, et al., 1999).
Characteristics of Emergent Knowledge Processes and Requirements for Systems That Support Them Taken together, these literatures form what Walls et al. (1992) call a kernel theory of emergent knowledge processes. Abstracting from and synthesizing these literatures, we identified 10 characteristics of emergent knowledge processes, classified into the three categories of process, knowledge, and people. Below, we briefly describe each characteristic, provide an example from the context of organization design in manufacturing settings, and derive the “meta-requirements” that a good information
04/18/00
5
system would support. (See Table 1, columns 1 and 2, for a summary of the characteristics and the systems support requirements).
Insert Table 1 about here
Characteristics of the process of emergent knowledge processes The first four characteristics pertain to the processual aspects of emergent knowledge processes. Characteristic #1: Many scenarios There are many situations in which an emergent knowledge process can be performed, and new situations can arise at any time. “Situation” here refers to the “who, what, when, where, and why” of a process. Emergent knowledge processes are sometimes performed by individuals, other times by groups. The individuals and groups may vary substantially in backgrounds and expertise. And the mix of backgrounds and expertise brought to bear on an emergent knowledge process can differ each time the process is performed. Organization design is triggered by a diverse set of events, such as the building of a new plant, the acquisition of new machinery, the requirement to build a new product, or the perception of operational problems. Sometimes, organization design is conducted by individuals acting alone, such as a foreman or an industrial engineer. Other times, it is conducted by a group, perhaps even facilitated by expert consultants from industry or academia. On occasion, virtually every functional area might be involved in some way; on another occasion, the process may be undertaken by a group with a single functional orientation. Meta-requirement for a Supportive Information System. A system that effectively supports an emergent knowledge process must be flexible enough and reconfigurable enough to work in a wide range of situations, not all of which are known in advance, and it must meet the diverse and sometimes contradictory needs of different user groups. Characteristic #2: Deliberations and tradeoffs In an emergent knowledge process, the work is largely intellectual, that is, cognitive and deliberative, rather than procedural, physical, or clerical. The work consists
04/18/00
6
of tradeoff analyses, which may occur largely in the mind, and deliberations, which may be formal and semi-structured or highly informal and diffuse. These deliberations may involve one or more people or groups of people and they may occur over a protracted period of time. In addition, the extent to which consensus is required in deliberations involving emergent knowledge processes is unknown. For example, in a study of strategic decision-making, Fiol (1994) concluded that consensus was required only on the number of issued involved in starting up a new venture, but not on the specific resolution of each issue. In organization design, analysts weigh a variety of incommensurate factors such as equipment capacities, human skills and training, degree of autonomy desired or permitted by culture or legalistic work rules, etc. The designer generates hypotheses about effective relationships and tests them in action and/or discussions with others. Meta-requirement for a Supportive Information System. A system that effectively supports an emergent knowledge process must take advantage of users’ knowledge, provide users with knowledge about relative tradeoffs and analyses, encourage consideration of alternatives, and provide information about alternatives that can be used in “off-line” deliberations. For example, in a study of senior executives using executive support systems, Vandenbosch and Higgins (1995) found that systems designed to answer specific questions did not foster creativity; only when systems were designed to promote scanning were creativity and new problem formulation fostered. Walls et al. (1992) make similar arguments. Characteristic #3: Unstructured, non-optimal sequences In an emergent knowledge process, the work is largely unstructured and to a great extent unstructurable. There is no optimal sequence of tradeoff analyses or deliberations that will lead to an optimal result. Consequently, human discretion, intuition, and creativity are essential to successful process execution. In the process of organization design, new information may arise at any point in the process, requiring analysts to revisit earlier decisions. Expert authorities on organization design disagree among themselves about the appropriate steps to take. The best solution may be blocked by some situational constraint. At its most basic, the process of problem finding and solution generation is highly creative. Meta-requirement for a Supportive Information System. A system that effectively supports an emergent knowledge process must support users’ intuitions without imposing a sequence of tasks. In Mark Silver’s terms, the system should provide non-directive, non-restrictive guidance (Silver, 1991). Davenport et al. recommend that
04/18/00
7
this be done by disaggregating knowledge into smaller units that can be reassembled into customized configurations (Davenport, et al., 1996). They give the example of idea modules in publishing that can be recombined in different ways for textbooks, courses, or media. Characteristic #4: Evolutionary process The content and structure of the deliberations in emergent knowledge processes change over time as new situations arise, as new people participate, and as new knowledge is gained. Catalysts of the organization design process emerge in unpredictable ways, sometimes resulting from external competitive forces and sometimes resulting from internally generated needs for higher performance. Moreover, as the purpose and focus of organization design change over time, so too do the people involved in the process. Formerly, most work teams involved members of one organization. Today, new technology makes virtual structures possible, and outsourcing makes multi-organizational teams commonplace. For a process to be robust, it must be able to adapt to such changing circumstances. Meta-requirement for a Supportive Information System. A system that effectively supports an emergent knowledge process must allow for a range of future adaptations. Characteristics of the knowledge in emergent knowledge processes These next three characteristics relate to the knowledge involved in emergent knowledge processes. Characteristic #5: Embedded action The knowledge created and used in emergent knowledge processes must be both general, so that it can be applied in many unforeseen contexts, and highly specific to a particular practical context. More importantly, knowledge must be actionable in application; good knowledge enables successful action. In organization design, there is a body of relevant general scientific knowledge. In addition, successful organization design on the factory floor requires great knowledge of, and sensitivity to, local circumstances such as union rules, management politics, and the capabilities of particular workers. Useful organization design knowledge helps people to interpret and act on the basis of both general and specific information.
04/18/00
8
Meta-requirement for a Supportive Information System. A system that effectively supports an emergent knowledge process must be directed at improving offline behavior and must tie knowledge to concrete practical action. In a review of 30 knowledge management projects Davenport el. al (1996) found that the objectives for knowledge management projects were generally less ambitious that those of administrative and operational reengineering projects, which often require tenfold improvements. Here, however, we are arguing for a much more ambitious goal than that found in many reengineering projects: we aim to influence off-line behavior as well as online actions. Characteristic #6: Distributed knowledge The knowledge created and used in emergent knowledge processes is distributed among multiple parties. No one individual or group has a complete grasp of both the general and specific knowledge that applies. This is clear from Leonard-Barton’s (1995) study of continuous improvement at Chaparral Steel and of Hutchin’s (1990) study of team navigation. In organization design, we see the distribute nature of knowledge at work, both in scientific theory building and in practice. The domain of organization comprises knowledge from multiple academic fields: organizational behavior, information systems, technology and innovation management, industrial engineering, manufacturing engineering, quality, just to name a few. Within an organization, the relevant specific knowledge may be distributed across the functions of human resources, industrial engineering, manufacturing engineering, quality, shopfloor supervision, and the work units themselves. Meta-requirement for a Supportive Information System. A system that effectively supports an emergent knowledge process must incorporate the frameworks and perspectives of several different kinds of participants. Characteristic #7: Evolving knowledge The knowledge involved in emergent processes changes over time as both general and specific knowledge grows and changes. In research and development contexts, knowledge is always accumulating; because new knowledge generation depends on an evolving prior base of knowledge, it is said that knowledge is difficult to “absorb” (Cohen and Levinthal, 1990). In organization design, the evolution of knowledge has been clearest in discussions of the emergence of virtual organizations and networked organization forms.
04/18/00
9
Meta-requirement for a Supportive Information System. A system that effectively supports an emergent knowledge process must have a sufficiently flexible architecture to allow major change in the structure and representation of knowledge. Characteristics of the people who perform emergent knowledge processes This last set of characteristics refers to the knowledge workers who perform emergent knowledge processes. Characteristic #8: Unknown participants In emergent knowledge processes, it is not possible to know in advance who will be involved. This is a direct consequence of the “many scenarios” characteristic of emergent knowledge processes. As a result, the IS designer cannot know in advance who will be the user of a potential support system. In organization design, the characteristic of unknown participants is observed when, in one organization, only managers are involved, whereas in another organization, design efforts are the responsibility of worker teams. It is not possible to make simplifying assumptions about the levels of expertise of users or the ways in which they might use an organization design support system. Meta-requirement for a Supportive Information System. A system that effectively supports an emergent knowledge process must employ terms, operations, and an interface that are usable by participants who are unknown in advance. Characteristic #9: Infrequent performers The people who participate in emergent knowledge processes may do so only infrequently or in circumstances that do not recur. Therefore, they have little opportunity to develop or maintain expertise in the process. Further, as knowledge workers, they may retain considerable discretion over their participation in training; they may resist training if they believe themselves too busy or the training of no lasting benefit. In organization design, some people may participate only once. If the effort is major, say, the design of a new plant, a substantial investment in training may be warranted. But in many situations where better knowledge of organization design could be beneficial, an investment in training may be perceived not to be worth the cost. Meta-requirement for a Supportive Information System. A system that effectively supports an emergent knowledge process must not be dependent on formal training in the knowledge, process, or operation of the system; however, the system must supply non-directive guidance (Silver, 1991).
04/18/00
10
Characteristic #10: Discretion over work methods As knowledge workers, the participants in emergent knowledge processes may have considerable discretion over their use of methods and tools. In the IS literature it has long been known that mandating use of an information system is rarely the best implementation tactic with managerial and professional users. For example, whereas the use of transaction processing systems is often mandatory, the use of decision support systems is often voluntary (Alavi and Joachimsthaler, 1992). The people who participate in organization design processes are no exception to the general rule. Consultants, engineers, human resources specialists as a rule do not take kindly to efforts to control how they do their jobs. Meta-requirement for a Supportive Information System. A system that effectively supports an emergent knowledge process must sell itself to users, since it is not possible to enforce system use. Moreover, as Todd and Benbaset (2000) have recently speculated, decision support systems will only lead to a more accurate decision strategy when the system is as easy to employ as a simple decision heuristic. Thus, for a system to sell itself to users involves more than presenting a pleasing interface but also requires ensuring that decisions are easier to make with the system than without it. This is a tall order when the system also presents more breath and depth of information with more analytic options than are possible with off-the-cuff decisions. Nevertheless, the desideratum of knowledge management systems is precisely this: motivating and enabling the user to make better decisions with the system more easily than is possible without the system.
From Requirements to Design Principles—The Illustrative Case of TOP Modeler In the section above, we presented the characteristics of emergent knowledge processes and their requirements for supportive information systems. A fully developed design theory, according to Walls et al. (Walls, et al., 1992), does not simply espouse requirements, but proposes ways the requirements can be achieved—design principles for the information system product and for the process of design. A few years ago, we participated in developing a system called TOP Modeler to support the emergent knowledge process of organization design. We use this case below to illustrate how the theoretical requirements for a support system were instantiated into design principles. The use of design principles and analogous concepts in the IS context has been discussed by (Markus and Keil, 1994; Weill and Broadbent, 1998; Davenport, et al., 1989). For each design principle, we describe the key characteristics and illustrate how it was
04/18/00
11
manifested as a set of features in TOP Modeler. In addition, each design principle has implications for how the development process unfolds. In this way, we have filled out the requirements for a design theory as suggested by Walls, et al. (1992)—a kernel theory, requirements for a supportive IS, design principles for the supportive IS, and a process for designing the system. In the concluding section of this paper, we discuss the generality of the resulting design theory. Purpose of TOP Modeler The system, called “TOP Modeler” (in which TOP stands for Technology, Organization, and People integration), resulted from a multi-year, $10 million project involving resources from five companies (Hewlett Packard, Texas Instruments, Digital, General Motors, Hughes), an industry consortium (National Center for Manufacturing Sciences), academic experts (specializing in job design, performance management, organization design, skills and training, human factors, information systems, and manufacturing technology), a software development team, and project leaders specializing in socio-technical systems theory and intelligent information systems. TOP Modeler was designed to provide decision support to people and groups involved in organizational redesign. The need for TOP Modeler sprang from two interacting facts: the competitive drive of many manufacturers to implement new technologies—such as CAD, CAM, ERP, MRP, robotics, and programmable automation—and the very high failure rates of most such implementations (Majchrzak and Meshkati, 2000). Research has clearly indicated that a primary reason for manufacturing technology implementation failures is the lack of pre-planning for the integration of organizational as well as technological changes (Majchrzak, 1988). Further research indicated that decision-makers would welcome a tool to identify and resolve socio-technical issues (Majchrzak and Finley, 1995; Majchrzak and Gasser, 1992). Our early action research showed a manual tool called HITOP to be useful in ensuring that critical issues were not overlooked during the planning phase (Majchrzak, et al., 1991). However, the manual tool did not ensure that users used the right knowledge to resolve socio-technical integration issues. For example, in one documented use of HITOP, engineers believed that a new technology would not require changes in the incentive structure. Later, this belief was proved incorrect. Experiences like this sowed the idea of a computer-based decision support system that would host a knowledgebase of best practices. This new tool would both encourage decision-makers to attend to the complete range of critical socio-technical issues and help them understand how best to resolve the issues.
04/18/00
12
TOP Modeler’s Features and Functions TOP Modeler contains a knowledgebase of best practices in organization design. The system allows a user to assess his/her own organization relative to the ideal best practices (a gap analysis) and to prioritize the gaps for improvement. The knowledgebase describes twelve sets of organizational features necessary to control each of 21 “process variances”2 (with four alternative control strategies for each) and to satisfy each of seven manufacturing organization objectives 3 (with three levels of criticality for each). The twelve sets of features are: norms, skills, customer involvement, discretion, organizational values, employee values, production process characteristics, reporting structure characteristics, technical system characteristics, performance measurement and rewards, information resources, and production tasks. Each feature set contains from five to 30 separate specific features, such as “reading and writing capabilities” for the “skill” feature set. In total, TOP Modeler allows users to specify over 300 features of an organization, each of which might be needed to meet a specific objective or process variance control strategy. The knowledgebase of best practices was derived from socio-technical systems theory, utilizing such fundamental principles as “organizations are effective only to the extent to which their organizational and technology structures are jointly optimized” and the principle of “requisite variety” (that is, organizations can adapt only to the extent that variability within the organization corresponds to variability in the environment) (Cherns, 1976; Cherns, 1986; Trist and Murray, 1993). Using these principles, academic experts were commissioned to summarize the empirical literature for each of the twelve feature domains into sets of rules for predicting when an organization would be effectively organized. These rules were then quantitatively validated using data gathered during intensive three-day site visits to 93 electronics manufacturing companies in the U.S. This validation effort demonstrated that the knowledgebase was able to predict with statistical significance differences between manufacturing firms achieving different levels of throughput time, a critical measure of effectiveness for electronics firms (Majchrzak, 1997). The system was judged to be successful on several criteria by all parties involved. First, the project produced its expected deliverable—a deployed knowledge—based system-on time and within budget. Second, the project eventually led to creation of a commercial product (TOP Modeler) and a new company to sell the product (TOP Integration, Inc; http://www.topintegration.com). Third, an evaluation study of naïve users in several organizations indicated that the users changed their behavior as a result of using the system. Finally, the system has been used in over two-dozen “real use”
04/18/00
13
situations of organizational redesign, involving over 3 dozen naïve users in more than ten organizations. Example situations of use include redesigning an automotive prototype building shop, evaluating a printed circuit board manufacturing facility for departures from industry-wide best practice, creating a cohesive strategic vision for a 1200 person assembly plant, and designing teams for a maintenance support group. In a structured evaluation of 19 of these uses, users reported that the system was useful in fostering new learning about organizational design and in changing the behavior of people redesigning their organizations. In addition, it helped users reassess their business strategies, clarify business issues, learn more about their organizations, and achieve consensus on important organizational design issues. One of these uses, for example, occurred in a high technology subsidiary that was considering moving its operations from Singapore to Thailand. The system-assisted evaluation revealed a number of weaknesses in the Thai plant, and the planned move was stopped. Another case involved a proposed joint venture between a U.S. manufacturer and a Chinese counterpart. Use of the system helped surface previously unrecognized differences in the strategic directions of these two organizations. The joint venture was postponed until these differences could be resolved. Additional cases of TOP Modeler’s use are described in (Majchrzak and Finley, 1995; Majchrzak and Gasser, 2000). The TOP Modeler Development Process The successful development of TOP Modeler was not achieved by following traditional IS development methods. We began with traditional methods, but found that they did not suffice. We studied many recommendations for developing information systems (Davis and Olson, 1985; Satzinger, et al., 2000), decision support systems (Silver, 1991; Watson, et al., 1997), and knowledge-based systems (Turban, 1995). Following such advice, we analyzed the knowledge process of organizational design, identified the knowledge needs of the knowledge workers, and initiated a user-led feasibility study of a system to support this process. We obtained executive sponsorship in each of the involved companies, and we created a management Steering Committee with representatives from each company. We negotiated the involvement of a user group called the “Domain Team.” This group comprised one member from each sponsoring company who allocated 50% time to the project. Each Domain Team member was an expert in how their home organization conducted organizational redesigns. For example, the GM Domain Team member represented Corporate Industrial Engineering because most production process improvements at GM were the responsibility of industrial engineers. In contrast, the Texas Instruments member held a post in a manufacturing business unit because most TI redesigns were done by business unit managers. We also
04/18/00
14
assembled a Development Team that included several Ph.D. computer scientists and a number of full-time and graduate student programmers. Finally, we developed a detailed set of specifications and a project workplan with several prototyping iterations meeting the specified requirements. Once we began to execute this workplan, however, we quickly found that a number of our fundamental underlying assumptions were wrong. We (the authors), the executive sponsors, the steering committee, and the user group had all incorrectly assumed that the accepted processes of IS development would apply to the design of a system to support the particular emergent knowledge process of organization design. For example, we had incorrectly assumed that a set of organization designers could be identified as “users”—the appropriate “target” of our development efforts. And, we had incorrectly assumed that users could be trained before using the system. When our difficulties with such assumptions became clear, we had to dispense with the carefully constructed workplan and system specifications. In its place, we had to develop both new principles for designing the IS product and a new type of IS development process for emergent knowledge process support systems. We now turn to this set of design principles. We have organized the design principles by the corresponding characteristic of the emergent knowledge process. Table 1, columns 3 and 4, summarize the design product and process principles. We developed these principles retrospectively, by examining the key features of TOP Modeler, the features of the development process we eventually used, and the characteristics of emergent knowledge processes.
Design Principles For Emergent Knowledge Processes What follows are the design principles that match the ten characteristics of emergent knowledge processes described earlier in this paper. For each characteristic we present the relevant design product principles and the design process principles illustrated with examples from the case of TOP Modeler. Emergent Knowledge Process Characteristic #1: Many Scenarios Our theoretical proposition that emergent knowledge processes have many scenarios creates the requirement that a support system be flexible, configurable, and capable of contradictions. In TOP Modeler, this led to the following design principles: •
The IS product should support contradictory requirements
•
The IS development process should transform contradictions into “both-and” capabilities.
04/18/00
15
Product Principle #1: Support contradictory requirements and uses Early on, the Domain Team identified many requirements that appeared on the surface to be contradictory. For instance, one specification called for an interface that would lead users through a step-by-step socio-technical analysis, since some users did not know where to begin; an alternative specification called for the system to facilitate varied, unknown types of system use. Initially, we did not see how a single system could effectively address both of these needs. Another apparent contradiction concerned the need to generate designs both for workgroups (e.g., an assembly line) and for whole manufacturing plants. Each option placed different requirements on the size, scope, and ground concepts of the knowledgebase, and they appeared irreconcilable. Other apparent contradictions arose between alternative goals for system use (designing a new organization versus evaluating an existing one), alternative patterns of system use (lots of online interaction versus a series of “batch” runs), and input demands on users 4 . Initially, all parties involved (the Development Team, the Domain Team, the Steering Committee) agreed to work on resolving these contradictions. To resolve the contradictions, the Development Team facilitated meetings in which members of the Domain Team debated intensely. There was no easy resolution, since good arguments were identified on both sides of every issue. Moreover, each time the Domain Team forged a compromise on an issue, observers in the parent organizations interpreted the result as a battle won or a battle lost. With this orientation, we risked alienating part of the user community with each consensus reached. Eventually, the Development Team created a new design principle: that the contradictions among requirements or user perspectives should become the source of creative design solutions, not ignored or resolved away. Said another way, “either-or contradictions” should be reconfigured into “both-and capabilities”. This new orientation shifted our efforts from resolving conflicts to investigating how both sides of an apparent contradiction could be met. For example, in the debate over whether designs should be generated for the workgroup or the plant, we realized that both workgroups and plantlevel personnel would redesign organizations. Thus the system needed to be applicable to both levels. This forced our knowledge representation effort to identify organizational concepts and signifiers that could be used and interpreted regardless of the organizational level at which the tool was applied. Thus, for example, users could usefully apply the feature “percent of unit members with a particular set of skills” regardless of whether the unit being modeled was a small workgroup, a department, or a several-hundred person plant. This solution to the contradictions among levels had an implication for online explanations, however: we needed to generate explanations that helped users understand
04/18/00
16
how TOP Modeler could be applied to analyze at different levels, while also cautioning users to maintain consistent levels of concept application in any given case. Development Process Principle #1: Transform to “both-and” capabilities; dialectical IS development Transforming contradictory requirements into “both-and” capabilities required a dialectical development process. First, we clearly articulated the apparent contradictions as dilemmas with arguments on each side. We then selected an approach to each side of a dilemma and developed a prototype that we hoped would epitomize that approach. To assess how successful we had been, we used criteria appropriate to that side of the dilemma. Next, we went through the same process for the other side of the dilemma. Last, if we liked both versions, we figured out a way to incorporate both in the final product. For example, our first prototype contained a highly flexible interface that supported many different types of use. Some members of the Domain Team were concerned that the flexibility would allow too much opportunity for errors in organization design. Therefore, we experimented with an interface that provided a more restrictive, step-by-step “roadmap” for users to follow. Eventually, we allowed users to switch back and forth at will. We no longer pushed for consensus on whether or not roadmaps should be used, but created a system that made roadmaps an option. From a development perspective, we dropped requirements consensus as an aim, and in fact welcomed requirements contradictions since they helped to alert us to the real needs of a diverse array of users. By focusing development on meeting the range of uses implied by these contradictions, we were able to create a system that met the needs of many different users versus one built for a non-existent “consensus” user. Emergent Knowledge Process Characteristic #2: Deliberations and Tradeoffs Our theoretical proposition that emergent knowledge processes focus on deliberations and tradeoffs creates the requirement that a support system provide knowledge of alternatives and tradeoffs, not simply of the steps or procedures required to reach an answer. In TOP Modeler, this lead to the following design principles: •
The IS product should reveal relationships among the elements involved in tradeoff decisions
•
The IS development process should create “drivable” prototypes to identify critical relationships.
04/18/00
17
Product Principle #2: Represent and reveal relationships; capture and check constraints The earliest version of TOP Modeler was called HITOP-A. It represented knowledge about socio-technical integration in the form of over 1000 if-then decision and control rules. An example of such a rule is: If input uncertainty for a [manufacturing] task is high, then degree of discretion [needed by the person performing the task] is high, where input uncertainty is determined by a set of specific process variances supplied by the user.
When working with this rule, HITOP-A required the user to supply companyspecific information about “process variances”. From it, HITOP-A computed the value of “input uncertainty.” The software then produced a recommendation about how much “discretion” to include in the job (or would use this result as an input to later calculations). In any single run of this system, the user might be asked to provide as many as 100 inputs, defining such issues as business strategies, environmental conditions, the allocation of specific tasks to humans and to machines, and the characteristics of the machines used in manufacturing. The outputs were a set of specifications for jobs, technology, and an organizational structure that matched the configuration of user inputs. We found that this knowledgebase had excellent predictive ability, based on tests comparing computer-based predictions to those provided by human experts. While HITOP-A’s predictions met expert users’ judgments, the way in which HITOP-A derived its outputs did not match the ways that decision-makers employed knowledge about organizational redesign. We found that organization designers don’t simply follow a single set of rules to arrive at a single solution. Instead, they compare and tradeoff alternative solutions and eventually arrive at a hybrid combination. They reason much less sequentially and much more flexibly than HITOP-A did, deliberating together with stakeholders and using knowledge in the form of tradeoffs, constraints, and comparative evaluations. In other words, useful systems for designers present knowledge about relationships among different factors, not specific rules and recommendations. This finding has also been noted in observations of research and development engineers (Moran and Carroll, 1996). With such systems, an organizational designer can determine the consequences of one decision on other aspects of his/her organization, as well as recognizing the full spectrum of issues to consider in making a decision. This analysis of how knowledge was actually used in organization design, and the implications for supportive IS that it entailed, led us to our second design principle: systems must help users see and understand relationships among concepts and relationships between concepts and outcomes, not tell them what to do.
04/18/00
18
Through the course of the project, we explored a range of feature-based ways to implement this design principle (e.g., by representing relationships as decision trees, decision flows, goal decompositions, and so forth) before settling on a matrix representation. The matrix representation depicts the specific relationships between each organizational feature and the achievement of business objectives and variance control as a color-coded (red, green, or yellow) matrix cell. The cell indicated whether providing a particular feature in the organization is critical for (green), hurts (red), or is neutral toward (yellow) the achievement of objectives. There were over 28 business objectives, 84 process variance control strategies, and 12 feature sets (with from 5 to 30 individual features each). In total, there were 12 matrices, with over 23,000 relationships. Developing this representation and then determining how to display it in easilyunderstandable ways was a major undertaking. In the end, we realized that users didn’t need access to all possible relationships at all times, and in fact too much relationship information was confusing. But which information to display? We realized first that many items of relationship knowledge were mutually exclusive—they couldn’t apply simultaneously. Moreover, though any of these relationships might be relevant for some organization design effort, only a subset was highly relevant and useful for a given effort. By coding information using the culturally accepted colors of red and green to signify bad and good, we found that users could infer information that was not explicitly displayed. For example, with a little experience, seeing a “hurtful” relationship coded red led users to infer the existence of a corresponding “critical-for” relationship that would have been coded green. This ability to quickly infer information not seen increased the speed and flexibility of using TOP Modeler. The Domain Team much preferred the matrix representation to the “if-then” rules of the earlier design. Color-coding made it easy for users to highlight and visualize areas of concern in a particular organization design. As one user said, “it makes the soft stuff hard.” Most importantly, however, it served as a quick signal to the user of potential concerns to the user, but not a specific resolution to the concerns; the user determined whether and how to respond. Development Process Principle #2: Create drivable prototypes to identify critical relationships When, three months into the project, the Steering Committee decided to drop the rule-based HITOP-A system as a technical and representational foundation, we were quite concerned. It had taken two years of effort to develop the rulebase and countless hours to prepare an expert shell to utilize and display the rules. However, we soon realized that we did not need to eliminate the concepts and relationships represented by
04/18/00
19
the rules. Thus, the matrix became essentially an alternative representation of the original rules. The critical difference was that, in the original system, the user saw the outcomes of rule-based deductions, while, in the new system, the user saw relationships among concepts. This focused the development process on migrating the rule-based knowledge to a relationship-based format. This involved determining which concepts and relationships were critical and how to aggregate across these relationships to generate conclusions about the organization. We had many discussions with the Domain Team, but we found that establishing the criticality of knowledge was best done using “testdrivable” prototypes. For example, early prototypes included knowledge about whether missing skills should be made up through selection or through training. These concepts were eventually dropped because skill replacement was generally so constrained by other factors (e.g., whether the firm is sufficiently profitable to hire; whether the firm provides formal training in these areas) that users just weren’t concerned with a skill replacement strategy. Establishing the important concepts and relationships became a primary focus of our development effort. Emergent Knowledge Process Characteristic #3: Unstructured, Nonoptimal Sequences Our theoretical proposition that emergent knowledge processes involve unstructured sequences creates the requirement that a support system provide nondirective, non-restrictive guidance. In TOP Modeler, this lead to the following design principles: •
The IS product should support the use of intuition about how the knowledge should be analyzed and used, rather than enforce a prescribed sequence of steps
•
The IS development process should focus on capturing users’ intuition points and styles.
Product Principle #3: Support Intuition Our understanding of organizational redesign as a knowledge process led us to expect that there was no optimal sequence of steps; that different workers would follow different sequences. Yet, there was an impression among all parties that insights from the knowledgebase would only be possible with some process guidance, which became known as “methodology guidance”. Only by making the process of organizational design explicit, we thought, could knowledge work proceed efficiently and effectively. Moreover, there was a concern that without methodological guidance, some users would use the knowledgebase too narrowly, attending only to those aspects of the knowledgebase with which they felt comfortable and ignoring the rest. 04/18/00
20
Therefore, the first interface we designed was a screen indicating a dozen different problems that might lead someone to use the system; clicking on one of these problems produced a screen with an interactive roadmap of the steps needed to address the problem. Through testing, we learned that many users did know enough about sociotechnical integration to define their problem accurately. Consequently, this interface did not succeed in providing adequate guidance for naïve users. After much brainstorming and prototyping we settled on a simpler version of the problem map, a two-by-two matrix of high-level organizational improvement tasks common in manufacturing settings: strategic assessment, strategic visioning, organizational diagnosis, and organizational design. Right clicking on a quadrant produced an explanation of the task; left clicking produced an interactive roadmap of the process involved in performing the task with the system. The roadmap specified the tasks to be performed; clicking on the task presented the appropriate part of the knowledgebase, with guidance on how to use the knowledgebase to complete the task. The feedback on this new roadmap interface from naïve users was so positive that we considered the methodology guidance issue resolved. Over time, however, we became increasingly concerned that the guidance we had given users was too restrictive. For instance, the strategic assessment roadmap led users to input their strategic objectives before assessing their current level of socio-technical integration. While logical and compatible with academic prescriptions, this sequence was not compatible with users’ personal process strategies. Some users, for example, apparently felt incapable of changing their organizations’ strategic objectives, so they simply ignored glaring strategic misalignments or questions about strategy and felt frustrated with the system for forcing them to think about something over which they had no control. They wanted a different “entry point” into the system, an entry point that was compatible with what they felt they had responsibility over. Moreover, we found that one of the goals of the roadmaps—to ensure that users gained insight from the analysis—was not achieved, despite users’ enthusiasm for the roadmaps. Some users simply “went through the motions”, showing no intent to explore all critical gaps and act on what they found. Instead, they continued to focus on the same organizational features they had always focused on. Thus, many naïve users thought about socio-technical integration in ways that were contrary to socio-technical integrative thinking. And our cleverly designed “roadmap” did not prevent them from taking analytic shortcuts. Learning that knowledge workers could and did easily circumvent our process guidance led us to reduce our dependence on system guidance as a force for behavioral
04/18/00
21
change. We began to see the process of organization redesign less as an explicit sequence of steps that workers must follow and more as one of gaining insight by intuition. Instead of users completing predefined tasks, organizational redesign involves users intuiting appropriate actions based on data. This led to our third design principle: a support system must convey an intuitive view of how the knowledge should be analyzed and used, rather than enforcing a prescribed sequence of steps. We called this design principle “support intuition.” We operationalized this design principle by replacing the roadmaps with what came to be called “the Ferris wheel”—the first display a user sees when signing on the system. The Ferris Wheel consists of the names of the 12 feature sets, organized with the process variance control strategies and business objectives in the center and the remaining feature sets organized around these in a circle. This presentation was meant to suggest (and was so interpreted intuitively by the users) that the place to begin an analysis of an organization is in the center of this circle; that is, by selecting business objectives, etc. Clicking on a feature set around the outside of the circle yielded the matrix structures, explanations, input screens, and outputted gap analysis. Next to each feature set on the Ferris wheel screen was what became known as “thermometers”, indicating the percentage of features in that set that were matched to best practice, given the business objectives inputted by the user. The thermometers defaulted to 0% (showing all red which users intuitively understood to be “bad”), thus encouraging the user to input the information describing the organization so that the red thermometers could be eliminated. User testing showed us that the Ferris wheel interface effectively communicated to naïve users both what needed to be done and the learning that they should derive from using the system. On seeing the Ferris Wheel screen with all 12 features and thermometers, they immediately recognized that the most obvious place to start was in the center box, which was to articulate their business strategy and then to work through the analysis around the outer circle. Once they confirmed this intuition by clicking on various parts of the screen, they quickly began exploring the system in personally relevant ways. One user, for example, explored the “skills” box on the outside circle. In so doing, he learned that certain fundamental skills would be important for his workforce no matter which of several business objectives his organization was considering pursuing. He concluded that he should start upgrading his workers’ skills immediately, without waiting for the results of the strategic visioning process that his superiors had just initiated. While allowing users to focus, however, having all 12 features and thermometers made it difficult for users to ignore other features, especially when red thermometers indicated significant gaps in these other areas. This created the stimulus 04/18/00
22
for users to make the red thermometers go away - by examining each feature in detail and making decisions about what changes in these other organizational features are needed. This interface, then, allowed users to gain their own insights in their own way; it elicited users’ insights intuitively, rather than forcing users to adopt to explicit formats and task sequences. Development Process Principle #3: Capture users’ intuition points and styles The switch from a sequence-based view of the organizational redesign process to an intuition-based view caused significant changes to our development process. The Development Team had been passively waiting on the Domain Team to come up with a final resolution about what the process guidance would look like and was frustrated at the lack of progress among the users; complaints about “why can’t they get their act together?”, “we shouldn’t let the users whipsaw us like this”, and “users don’t know what they want” were heard frequently by the Development Team during this stage of the development effort. When it was finally realized that a small set of process steps would not be forthcoming and that, instead, a representation that intuitively mirrored how users analyzed redesign knowledge was needed, the Development Team members became more proactive at trying to determine what such a representation should be. The development effort therefore switched from passively waiting on user acceptance to observing how users “intuitively” analyzed knowledge about organizational redesign. By studying and interviewing workers involved in organizational redesign, we determined that their intuitions led them to attend to the bad things first (“cut to the chase: what do I have to fix?”); identify what’s wrong first before worrying about how to fix it (“let me determine what’s wrong first, then I’ll figure out what I want to do about it”); skim across a screen to quickly gain a sense of the “big picture” (“let me see if there is a pattern to what’s wrong”); ignore details (“I’m not interested in details yet”); and presume that what’s in the center of a screen is more important than what’s around the outside. These intuitions were incorporated into a series of prototypes that eventually led to the Ferris Wheel. Emergent Knowledge Process Characteristic #4: Evolutionary Process Our theoretical proposition that emergent knowledge processes have an evolutionary nature creates the requirement that a support system be structured to allow for future unforeseen adaptability. In TOP Modeler, this lead to the following design principles: •
The IS product should accommodate unforeseen adaptations by removing as many constraints of usage as possible
04/18/00
23
•
The IS development process should support dynamism in roles as well as role shifts between developers and users.
Product Principle #4: Accommodate unforeseen adaptations Because organizational redesign involves deliberations and intuition, the exact nature of how the decisions are made, as well as what the decisions will be, is unpredictable and emergent. This suggests that the exact manner in which TOP Modeler would be used could not be known in advance. To encourage broad usage, then, leads to our next design principle: Remove as many constraints on usage as possible. At a minimum, this means that training should be avoided, no particular sequence of steps should be required, and access should be easily available, via the Internet. These features are described in more detail below. While all these specific principles help to encourage the system’s use in an emergent way, none of these principles guarantee that usage will emerge in “intended” or “desired” ways. An excellent illustration of an unintended emergence occurred about one year into the development effort when a manufacturing manager supportive of the TOP Modeler development effort decided to see if the system could in fact be used by anyone. So he contacted one of his supervisors who had no advanced degree or knowledge of the system and socio-technical design and asked her to take a look at TOP Modeler. The system was downloaded to a computer within her work area, with no additional instruction provided. The next day, the supervisor came back to the manager reporting, “I took a look at it; I inputted information about my area; and it confirmed for me what I’ve been telling you for years. It tells me I have to have responsibility over the delivery of materials into my area if I’m going to make good on my goals for reducing throughput time. However, a different organization is responsible for material delivery. So what are we going to do?” The manager was so taken by this supervisor’s rapid use of the system that he first called his member of the Domain Team to confirm the results of the analysis. When the results were confirmed, he called in the manager of the material handling system and told the manager that it was really time to work out a shared responsibility arrangement so that material could get to the production line on time. The material manager was so unhappy with this request that he complained to the executive management team. The executive management team was concerned that an information system was used as a catalyst for an organization design request and that such a system could “instigate” lower-level workers to engage in organizational design, a prerogative of top managers. As a result, the manufacturing manager was forbidden future contact with the
04/18/00
24
development effort, TOP Modeler was pulled from the supervisor’s work area, and only sanctioned “pilots” were allowed to proceed. This story suggests that the design was indeed adaptable; anyone could use it. But it also points out that even adaptable systems have limitations. Development Process Principle #4: Expect/support dynamic role shifts Accepting the design principle that the system needs to be adaptable forces one also to accept a development principle that the development process needs to be adaptable as well. This suggests that the traditional fixed allocation of roles in systems development needs to be replaced with a more emergent allocation of roles. What gets monitored then is not who is performing which roles, but rather whether the different needs of the development process are being met, regardless of whom (e.g., the analyst or user) is meeting the need. Initially, the Domain Team only had the responsibility to review technical documents and non-functional prototypes generated by the development team. Over time, however, the Domain Team took over responsibility of managing the Development Team and interfacing with the Management Steering Committee. For example, about one year into the project, the Domain Team began making formal reports on development progress to the Steering Committee. And, when the system was released for field use, the Domain Team found pilot sites. They also grew more active in management of the software development effort. Since they were making so many demands on the Development Team (who perceived these demands as contradictory), they were eventually asked to help the Developers evaluate tradeoffs and prioritize development requests. Naturally, the evolution of the project plan and the changing role of the user group had an impact—not entirely positive—on the software development staff. Each of the developers had strong views, not always the same, about the appropriate role of “users” in a development effort. (From the perspective of the Development Team, Steering Committee members, user group members, and naïve users were all just “users.”) One developer insisted that users simply did not know what they wanted. Another believed that users did not think through their reactions to the prototypes thoroughly enough to be helpful in the development process. A third found dealing with users to be an interesting challenge. All but one developer frequently expressed concerns about “being jerked around by the user,” and they often ridiculed or resisted implementing the changes the users suggested.
04/18/00
25
To mediate the on-going tensions about development priorities, the project leader (a the socio-technical systems expert) formed a Project Management Team with the senior computer scientist from the Development Team. Their combination of content and technical expertise gave them the credibility to resolve the more difficult conflicts between developers and users. Another helpful coordination mechanism was having individual Development Team members make their own progress reports directly to the user group instead of through the Project Management Team. This allowed the “users” to ask questions and the developers to hear their concerns first hand. Despite these efforts, conflict between “developers” and “users” remained an ongoing issue. To the end of the project, the developers maintained a stable view of their role in the project—to deliver a system that met specifications on time and on budget. But the Project Management Team came to view their role in very different terms. Initially, the Project Managers saw themselves as responsible for project success, as interpreters of the customer’s (the Steering Committee’s) requirements, and as experts in content domain and software development. Then, during development of the new knowledgebase, they became facilitators. The new job was to structure a process that would engage customers, users, software developers, and the community of academic and consulting experts in a process of joint knowledge creation (while simultaneously meeting budget and time constraints). Finally, as naïve users began working with the prototypes and it was possible to assess the changes (or lack thereof) in their knowledge and behavior, the Project Management Team initially tried to be only researchers of a piloting process but eventually became advocates of its use. (See (Markus and Benjamin, 1997) for a more detailed discussion of the facilitator and advocate roles.) Emergent Knowledge Process Characteristic #5: Embedded Action Our theoretical proposition that the knowledge in emergent knowledge processes is essentially embedded action creates the requirement that a support system tie its knowledge to specific behavioral improvements. In TOP Modeler, this lead to the following design principles: •
The IS product should encourage changes in off-line behavior, not just user satisfaction or productivity
•
The IS development process should broaden scope to define project success as offline behavioral change, not simply meeting specifications.
04/18/00
26
Product Principle #5: Encourage changes in off-line behavior At the outset of the project, we defined success primarily in terms of the contractual performance measures specified in the workplan, such as a user-friendly system deployed by such a date at a customer’s site, and a pilot experience indicating that users were satisfied with their use. These success measures focused our early efforts toward responding to users’ generalized reactions to the system (“I’m overwhelmed by so much complexity and freedom”) and to their feedback about screen layouts. Eventually, we came to realize that we were ignoring a key characteristic of emergent knowledge processes: that knowledge is embedded in action. Thus, to ignore the action that results from the use of TOP Modeler ignores the value potential of the tool. This meant, then, that the knowledgebase had to become action-oriented, rather than learning-oriented. It couldn’t just encourage users to learn something; they had to want to act on their results. Moreover, since most organizational redesign decisions are made in group meetings rather than when staring at computer screens, this meant that the system needed to be designed to encourage “off-line” behavioral change among groups. Thus, we needed to be interested, not just in what users did when they were using the system, but also in what was happening off-line after the system was used. For instance, by observing uses of early system prototypes we found that naïve users often only explored aspects of the system related to their own functional domain. In other words, even when the users claimed that they had learned something from using TOP Modeler, we were unable to observe changes in behaviors and decisions that would indicate application of that learning to their business problems. Our dissatisfaction with the failure to change off-line behavior led us to our fifth design principle: that systems for emergent knowledge processes must be designed to encourage off-line behavior not just user satisfaction or productivity. To operationalize this principle, we redesigned the knowledgebase to foster action. For example, terms were defined not using abstract concepts, but rather using action-oriented examples. Organizational features were grouped not by functional departments, which might have encouraged decomposing the analysis along functional lines, but by capabilities (which everyone wants to have a say in). Gaps were highlighted in red, with suggestions on how to close the gaps and increase the count of “greens”. Development Process Principle #5: Define success as off-line behavior change; change scope as needed; use prototypes in context The first change to the development process that this principle brought about was that we changed our success criteria away from contractual obligations toward effecting observable workplace change attributable to use of the system. We stopped asking users
04/18/00
27
“what did you learn when you used the system?” and started observing what they did in their organizations after they used the system. We made this change halfway through the project, over the concerns of the steering committee. In the end, all parties agreed that the change was the right decision, as painful and risky as it appeared at the time. In addition, in order to assess if we were in fact changing off-line behavior, we needed to place increasing emphasis on prototyping in context. Therefore, we conducted a 6-month “beta test” in which users already engaged in organizational redesign efforts were asked to use TOP Modeler and decide for themselves how to incorporate the knowledge into their decision-making process. The beta test involved not just observations of the users during TOP Modeler operation but, more importantly, interviews and observations of decision-makers following TOP Modeler use. Emergent Knowledge Process Characteristic #6: Distributed Knowledge Our theoretical proposition that knowledge in emergent knowledge processes is distributed creates the requirement that a support system provide ways to share multiple incomplete perspectives. In TOP Modeler, this lead to the following design principles: •
The IS product should support the inclusion and synthesis of multiple incomplete representations
•
The IS development process should focus on identifying critical players in the process and articulate the similarities and differences in their perspectives.
Product Principle #6: Support multiple incomplete representations and contexts In an organizational redesign effort, people throughout an organization have relevant knowledge. Shopfloor workers understand how the product really flows through the production line; purchasing departments understand the quality of suppliers; engineers understand the characteristics of the technologies; managers can articulate business strategies; supervisors best understand the skills of the workers. For an accurate assessment of the existing organization to be made and for designs to be tailored to best practice models require that all relevant perspectives be included in the organization design process. This led to our sixth design principle: a support system must offer multiple representations for different parties to foster inclusiveness, not decomposed perspectives. Thus, not only did the system need to allow for multiple perspectives, it also needed to integrate these perspectives in a way that facilitated decision-making. Multiple perspectives were accommodated in the TOP Modeler system in several ways. First, the content of the knowledgebase addressed the multiple perspectives of organizational redesign, ranging from technology design to team design, from skills and
04/18/00
28
training to culture, from performance rewards to information systems design, from production process design to supplier management. Second, the knowledge was represented in multiple ways, conforming to different perspectives. The matrix relational database accommodated those users who thought best in simple relational terms, for example “what are the consequences of this decision on other decisions”? Most users, being organizational design practitioners rather than sociotechnical systems experts, preferred the matrix representation. However, we also provided the detailed design model which explained how the different elements of an organization complement each other; the design model was constructed to encourage users to conduct “what-if” analyses, for example “by deciding not to improve this feature, what effect will that have on the bottom-line?” More sophisticated users preferred the more detailed representation. In addition, we found that different ways to prioritize gaps reflected multiple perspectives. Some users prioritized gaps by focused on the worst cases first. These users especially appreciated the color-coded thermometers. Other users prioritized gaps by focusing on those related the highest priority business objectives. Finally, still other users focused on what they called “show stoppers”, that is, gaps that could not be compensated for by other features of the organization. For example, if the workforce in a unit is missing a necessary skill (such repair), that skill might be provided by making available someone with the skill (a repairman); thus, thus the missing skill does not have to fixed immediately through training. In contrast, failing to measure or reward workers for the right outputs cannot be compensated for in other ways and is thus a “show-stopper”. The design model provided the necessary information to users who prioritized gaps this way. In addition to accommodating different perspectives, we needed to construct a system that could integrate them. The initial designs for TOP Modeler included a separate module for each of the different content perspectives. That is, there was a module for job design, a different module for technology, a different module for performance rewards, etc. Each module computed gaps based on its own set of rules and inference engines. We found, however, that while this architecture succeeded at ensuring that different perspectives were addressed, it did not succeed at integrating those perspectives. Therefore, we decided to ensure that knowledge was represented in sufficiently similar ways across perspectives. Thus, the Ferris Wheel portrays both the percentage of skills not provided as well as the percentage of technology characteristics not provided: both use the same scale of percentage values, although each have very different features and relationships from which the percentages are computed.
04/18/00
29
Development Process Principle #6: Identify critical players and perspectives; articulate similarities and differences In order for the different perspectives to be considered in a knowledge process, the obvious first step is to identify what those different perspectives are. The next, far more difficult, step is to identify the critical similarities and differences among the perspectives. Initially, we thought that the critical differences were in how the knowledge was represented. For example, we agreed with team-oriented job designers that the goal of team-based job designs was job flexibility, not productivity, and we agreed with the technology designers that the goal of technology was more likely to be productivity than job flexibility. Thus, initially, we assumed that the representations had to be different for the different content areas of the knowledgebase. However, after many iterations and discussions, it became clear that there were as many differences of opinions within a content area as between content areas, and that any knowledgebase needed to allow representation of these differences as well. Therefore, paradoxically, the knowledgebase representations gradually became more similar across the content areas. Emergent Knowledge Process Characteristic #7. Evolving Knowledge Our theoretical proposition that the knowledge in emergent knowledge processes evolves over time creates the requirement that a support system allow for major changes in the knowledge structure. In TOP Modeler, this lead to the following design principles: •
The IS product should support evolving knowledge structures through a componentbased architecture
•
The IS development process should generate frequent integration builds and prototypes to allow for evolution over time.
Product Principle #7: Support evolving knowledge structures through componentbased architecture The decision to allow the knowledgebase to evolve throughout the project created the need for a new design principle: a system designed for emergent knowledge processes must have a sufficiently flexible software architecture to incorporate knowledge as it emerges from the ongoing design process. To create that flexibility, the system was separated into as many different components as possible. At the highest level of decomposition were the three components of knowledgebase, user interface, and inference engine. Within each component, there were subcomponents. For example, the knowledgebase was divided into the matrix relationships, the more complex Detailed Design Model, definitions of terms and concepts, the explanations for the matrix relationships, explanations for each node in the Detailed Design Model, and the text for
04/18/00
30
explaining how each screen works. The user interface was separated into different components as well: matrix viewer (used with the relational matrix database representation discussed above), gap viewer, Ferris wheel viewer, roadmaps, and design model viewer. The inference engine was also separated into the components of the design model constraint satisfaction system and the gap aggregator (for computing thermometer values in the Ferris wheel). This level of decomposition allowed the architecture to be “pluggable” such that changes to any single component could be made without affecting other components. While component-based designs are increasingly used today, the concept of applying components to the structure of the knowledgebase itself was untried. First, as indicated above, we separated explanations, the relational matrix database, and the Detailed Design Model as three different components of the knowledgebase. Additionally, by identifying over 300 separate features of an organizational design, we disaggregated the knowledgebase into smaller bits that could be reassembled by the user into customized configurations. (Davenport, et al., 1996) observed this redesign strategy in several of the knowledge work redesign projects they studied. They featured the example of a publishing firm that created idea modules that could be recombined into textbooks, courses, and media. For TOP Modeler, we identified not only that skills or information were needed, for example, but exactly which of 25 skills and18 information resources were needed for each of 21 business objectives and 72 process variance control strategies. The user could then combine these specific features to create an organizational design. This approach is consistent with Litterer and Jelinek’s (1983) observation that organization design involves designers selecting elements from different organizational patterns and combining them into a hybrid plan. Development Process Principle #7: Generate frequent integration builds; share prototypes with customers and users A pluggable architecture was adopted to allow the different modules to evolve independently and iteratively. The components were not identifiable at the outset, however. In fact, the component-based nature of the design only became apparent as the knowledgebase became more regular so that recurring themes in the design process could be separated out. For example, how to view matrices so that users are not overloaded became a recurring theme once it was clear that the knowledge would be represented by a matrix, even though the exact contents of the database supporting the matrix were changing. Although the components were relatively independent, critical assumptions about how the components would work together needed to be worked out. Again, many of
04/18/00
31
these assumptions were not known at the outset; they became clear only as the initial implicit assumptions were violated by experience. For example, the builder of the matrix viewer assumed a certain regularity in the database that was not initially intended but was initially present. As the matrix database development evolved and that regularity was removed, problems with the matrix viewer arose. Much discussion ensued as to whether the matrix viewer needed to accommodate lack of regularity in the matrix database, or whether the database should be made more regular. To surface these issues as soon as possible, we instituted at least weekly integration builds, a practice employed in numerous open-source development projects and later popularized by Microsoft (Cusumano and Selby, 1995; Dibona, et al., 1999; Raymond, 1999). The pluggable architecture adopted to allow flexibility in the development process had spillover benefits for overall project goals. It enabled the system to accommodate alternative knowledgebases addressing different emergent knowledge processes. Two such knowledgebases were developed: one for software development and another for new product development. An additional benefit of the pluggable architecture, unforeseen at the time, was that it enabled the team to suggest and implement an entirely new interface with only a few months left to go in the project. Again, these are illustrations of the emergent nature of the development process. Emergent Knowledge Process Characteristic #8: Unknown Participants Our theoretical proposition that the participants in emergent knowledge processes are unknown in advance creates the requirement that a support system contain terms, operations, and interfaces usable by almost anyone. In TOP Modeler, this lead to the following design principles: •
The IS product should support unknown users
•
The IS development process should progressively add layers to the user team; use drivable prototypes, and generalize the user concept to saturation.
Product Principle #8: Support unknown users A key requirement for an emergent knowledge process support system is that the system be usable by a wide range of different potential users, where the user and their needs are unknown at the time of development. For the organizational design context of TOP Modeler, we uncovered this requirement through a set of interviews in our sponsor organizations to ascertain who the sponsors thought the users would be and what their requirements were. One sponsor (DEC) wanted the system to be used by its salespeople while they helped customers determine which hardware they should purchase. DEC
04/18/00
32
hoped that by having the salespeople model the customer’s organization, the customer would realize, and begin to act on, organizational changes that might be required so that the new hardware would be fully utilized. In this way, customers would be more satisfied with the DEC product. Another sponsor (GM) wanted the system to be used by their industrial engineers who were responsible for process improvements in the plants. Another sponsor (HP) wanted the system to be used by its in-plant socio-technical consultants to help plant redesign efforts. In short, the range of potential users and uses was very which. More importantly, however, the interviews indicated that sponsors’ notions of users and uses were often simplistic and wishful. They did not reflect the complexity and unpredictability of how organizational redesign actually unfolds. The notion that the system would help salespersons to help customers make decisions about organizational redesign assumed that salespersons played the only or most critical role in such decisions. After some questioning, however, it was determined that many people - such as installation personnel, customer training, the integration team, as well as outside consultants and the customer organization itself might also become involved in making design decisions. As another example, the HP sponsor admitted that the in-plant sociotechnical consultant, who was envisioned to use the system, was rarely responsible for organizational redesign; instead, redesign responsibility was in the hands of each plant’s management team, who either formally delegated to design teams or informally ceded it to production technology experts when a new technology was acquired. Thus, the initially envisioned user might play a relatively small role in the redesign process; instead, equally critical workers in the organizational redesign process might be the lowest-level production worker working along side a process engineer installing a new technology. These interviews were pivotal for our design effort. We had come to the development effort expecting to adhere to a principle such as “design for plausible and varied users”. However, since even our sponsors didn’t really know who would be involved in organization redesign, we had to “design for unknown users”. This meant that we needed a system that just about anybody involved in organization design could use, regardless of their role. For example, we could not assume that users knew about sociotechnical systems (as they would at HP) or about basic industrial engineering principles (as they would at GM). Nor could we assume even basic knowledge of what organizational redesign was about. Nor could we presume users with more than an 8th grade education, if the solution was to be usable by shopfloor workers. Therefore, to operationalize the principle of “design for unknown users”, we needed to look for commonalities across the broad spectrum of possible users. We
04/18/00
33
realized that there was much we could presume, regardless of who used it. We could presume that any user of the system would know the manufacturing context and terms unique to that context. Moreover, we could presume that users were motivated by the purpose of the system: manufacturing workers are inherently interested to know how their operations compare to benchmarks and how to excel relative to the competition. Finally, we could presume countless different use scenarios and the need for a system that flexible enough to accommodate them all. Thus, to ‘design for unknown users”, we did three things: 1)
ensured that the knowledgebase was understandable by all potential users (e.g., by using only language that fit industry terminology, rather than language familiar to organizational design experts, and by presenting outputs in an easily-understandable fashion);
2)
ensured that the knowledgebase was likely to be desired by all potential users (e.g., by developing and presenting the knowledgebase as a way for users to objectively benchmark their current operations against other firms; and
3)
ensured that the system did not preclude certain ways of using it (e.g., by allowing users to use the system without following a required sequence).
Development Process Principle #8: Progressively add layers to user team; use prototypes; generalize user concept to saturation Embracing this design principle had a critical implication for our development process as well. The Domain Team’s original responsibility was to represent potential users and review non-functional prototypes 5 (prototypes that illustrated points but didn’t process data) in an iterative development methodology. However, upon embracing this design principle, we realized that the Domain Team did not represent the unknown users. Therefore, the Domain Team’s reactions to non-functional prototypes would not provide us with sufficient input for development; instead, we needed to create working prototypes that other potential users could review. So we began developing what we called “testdrivable prototypes.” These were fielded via the Internet to the Domain Team members, who took them out to potential users and reported on their reactions to the Development Team. This was done on a weekly basis. Similarly, we also found that the Domain Team soon became so knowledgeable about the system that they lost their representativeness as “naïve users”; they had “crossed cultures” to the Development Team. Since they were still an excellent source of ideas and expertise, it was inappropriate to replace them entirely. Instead, we did what we called “onion layering” the user team. As the naïveté of Domain Team members faded, we added a new layer of users. As that second layer of users became too enculturated into the development culture to maintain their naïveté, yet another new layer of naïve users
04/18/00
34
was added. By the end of the project, four layers of “naïve users” had become involved in the project. By never actually replacing the previous users, we were able to maintain their commitment to the project while simultaneously enlarging our user community. Emergent Knowledge Process Characteristic #9: Infrequent Performers Our theoretical proposition that emergent knowledge processes are carried out infrequently creates the requirement that a support system cannot depend on user training. In TOP Modeler, this lead to the following design principles: •
The IS product should be self-explanatory
•
The IS development process should incorporate a user-centered explanation-building process
Design Product Principle #9: Design a self-explanatory system In our early study of potential users of TOP Modeler, we found that three types of knowledge were needed for effective use of the system: 1)
Awareness that socio-technical integration is important and its absence causes organizational problems. (Many organization designers believe that organizational ineffectiveness results from poorly implemented designs rather than from implementing poor designs.)
2)
Knowledge of the principles and practices of sociotechnical systems integration. (Many organization designers have at best an intuitive feel for how to achieve this integration, rather than a well articulated understanding of effective and ineffective designs.)
3)
Knowledge of how to use the system, including the organization of the information in the system.
Designing in the third type of knowledge is challenging, but rather structured. However, the first two types of knowledge are quite difficult to build in. But for emergent knowledge processes, it is necessary to do so, despite the difficulties. Because the processes are emergent, and the users are unknown at the time of design, it is not feasible to assume that users will have relevant prior knowledge nor that they will have been trained before using the system. For TOP Modeler, this meant we could not even assume that a user would believe socio-technical integration to be an important objective—let alone understand fully what it means. There are many in the potential user community who are either unfamiliar with socio-technical integration, have inaccurate preconceived notions of what it is, or fail to understand why it is important for the organization or them personally. For instance, our HP partners used trained, experienced, internal sociotechnical systems consultants for organization redesign. GM, on the other hand used
04/18/00
35
industrial engineers, many of whom incorrectly viewed “socio-technical integration” as a code word for “quality of worklife initiatives” in which workers’ personal needs and wants would be given higher priority than organizational objectives. To manage these challenges, we developed our ninth design principle: that a system for emergent knowledge processes must speak for itself. First, it must provide explanations for everything. In TOP Modeler, we operationalized this principle in several ways. First, we included three types of explanation for each object (screen, data element/term, or result) in system. The explanation types are: •
definitions (what does a term or object signify)
•
functional explanations (how is the object used)
•
contextual/intentional explanations (in what context is the object important, and what aims does it serve?)
For example, TOP Modeler provides information including how the knowledgebase was created, why a particular screen is useful, how to use the information on the screen, what the terms on the screen mean, what is the basis for particular results, and what are the implications of results. Second, TOP Modeler’s explanations had to be credible to its users. For example, we found that we had to give explanations that made “common sense” when explaining relationships between two organizational design elements (e.g., skills and input quality process variances). Supplier management skills can help to improve the quality of the supplier’s products, but the academic descriptions of this relationship left the users cold. We maintained knowledgebase integrity by only including knowledge with academic support. But we found that explanations were far more credible when we kept each relationship explanation grounded in the users’ common sense and left academic information in a separate help facility (called “TOP Modeler Tutorial”). Third, we made all explanations context-specific, using language familiar to our user community and referencing specific applications they knew about. Since writing a very large number of explanations was laborious, at one point we tried to create generic explanations for classes of results so that the same basic explanation could be used for many different specific situations. But users viewed the generic explanations very negatively, and we reverted to context-specific ones. With these explanations, then, the system itself became the awareness and training program on socio-technical integration. In one user’s words, “the [system] is now a teaching tool, not just a decision support system.” We observed an additional benefit
04/18/00
36
from this self-explanatory system design: users employed the built-in explanation capabilities to help them understand the validity of the knowledgebase. Most users were skeptical of the system initially; they were uncertain that a product originating at a university would be useful or even meaningful to them. However, when the system produced plausible, easily understood explanations of its outputs, they remarked about how “down-to-earth” the system seemed, and noted that the explanations fit their understanding of organizational dynamics. Thus, self-explanation allowed us to eliminate training, which is impossible to provide effectively in emergent processes, and it facilitated users’ efforts to validate the system. Development Process Principle #9: Use a user-centered explanation-building process Designing a self-explanatory system increased the emphasis we placed on usercentered design principles. We had drawn up a list of user-centered interface principles at the outset of the project (e.g., “every input should create a response”). But a selfdescribing system poses additional demands on development. For a system to be selfexplanatory, it must have an architecture and knowledge representation that exhibit both regular structure and transparency. “Regular structure” means that knowledge components for similar subdomains are structured similarly and that interfaces for similar functionalities are structured similarly. “Transparency” means that the causes of the system’s behaviors and responses are clear to users. We began the development effort intending to extend HITOP-A, which enjoyed a regular structure but was not transparent. HITOP-A included over a thousand decision rules that were used to generate an organizational design. The designs created by the system had high credibility with users. However, the number of rules invoked in any one run of HITOP-A was quite large and, despite some ability to “backtrace” rules, users had considerable difficulty determining which rules affected various parts of the final design. Along with diminished comprehension and decreased ability to articulate or defend designs they generated, this lack of visible causality severely impeded users’ ability to improve designs by changing inputs. Partially as a result of this, HITOP-A was dropped as a foundation for the TOP Modeler product. Instead, a much more transparent system was constructed by using a constraint-oriented approach. The self-explanatory system principle led us initially to assign the Development Team to create explanations, as they were experts on both the knowledgebase and the software architecture. The Domain Team was initially assigned to review the explanations for usability and effectiveness. As work progressed, however, it became clear that the Development Team could not write in the “common sense” language of the
04/18/00
37
manufacturing-oriented users. We discussed hiring a technical writer but realized that the problem was grounding the explanations in the manufacturing culture of the user community not writing skill in the abstract. We had to find a way to create contextspecific and personalized explanations that read as though they had been written by users themselves. This led us to a user-centered explanation-building process, in which the Domain Team and several successive groups of naïve users wrote explanations iteratively and the Development Team reviewed them for academic accuracy. In this way, the roles of the Domain Team and Development Team reversed. Emergent Knowledge Process Characteristic #10: Discretion Over Work Methods Our theoretical proposition that performers of emergent knowledge processes enjoy wide discretion over work methods creates the requirement that a support system be self-motivating. In TOP Modeler, this lead to the following design principles: •
The IS product should seduce users through customer-engagement and incentive engineering
•
The IS development process should engage users in existing, typical settings and identify customer-oriented user interfaces.
IS Design Product Principle #10: Seduce users through customer engagement and incentive-engineering Knowledge workers are highly autonomous in their choice of tools and techniques, and a supportive IS must sell itself. This insight led us to our final design principle: If a system is to sell itself, it must captivate people into starting to use the system, entice them to do more with it, encourage them to learn more about its underlying knowledgebase, and impel them to apply these learnings to effect real change in their behavior and the organization. We came to think of this principle as “user seduction.” User seduction is a far greater challenge for developers than mere “userfriendliness.” The best that user-friendliness can accomplish is to reduce the occurrence of the barriers software often erects before people who would otherwise be disposed to use it. User-friendliness alone cannot create an incentive to use a tool. We could not afford to rely on making the system merely unforbidding; we had to make it sexy. We had to have people who tried it just once discover that they could actually enjoy learning how to make their organizations better. To make our system seductive, we began to think of its use as a “customer engagement process.” The first phase of this process was having people try the system.
04/18/00
38
The second phase was having them gain some immediate value from trying it. The third phase was enticing them to use the tool long enough to gain the deeper, richer value we had intended. Thus, we were trying to engineer a situation in which users received lastly value from the system when they tried it without expecting much. Designing for the first “try-it-out” phase of the engagement process involved examining the concerns, interests, and unresolved issues that potential users might harbor about their organizations. We discovered that users liked computer games, color-coded ways of evaluating the “soft-side” of technology, benchmarking information about how their organization “measured up” to others, and immediate results from minimal input. We also learned that users were most concerned with “bottom-line” improvements in performance metrics like throughput times, quality, or new product development and ramp-ups. We also learned that users were constantly wary of “missing something”: was there some critical issue that they had not considered? Using these insights, we designed a user interface for results that: •
appeared immediately for each input,
•
incorporated bottom line performance metrics,
•
represented values with color-coding,
•
easily revealed gaps in knowledge.
•
allowed users to “drill down” for additional analysis or explanation.
In the second stage of the customer engagement process we ensured that immediate use resulted in immediate learning. This was done by providing the user with fast feedback about the socio-technical gaps in the user’s organization and by providing accessible explanations and recommendations for action. In the third stage of the customer engagement process we encouraged the user to stay with the system long enough to complete an analysis rather than leaving with only disconnected bits of knowledge. To achieve this, we explored how to manipulate initial data values in the system. When we initialized system values so that the starting organization model was gap-free and consistent, users rarely completed the analysis. When the initial values were set so that the model contained none of the required organizational features and gaps were prevalent, however, the users worked for much longer periods. These efforts paid off. On numerous occasions during user testing, we encountered skeptical or unwilling users who had clearly planned to humor us by politely clicking a few buttons and then begging off. Instead, time and again they explored the system for hours. They acknowledged the intuitive nature and face validity of the system
04/18/00
39
results, and we believed this seductiveness reduced latent fears about the potential for the system to recommend undesirable changes. Some users became comfortable enough with the matrices to begin conducting more sophisticated and input-intensive analyses. IS Development Process Principle #10: Engage users in existing, typical settings’ identify customer-oriented user incentives Embracing a design principle of seducing users into an emergent use process changes the development process from one focused first on development followed by implementation, to one focused exclusively on designing the system as it is being used. This process involves delivering test-drivable prototypes early on, making them broadly available, and observing who uses them. With TOP Modeler, early prototypes were distributed electronically to the sponsoring companies to see which users explored it and what they did with it. From these data, we were able to draw conclusions about how it might be used and how to encourage broader use.
Conclusion Elam and Mean (1987; 1990) have found that decision support systems generally decrease creativity; they conclude that systems that increase creativity are those that are “fun to use”. Similarly, Vandenbosch and Higgins (1995) found that executive support systems were mostly used to maintain existing mental models and to confirm existing views. However, they observed a few cases in which executive support systems contributed to mental model “building” in which executives’ views changed. They concluded that the features and structure of an executive support system (for example, whether it promotes scanning behavior) helps determine the system’s impact. Our own experiences accord with Vandenbosch and Higgins’ about the influence of system features on user behavior, but we believe that these features are substantive rather that merely “fun to use”. Our design principles concern the architecture and the representation of knowledge in a knowledge support system, as well as the process by which such a system is best developed. Although we set out to do so, we ended up not following the IS development process as it is set out in textbooks and training sessions. Instead, we forged a path of “design”, not of “development”, where we use the term design to mean an emergent sense-making where knowledge-building was recursive, participative, and evolutionary (Boland, et al.; Gasson, 1999). Our experience suggests that, where emergent knowledge processes are concerned, IS development should proceed differently than it does for transaction processing systems or for ordinary decision support. Instead of using techniques that encourage convergent thinking, design for emergent knowledge processes
04/18/00
40
needs to foster techniques that encourage alternative ideas. For example, Zmud, et al. (1993) have recommended several techniques for requirements definition that do not force participants to be grounded in an existing as-is process. These include mental imagery, abstractions, and scenarios. We believe that techniques such as these are absolutely critical in the development of systems for emergent knowledge processes. We agree with Couger, et al., (1993) that creativity in IS development process does not eliminate the structure necessary to ensure adequate governance and completion of the project; instead, we believe that creativity can be managed. In our paper, we have presented only a single illustrative case, a case that many readers might find unique or extreme. However, we believe that there are many other examples of what we call emergent knowledge processes—processes that may someday benefit from comprehensive IS support. Examples of other emergent knowledge processes include: •
Basic research
•
Strategic planning and competitive assessment
•
Intellectual asset management
•
New product development
•
Handling “data calls” (non-routine information requests) in staff units.
Interest in and use of knowledge management systems is on the rise (Davenport and Prusak, 1997). Many knowledge management systems foster relatively routine and limited knowledge reuse, for example, quick location of CAD drawings, reuse of consulting boilerplate, data mining to identify high-profitability customers. At the less structured end of the knowledge management spectrum, IT support has often been confined to discussion databases, question-and-answer information retrieval systems, and collaborative technologies that provide only minimal positive support for creativity (Davenport, et al., 1996). Indeed, when the goal is knowledge creation the solution still seems to be co-location of people and better use of communication systems (Davenport, et al., 1996). We argue that IT can offer much greater support for unstructured knowledge processes: it can offer news ways to structure knowledgebases, synthesis mechanisms, and user interfaces that engender creativity. Thus, principles we present here derived from the case of the TOP modeler system are not limited to the emergent knowledge process of organizational design but rather have wide and growing applicability in the domain of knowledge management. Figure 1 summarizes our design theory. The design theory we have proposed here suggests several avenues for future research. First, the theory and principles we have
04/18/00
41
proposed were derived from an iterative prototyping approach to action research in which we tested principles in action and modified our designs based on how users reacted. Thus, our conclusions are based on a series of “learned mistakes”. Clearly, conclusions derived in this fashion should be subjected to systematic verification. As suggested by Walls, et al. (1992), each IS product design principle can be turned into hypotheses and then tested. For example, the design principle “design to create insights intuitively” could be reformulated as the hypothesis: “systems that require explicit representations of knowledge will provide less value in an emergent knowledge process than systems that allow knowledge to remain intuitive.” Experiments can be conducted to test these specific hypotheses. Thus, our design theory offers a rich set of highly testable hypotheses for future research. Insert Figure 1 about here Further, our design theory challenges researchers to test the implications of our design process principles, thereby further improving our nascent theory about how to build systems for emergent knowledge processes. For example, we have suggested that consensus should not be forced during the design process. Certainly, however, some degree of consensus may be required for a successful solution to emerge. Culling out precisely where consensus in IS development is needed (as Fiol, 1994, attempted in a different domain) would be helpful contribution to the IS development literature. Similarly, our theory suggests that developers should avoid requiring users to use the terms, techniques, and structures preferred by developers and domain experts. But, unless this principle is better refined, the obvious (and inefficient!) implication is that developers need to become users and the other way around. What is needed is a common ground (Clark, 1996) between users and developers so that development can proceed efficiently and effectively. And techniques that offer radically different ways to conceptualize the common ground between developers and users (such as Contextual Design, cf. Beyer and Holtzenblatt, 1998) need rigorous testing. It is our hope that the design theory outlined in this pages will provoke new research into the products and processes of IS development and help other developers tackle emergent knowledge processes that have not previously been viewed as amenable to comprehensive IT support.
Notes 1
Majchrzak was principal investigator, project manager, and chief socio-technical systems domain knowledge expert. Gasser was chief developer. Markus was a consultant on user requirements analysis.
04/18/00
42
2
Example process variances include: variations in customer requirements, incoming material quality, and machine setup time. 3 Example manufacturing organization objectives are: maximize quality, minimize throughput time, and maximize product development responsiveness. 4 Predicting organization design success requires lots of inputs, which might discourage casual experimentation and learning. 5 Non-functional prototypes are prototypes that illustrate system features but do not process data. By contrast, our “test-drivable” prototypes actually yielded outputs based on users’ inputs.
04/18/00
43
References Alavi, M., "Systems for Managing Organizational Knowledge," In Framing the Domains of IT Management: Projecting the Future Through the Past, R. W. Zmud (Ed.), Pinnaflex Educational Resources, Inc., Cincinnati, OH, 2000. Alavi, M., and Joachimsthaler, E. A., "Revisiting DSS Implementation Research: A Meta-Analysis of the Literature and Suggestions for Researchers," MIS Quarterly, Number March, 1992, pp. 95-116. Beyer, H., and Holtzenblatt, K., Contextual Design: Defining Customer-Centered Systems, Morgan Kaufmann Publishers, Inc., San Francisco, CA, 1998. Boland, R.; Tensaki, R.; and Te'eni, D., "Designing IT To Support Distributed Cognition," Organization Science, Volume 5, Number 3, 1994, pp. 456-475. Brown, J. S. , and Duguid, P., "Organizing Knowledge," California Management Review, Volume 40, Number 3 (Spring), 1998, pp. 90-111. Cherns, A., "Principles of Sociotechnical Design," Human Relations, Volume 29, 1976, pp. 183-792. Cherns, A., "Principles of Sociotechnical Design Revisited," Human Relations, Volume 40, 1986, pp. 153-162. Cohen, W. M., and Levinthal, D. A., "Absorptive Capacity: A New Perspective on Leaning and Innovation," Administrative Science Quarterly, Volume 35, Number 1, 1990, pp. 128-152. Couger, J. D.; Higgins, L. F.; and McIntyre, S. C., "( Un)structured Creativity in Information Systems Organizations," MIS Quarterly, Volume 17, Number 4 (December), 1993, pp. 375-397. Cusumano, M. A., and Selby, R. W., Microsoft Secrets: How the World's Most Powerful Software Company Creates Technology, Shapes Markets, and Manages People, The Free Press, New York, NY, 1995. Davenport, T. H.; De Long, D. W.; and Beers, M. C., "Successful Knowledge Management Projects," Sloan Management Review, Volume 39, Number 2, 1998, pp. 4357. Davenport, T. H.; Jarvenpaa, S. L.; and Beers, M. C., "Improving Knowledge Work Processes," Sloan Management review, Volume 37, Number 4 (Summer), 1996, pp. 5365.
04/18/00
44
Davenport, T. H., and Prusak, L., Working Knowledge: How Organizations Manage What They Know, Harvard Business School Press, Boston, MA, 1997. Davenport, T. L.; Hammer, M.; and Metsisto, T., "How Executives Can Shape Their Company's Information Systems," Harvard Business Review, Volume 67, Number 2 (March/April), 1989, pp. 130-134. Davis, G., and Olson, M., Management Information Systems, McGraw-Hill, New York, NY, 1985. Dibona, C.; Stone, M.; and (Eds.), S. O., Voice From the Open Source Revolution, O'Reilly and Associates, 1999. Elam, J., and Mean, M., "Designing For Creativity: Considerations for DSS Development," Information and Management, Volume 13, 1987, pp. 215-222. Elam, J., and Mean, M., "Can Software Influence Creativity?," Information Systems Research, Volume 1, Number 1, 1990, pp. 1-22. Fahey, L. , and Prusak, L., "The Eleven Deadliest Sins of Knowledge Management," California Management Review, Volume 40, Number 3 (Spring), 1998, pp. 265-276. Fiol, C. M., "Consensus, Diversity, and Learning in Organizations," Organizational Science, Volume 5, Number 3, 1994, pp. 403-420. Frenkel, S. J.; Korczynski, M.; Shire, K. A.; and Tam, M., On the Front Line: Organization of Work in the Information Economy, Cornell University Press, Ithaca, NY, 1999. Gasson, S., "Organization Information System Design As Social Cognition: An Empirical Study," Unpublished manuscript, Southern Methodist University, available from the author, 1999. Hutchins, E., "The Technology of Team Navigation," In Intellectual Teamwork: Social and Technological Foundations of Cooperative Work, J. Galegher, R. E. Kraut and C. Egido (Ed.), Lawrence Erlbaum Associates, Hillsdale, NJ, 1990, pp. 191-220. Leonard-Barton, D., Wellsprings of Knowledge: Building and Sustaining the Sources of Innovation, Harvard Business School Press, Boston, MA, 1995. Litterer, J. A. , and Jelinek, M., "Design As a Setting For Useful Research," In Producing Useful Knowledge For Organizations, R. H. K. e. al. (Ed.), Jossey-Bass, San Francisco, CA, 1983.
04/18/00
45
Majchrzak, A., The Human Side of Factory Automation: Managerial and Human Resource Strategies for Making Automation Success, Jossey-Bass, San Francisco, CA, 1988. Majchrzak, A., "What To Do When You Don't Have It All: Toward a Theory of Sociotechnical Dependencies," Human Relations, Volume 50, Number 5, 1997, pp. 535565. Majchrzak, A. , and Finley, L., "A Practical Theory and Tool for Specifying Sociotechnical Requirements To Achieve Organizational Effectiveness," In The Symbiosis of Work and Technology, J. Benders, J. d. Haan and D. Bennett (Ed.), Taylor & Francis, London, UK, 1995. Majchrzak, A.; Fleischer, M.; Roitman, D.; and Mokray, J., Reference Manual for Performing the HITOP Analysis, Industrial Technology Institute, Ann Arbor, MI, 1991. Majchrzak, A., and Gasser, L., "HITOP-A: A Tool to Facilitate Interdisciplinary Manufacturing Systems Design," International Journal of Human Factors in Manufacturing, Volume 2, Number 3, 1992, pp. 255-276. Majchrzak, A., and Gasser, L., "Top Modelerã: Supporting Complex Strategic and Operational Decision Making," Information, Knowledge, Systems Management, Volume 2, Number 1 (Spring), 2000. Majchrzak, A., and Meshkati, N., "Aligning Technological and Organizational Change When Implementing New Technology," In The Handbook of Industrial Engineering, G. Salvendy (Ed.), 2000. Markus, M. L., and Benjamin, R. I., "Change Agentry--The Next IS Frontier," MIS Quarterly, Volume 20, Number 4 (December), 1997, pp. 385-407. Markus, M. L., and Keil, M., "If We Build It They Will Come: Designing Systems That Users Want To Use," Sloan Management Review, Number Summer, 1994, pp. 11-25. Moran, T. P., and Carroll, J. M., Design Rationale, Lawrence Erlbaum Associates, Hillsdale, NJ, 1996. Nonaka, I., "The Knowledge-Creating Company," Harvard Business Review, Volume 69, Number 6, 1991, pp. 96-104. Orlikowski, W. J., "Improvising Organizational Transformation Over Time: A Situated Change Perspective," Information Systems Research, Volume 7, Number 1, 1996, pp. 6392. Pava, C. H. P., Managing New Office Technology: An Organizational Strategy, The Free Press, New York, NY, 1983.
04/18/00
46
Raymond, E. S., The Cathedral and the Bazaar: Musings on Linus and Open Source by an Accidental Revolutionary, O'Reilly and Associates, 1999. Satzinger, J. W.; Jackson, R. B.; and Burd, S. D., Systems Analysis and Design in a Changing World, Course Technology, Cambridge, MA, 2000. Silver, M. S., Systems That Support Decision Makers: Description and Analysis, John Wiley & Sons, Chichester, UK, 1991. Spender, J.-C., "Making Knowledge the Basis of a Dynamic Theory of the Firm," Strategic Management Journal, Volume 17, Number Winter, 1996, pp. 45-62. Suchman, L. L., Plans and Situated Actions: The Problem of Human-Machine Interaction, Cambridge University Press, Cambridge, UK, 1987. Taylor, J., and Felten, D., Performance by Design: Sociotechnical Systems in North America, Prentice-Hall, Englewood Cliffs, NJ, 1973. Todd, P., and Benbasat, I., "The Impact of Information Technology on Decision-Making: A Cognitive Perspective," In Framing the Domains of IT Management: Projecting the Future Through the Past, R. W. Zmud (Ed.), Pinnaflex Educational Resources, Inc., Cincinnati, OH, 2000,. Trist, E. L., and Murray, H. M., The Social Engagement of Social Science: A Tavistock Anthology, University of Pennsylvania Press, Philadelphia, PA, 1993. Truex, D. P.; Baskerville, R.; and Klein, H., "Growing Systems in Emergent Organizations," Communications of the ACM, Volume 42, Number 8 (August), 1999, pp. 117-123. Turban, E., Decision Support and Expert Systems, Prentice-Hall, Englewood Cliffs, NJ, 1995. Vandenbosch, B. , and Higgins, C. A., "Executive Support Systems and Learning: A Model and Empirical Test," Journal of Management Information Systems, Volume 12, Number 2, 1995, pp. 99-131. Walls, J. G.; Widemeyer, G. R.; and Sawy, O. A. E., "Building an Information System Design Theory for Vigilent EIS," Information Systems Research, Volume 3, Number 1 (March), 1992, pp. 36-59. Watson, H. J.; Houdeshel, G.; and Rainer, R. K., Buidling Executive Information Systems and Other Decision Support Applications,, John Wiley & Sons, New York, 1997.
04/18/00
47
Weill, P., and Broadbent, M., Leveraging the New Infrastructure: How Market Leaders Capitalize on Information Technology, Harvard Business School Press, Boston, MA, 1998. Wenger, E., Communities of Practice: Learning, Meaning, and Identity, Cambridge University Press, Cambridge, UK, 1998. Zmud, R. W.; Anthony, W. P.; and Stair, R. M., "The Use of Mental Imagery to Facilitate Information Identification in Requirements Analysis," Journal of Management Information Systems, Volume 17, Number 4, 1993, pp. 375-.
04/18/00
48
Figure 1. Design Theory For Systems That Support Emergent Knowledge Processes IS Product Design Principles
Meta-requirements Characteristics of Emergent Knowledge Processes 1. Many scenarios 2. Deliberations & tradeoffs 3. Unstructured, non-optimal sequences 4. Evolutionary process 5. Embedded action 6. Distributed knowledge 7. Evolving knowledge 8. Unknown participants 9. Infrequent performers 10. Discretion over work methods
04/18/00
1. Flexible, configurable, contradictioncapable 2. Knowledge of alternatives & tradeoffs 3. Non-directive, nonrestrictive guidance 4. Future adaptability of content & structure 5. Knowledge tied to specific behavior improvements 6. Multiple incomplete perspectives 7. Allowing major change in knowledge structure 8. Terms, operations, & interfaces usable by known parties 9. Extra-system training not feasible 10. System use selfmotivating
1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
Support contradictions Represent & reveal relationships Support intuition Accommodate unforeseen uses Encourage changes in off-line behavior Support multiple incomplete representations Support evolving knowledge structures through a componentbased architecture Design for unknown users Design a self-explanatory system Seduce users through engagement
IS Development Process Principles 1. Transform “both/and” capabilities 2. Create drivable prototypes to identify critical relationships 3. Capture users’ intuition 4. Expect/support dynamic role shifts 5. Define success as off-line behavior change 6. Identify critical players & perspectives 7. Generate frequent integration builds 8. Add layers to user team progressively 9. Use user-centered explanationbuilding process 10. Engage users in existing, typical settings
Outcome Comprehensive improvement in performance of emergent knowledge process
49
Table 1. A Design Theory For Emergent Knowledge Processes Emergent Knowledge Process Characteristic 1. Many scenarios 2. Deliberations and tradeoffs 3. Unstructured, nonoptimal sequences 4. Evolutionary process 5. Embedded action
6. Distributed knowledge 7. Evolving knowledge
8. Unknown participants 9. Infrequent performers 10. Discretion over work methods
04/18/00
Requirements for a Supportive IS
Design Principles for the IS Principles for IS Development Product Process
Characteristics of the Process of EKPs Flexible, configurable, Support contradictory contradiction-capable requirements and uses Knowledge of alternatives and Represent and reveal relationships; tradeoffs capture and check constraints Non-directive, non-restrictive Support intuition guidance Future adaptability of content Accommodate unforeseen and structure adaptations Characteristics of Knowledge in EKPs Knowledge tied to specific Encourage changes in off-line behavior improvements behavior
Transform to “both-and” capabilities; dialectical IS development Create drivable prototypes to identify critical relationships Capture users’ intuition points and styles Expect/support dynamic role shifts
Success as off-line behavior change; change scope as needed; use prototypes in context Identify critical players and perspectives; articulate similarities and differences Generate frequent integration builds; share prototypes with customers/users
Multiple incomplete perspectives Support multiple incomplete representations and contexts Allowing major change in Support evolving knowledge knowledge structure structures through componentbased architecture Characteristics of People who perform EKPs Terms, operations, and interfaces Support unknown users Progressively add layers to user team; Use usable by unknown parties prototypes; Generalize user concept to saturation Extra-system training not Design a self-explanatory system Use user-centered explanation-building feasible process System use self-motivating Seduce users through customerEngage users in existing, typical settings; engagement and incentiveidentify customer-oriented user incentives engineering
1
04/18/00
2
