Technology Flexibility: Conceptualization, Validation, and Measurement Kay M. Nelson (
[email protected]) and H. James Nelson (
[email protected]) The University of Kansas, Division of Accounting and Information Systems Mehdi Ghods (
[email protected]) The Boeing Company Abstract This research investigates technology flexibility, which is the technology characteristic that allows or enables adjustments and other changes to the business process. Technology flexibility has two dimensions, structural and process flexibility, encompassing both the actual technology application and the people and processes that support it. The flexibility of technology that supports business processes can greatly influence the organization's capacity for change. Existing technology can present opportunities for or barriers to business process flexibility through structural characteristics such as language, platform, and design. Technology can also indirectly affect flexibility through the relationship between the technology maintenance organization and the business process owners, change request processing, and other response characteristics. These indirect effects reflect a more organizational perspective of flexibility. This paper asks the question, "what makes technology flexible?" This question is addressed by developing and validating a measurement model of technology flexibility. Constructs and definitions of technology flexibility are developed by examining the concept of flexibility in other disciplines, and the demands imposed on technology by business processes. The purpose of building a measurement model is to show validity for the constructs of technology flexibility. This paper discusses the theory of technology flexibility, develops constructs and determinants of this phenomenon, and proposes a methodology for the validation and study of the flexibility of emerging technologies.
1. Introduction Organizations are facing an environment of ever increasing turbulence and change. Shorter product cycle times, global competition, an increasing regulatory environment, and constant demands to reduce and control costs require dynamic, flexible business processes. Business process flexibility can be a key factor in an organization's ability to adapt and compete [1], [2], [3]. Increased flexibility can give an organization competitive advantage through faster response to customer needs and environmental conditions [4]. The flexibility of technology that supports business processes can greatly influence the organization's capacity for change [5]. Emerging technologies need to enable changes in the business user
environment [6]. This paper investigates technology flexibility, which is the characteristics of technology that allow or enable adjustments and other changes to the business process. One of the first places that many organizations have tried to gain flexibility through the use of technology is in the manufacturing area [7], [8], [9]. The primary way that this has been accomplished is through investments in computer integrated manufacturing (CIM). The lessons learned from these CIM projects yield insights into the roles and interaction of technology and people in attaining process flexibility. In a recent study of 61 North American paper mills, managers rated 40% of flexibility-improvement efforts to be unsuccessful or disappointing [10]. The primary cause of these disappointments was the reliance on technology alone to provide flexibility. Upton found that in reality, the flexibility of these plants depended much more on the people than the technology. It was the interaction and alignment of technology and people that produced flexibility in successful CIM projects. This example demonstrates that the combination of technology and people is key to achieving flexibility. In this paper, technology is discussed in terms of the technology application itself and the maintainers and users of the application and the management processes they use to support the application. In the past, technology flexibility has been evaluated from a purely computer science perspective [11], [12], only taking the technology itself into consideration. Technology has often made organizations more rigid rather than more flexible by being too time consuming to redesign or reflecting obsolete world views [13]. Existing technology can present opportunities for or barriers to business process flexibility through structural characteristics such as language and design [11]. Technology can also indirectly affect flexibility (either positively or negatively) through the relationship between the technology organization and the business process owners, change request processing, and other response characteristics. These indirect effects reflect a more organizational perspective of flexibility [14], [15]. This paper combines the computer science and organizational perspectives to ask the question, "What makes a technology flexible?" This question is addressed by developing a measurement model of technology flexibility. First we describe the need for flexibility in the business process environment. The literature on software engineering and measurement is then examined for
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
contributions to the measurement of flexibility. Constructs and definitions of technology flexibility are then developed by examining the concept of flexibility in other disciplines. Measures of flexibility adapted from behavioral psychology are used as a foundation for conceptualizing measures of technology flexibility. A methodology for ensuring the validity of these constructs is then discussed. Interviews with I/S managers, system maintainers, users, and business process owners are suggested as a means to test measures drawn from the literature and to construct a measurement model.
2. The business process change environment as a driver of technology flexibility This paper addresses emerging technologies that support business processes. Davenport [16] defines a business process as "a structured set of activities designed to produce a specified output for a particular customer or market" (p.5). Harrington [3] defines a business process as a group of logically related tasks that use the resources of the organization to produce results. These tasks work interdependently to consistently produce a specified result [1]. The business process is defined as: a specific set of interdependent tasks that produce a specified output for a particular customer or market. This section explores the business process environment in terms of incremental and revolutionary change. Business processes are subject to ever increasing environmental pressures to change [3]. Organizations change both continuously and discontinuously, resulting in different types of pressures in the business process environment [17]. Understanding this environment is critical to the study of technology flexibility [18] as it is a determining condition of the nature of change required of a technology that supports a business process. The results of environmental pressures to change have been categorized as long periods of relative organizational stability followed by shorter periods of revolutionary change [19], [20], [21]. Turbulent environments have the potential to both reorganize organizational structure and to add, change, or delete business processes. Once new organizational structures and activities are in place, a cycle of continuous improvement begins. Similarly, the radically changed or replaced technology begins a cycle of continuous improvement to support these new business processes. Technologies that function well in stable periods of incremental change are often unable to survive revolutionary change [5]. Technologies need to be flexible enough to function well in both types of environments. Adaptability and Adaptation. Turbulent changes in the environment can result in the need for rapid, radical changes in ways of doing business [19], [22], [17]. These changes can take a complacent management team by surprise, and force it to engage the unfamiliar. The characteristic of adaptability allows organizations to engage the unfamiliar. Huber [23] defines adaptability as "the capacity to expand niches or to find new niches" (p. 940). Adaptability is often the result of or reaction to turbulence in the environment. One characteristic of adaptability is a willingness to engage an unfamiliar
environment [23]. Adaptable organizations seek opportunities in times of environmental turbulence. They are prepared to take on the unknown and incorporate the unfamiliar. Technology that supports these types of organizations must have this ability built into its structure to support the results of organizational adaptation. The data and functions of the adapted organization must be readily assumed and obsolete ones easily discarded. The capacity to change should be an integral part of the original technology design. Huber [23] describes a second characteristic of adaptability is the ability to scan the external environment for expansion into new niches. Not only is the organization prepared to engage the unfamiliar -- it seeks it. Scanning the environment forces organizations to examine the current business and to discard niches that are no longer profitable or feasible. The business is often redefined. Unfortunately, much organizational technology has been developed or refined expressly for existing niches. A test of technology adaptability is its ability to withstand this external scanning and to accommodate new business niches that are adopted. In the MIS literature, Allen and Boynton [24] describe adaptable systems as dynamically stable information systems. These systems have a stable base of capabilities and at the same time are flexible in the long term. They possess high levels of modularity, applicability, reusability, re-combinability, and are open to links with other systems. The structure of these systems provides the capability to dramatically change, while customized modules can be incrementally improved for increased applicability to a specific business process. Hedberg and Jonsson [24] maintain that information systems should incorporate the capacity to predict the future along with the ability to cope with unexpected developments in the current environment. This implies that both the ability to deal with revolutionary future changes, as well as unanticipated incremental changes in the current environment are critical for technology flexibility. While adaptability characterizes revolutionary changes in the business process environment, adaptation, as defined by Huber [23], is the optimization of a particular niche or business process. Business process improvement (BPI) and total quality management (TQM) are examples of initiatives that emphasize adaptation. These processes stress incremental change and improvement [24]. These requirements call for a definition of technology flexibility that encompasses both adaptability and adaptation. Flexibility, to be employed in a realistic manner, should result in “little penalty in time, effort, cost, or performance" [10 p. 73]. According to De Groote [26], flexibility is defined in terms of yielding the best possible performance in the face of environmental variability. In the case of the technology, the business process environment contains both incremental and revolutionary variability. This research incorporates these ideas in a definition of technology flexibility: The ability to adapt to both incremental and revolutionary change in the business or business process with minimal penalty to current time, effort, cost, or performance.
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
The following section examines the dimensions of technology flexibility. To understand these dimensions, definitions of flexibility from the MIS and other literatures are examined for context and applicability.
3. Dimensions of technology flexibility Definitions of flexibility vary considerably within and between disciplines. The information systems literature has examined flexibility in several contexts. Silver defines flexibility as the opposite of restrictiveness [27], [28], [29]. He identifies factors that impact the design of less restrictive and hence more flexible decision support systems. These factors include providing a broad repertoire of decision support tools, supporting multiple, new, and/or changing decision-making environments, and providing opportunities for creativity and learning. These ideas can be extended to technologies other than decision support systems. Does the existing system have the breadth to support change or growth in the business process environment? Has the ability to adapt to creative changes or new learning been designed into the technology product and process? Differences in flexibility can be due to variations in design and construction of systems or caused by changing hardware or business environments [11]. Although both the work of Silver and Schwan and Jones discusses characteristics of information system flexibility, no formal dimensions of the concept are developed. However, several examples of dimensions of flexibility are found in the information systems, behavioral psychology, manufacturing, and organizational literature. Dimensions of Flexibility. There are three conceptual senses of flexibility in Information Technologies (IT): flexibility in functionality, flexibility in use, and flexibility in modification [30]. Flexibility in functionality and modification address the response of an IT to incremental change or variability. Flexibility in use addresses incremental change, but also addresses the ability to encompass new relationships and opportunities; characteristics of revolutionary change. It implies that this capacity for change is built into the system. These IT dimensions of flexibility are closely linked to environmental pressures on organizations. Flexibility is shown as a multidimensional concept, which provides a starting point for conceptualizing technology flexibility. While the three dimensions proposed by Knoll and Jarvenpaa encompass the ability of technology to adjust to both incremental and dramatic change, these three dimensions all address the technology itself, in this case, the IT application. The people and process side of technology flexibility is not specifically addressed. In behavioral psychology, Scott [31] defines flexibility using a two dimensional approach. The first dimension of flexibility is the degree of response variability found in an organism. Response variability is defined the degree of diversity in reactions shown by a particular person under normal, everyday conditions, which are the type of conditions under which incremental change occurs. The second dimension of flexibility is responsiveness to environmental pressures to change. People are judged on whether they have the capacity to adapt to dramatic
changes in the environment, such as moving to a new home or responding to a new caretaker. This differs from response variability in that it is seen as an internal capacity to accommodate dramatic environmental change rather than a reaction to ongoing, incremental change such as new activities being added to a daily play schedule. This internal ability to accommodate change can be represented by a technology having a design that allows for rapid additions, deletions, or restructuring in periods of dramatic change. The ability to react to ongoing change can be represented in a technology by processes and procedures designed to handle change. In the manufacturing flexibility literature, Garud and Kotha [32] propose that the “time is right” (pg. 673) for a broader perspective of flexibility. They suggest using a systems approach [1] [33] for adopting social and technical dimensions of flexibility. This perspective incorporates both the technical aspects of the manufacturing equipment and facility with human aspects such as job design, management organization, work-team structure, selection and training, and compensation and appraisal. These aspects of flexibility are theorized as interacting in a dynamic network, although the form of these interactions is not specified. The organizational literature also provides dimensions of flexibility [15]. The conceptual framework of strategic organizational flexibility is based on two dimensions; ex ante and ex post. Each of these dimensions has offensive and defensive characteristics associated with it. The ex ante dimension anticipates change before it happens. Agility and versatility are offensive ex ante characteristics that provide a repertoire for dealing with novel or unexpected situations. Robustness and hedging are defensive ex ante characteristics that seek to deflect or avoid the unexpected. These dimensions reflect a built-in capacity to anticipate and deal with change. The ex post dimension of strategic organizational flexibility has both offensive and defensive characteristic that are incremental in nature. Liquidity and elasticity are offensive characteristics that allow continual change with the environment. Corrigibility and resilience are defensive characteristics that aid a system in recuperating or returning to functionality after a change. This dimension of flexibility reflects an ongoing ability to deal with change. This two dimensional definition of flexibility is consistent with the two types of environmental changes to which technologies must adapt. Liquidity and elasticity would allow technologies to incrementally change. Corrigibility and resilience would allow technologies to dramatically change and to recover from these changes. While this two dimensional approach includes both types of environmental change, it does not directly address the components of a technology; the application itself and the people who support it. Huber and McDaniel [14] define organizational flexibility as "the ease with which the organization's structures and processes can be changed" (p. 583). This definition can be applied to technologies. The structure of a technology can be viewed as the design and organization of the programs and data contained in the technology application. The processes of a technology may include the management and technical processes and procedures
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
used to maintain the application. These two dimensions of flexibility can combine to impact the overall flexibility of the technology. The dimensions of structure and process are particularly relevant for describing the flexibility of a technology that includes both the technology itself and the people who support it. The dimensions of structure and process interact in a similar fashion to the technical and social dimensions proposed by Garud and Kotha [32]. For example, a technology application can be written in a way that makes changing a data element a relatively quick and easy task. However, the change request process that initiates this change could be bureaucratic and time consuming, decreasing the overall flexibility of the entire technology. This two dimensional conceptualization of flexibility is appropriate for technology as it captures both the technological and social parts of the system. Each of these individual dimensions, as well as the interaction of the two dimensions of flexibility impact system flexibility. Utilizing Huber and McDaniel’s [14] definition of flexibility, this research defines the dimensions of technology flexibility as structural and process flexibility. Structural flexibility reflects the ability of the design of a technology to be adapted to changes in the business process and is pro-actively designed into the technology. Structural flexibility is the capability of the design and organization of a technology to be successfully adapted to business process changes. The dimension of process flexibility refers to a the ability of a technology's technical and management processes to accommodate change. Process flexibility is the ability of people to make changes to the technology using management processes that support business process changes. The structure of a technology is viewed as it’s design and organization. The processes of a technology include the management and technical processes and procedures used to maintain and change the application. These two dimensions combine to impact the overall flexibility of the technology. We propose that both of these constructs must be present to completely capture technology flexibility.
4. Determinants of structural flexibility What are the characteristics of technology structural flexibility? The dimension of structural flexibility has three determinants that represent the structural characteristics and organization of the technology: modularity, change acceptance, and consistency. These determinants have been derived from determinants of psychological structural flexibility found in the behavioral psychology literature, and chosen for their applicability to technologies. Garud and Kotha [32] use the brain as a metaphor for modeling flexible production systems. Modularity. In behavioral psychology, the number of different arrangements that a subject can perform by moving an object or objects into different forms or patterns reflects cognitive flexibility on the part of the subject [31].
Using this characteristic as guide, we examine the modularity present in a technology. One way to achieve technology flexibility is through modularity [11], [34]. This approach has been used with success by corporations such as Mrs. Field’s Cookies [35]. By structuring technology with smaller modules, changes that involve adding or removing functions is simplified. The technology can potentially support a greater number of arrangements and modifications. The structuredness of a system is often a reflection of the tools or methodologies used to develop it. Modularity is defined as: The degree of formal design separation within a technology application. Modularity can contribute to flexibility by providing manageable units of programs or hardware that can be modified as business processes change as well as the ability to easily create or destroy modules [36]. For example, software modularity is measured by the degree to which programs are grouped to obtain a high degree of functional relatedness [36]. However, it is possible that modularity is not sufficient to encompass changes that require unforeseen functionalities. More advanced types of technologies such as object oriented, neural networks and artificial intelligence have the potential to expand structural flexibility past the modularity of procedural programming [34]. Change Acceptance. The acceptance of pressures to change is another indicator of psychological responsiveness to the environment. In human beings, this is measured by the subject's ability to "cope" with change [31]. Can a person accept change and to what degree (coping) is this change integrated into the person's makeup? The magnitude of aftereffects of change is an indicator of coping ability. Human beings are equipped with various amounts of ability to cope with or accept change. Likewise, technology is written pro-actively with various amounts of built-in restrictiveness or flexibility [29]. This restrictiveness or flexibility may be intentionally or unintentionally a part of the technology design. An example of an intentional design that enhances change acceptance is the inclusion of data control tables that users can access. An example of unintentional restriction of flexibility are older systems which do not accept dates past the year 1999. Change acceptance is defined as: The degree to which a technology contains built-in capacity for change. Features such as reusable code and object libraries should have a positive impact on change acceptance. Consistency. The ability of a person to integrate different psychological regions of thinking when solving puzzles or problems indicates cognitive flexibility [38]. Psychology assumes that different concepts will remain in independent cognitive regions until the biologicalpsychological system integrates them. The greater the integration and number of solutions, the greater the ability to adapt to change. The ability of data and components to be integrated consistently across a technology application can greatly affect flexibility. If data that is stored independently in the application is integrated and the result is inconsistent representations or data elements, the ability to change data becomes very complex. This
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
complexity will impact how much of the overall product needs to be changed or adapted to support business process changes. Data that is maintained in one location and integrated consistently throughout the system can be more readily changed. This is especially significant in the case of reverse engineering of technology [34], where an existing application is dismantled to accommodate major change in structure, language, or platform integration. Technologies that have a high consistency should be more adaptable to changes in the business process environment. Consistency, the third determinant of the technology dimension, is defined as: The degree to which data and components are integrated consistently across a technology. Modularity, Change Acceptance, and Consistency are all determinants of structural flexibility.
5. Determinants of process flexibility The second dimension of technology flexibility is process flexibility. The three determinants of this dimension are rate of response, expertise, and coordination of action. The personnel who maintain and support technology and the management and change processes they use affect technology flexibility. A technology’s structure may be highly flexible, but if the management processes supporting it are rigid, overall flexibility will be impacted. The effectiveness of people in reacting to, implementing, and performing change enhances or impedes technology maintenance [39], [40], [41]. It is therefore proposed that maintainer and user effectiveness in addressing change, and the quality of processes that support change also characterize the overall flexibility of the system. Rate of Response. Rate of reversal [42] is one way of measuring the rate of response to change. In a human being, this refers to both the physical and mental abilities needed to transition from one task or perception to another. The faster the rate of transition, the less rigid and more flexible the individual. Similarly, the speed of transition of a technology contributes to flexibility. This rate of response is impacted by the people and management processes employed in changing technology. It is possible for a change that takes two hours to program to be caught up in a change request process for more than a week. Rate of response is often measured as cycle time in technology maintenance. When this operationalization is used, it is important to consider both the actual time it takes to make the change and the time involved in approving or processing the request for change. The response of the technology group to the users is often impacted by process characteristics such as prioritization, limited resources, or approval processes which are unrelated to the amount of time it takes to actually perform a change. This is the process rate of response that is captured in this research, which includes all of the time it takes to make changes, not just the actual labor hours required. Rate of response is defined as: The degree to which changes can be made to a technology in a timely manner.
Expertise. The expertise determinant of technology flexibility is derived from the psychological concept of facilitation of communication. Facilitation of communication is the ability of a subject to articulate knowledge about an object or circumstance, and to construct inquiries leading to additional knowledge of the area in response to variations in stimuli [42]. In humans, this construct is measured through a range of descriptive behaviors ranging from pre-verbal expression to complex instructions and conversations. The ability to communicate knowledge about a technology and to construct inquiries that lead to new knowledge about the system is a result of the level of expertise in the maintainers and users of the technology. It is also a result of the "expertness" of the documentation and management processes of the system. Accurate version control and complete documentation allow more rapid and accurate response to incremental changes in the business process. The system becomes less dependent on the availability of expert individual maintainers and is less susceptible to personnel turnover. Expertise is defined as: The degree to which up-to-date knowledge about the operation and maintenance of a technology exists and is communicated. Maintainers of technologies are often dependent on the quality of documentation about a technology application, especially if the maintainers are not the developers of the application. Similarly, the maintainers also impact the quality and control of technology documentation by performing or not performing updates to these documents. Well documented technologies are more flexible and easy to change because the development process and logic is more readily traced and can be understood by someone other than the initial developer. In the same respect, accurate version control allows maintainers to trace changes already made in the system and their potential impacts. While it is important that personnel be skilled in and knowledgeable about the technology themselves, it is equally important that they have expertise in maintaining the standards and procedures of the organization and in the business domain they support. This level of expertise is often a function of the training and assessment procedures present in the organization [34]. Coordination of Action. Georgopoulos and Mann [44] define coordination as the extent that interdependent parts of an organization function according to the needs and requirements of the other parts and of the total system. The interdependent organizations that need to coordinate in a technology are the users and developers and maintainers of the application [35]. When these groups coordinate successfully, flexibility can be enhanced. Barriers to flexibility can arise when changes to the technology are not agreed upon or mutually understood [45]. Standards agreed to and used by both groups can enhance coordination of action. Another characteristic of coordination is the ability of the user group to communicate changes in the business process to the information systems developers and maintainers. In this way, coordination can lead to flexible technologies that adapt to the business processes. Coordination of action is defined as:
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
The degree to which the technology maintenance and user organizations operate according to the requirements of each other and the total organization. Rate of Response, Expertise, and Coordination of Action are the determinants of the dimension of process flexibility.
6. Validating the measures of technology flexibility Measurement is often viewed as a process in which numeric values are assigned to objects or occurrences according to a specified set of rules. In this view, it is a strictly empirical process, distinct from the theory being utilized in the study. Bagozzi [47], however, proposes an alternative view of measurement that links it holistically to theory. In this view, measurement is seen as an intellectual and empirical way of giving meaning to theory. Well done measurement validation is a key to performing successful MIS research [48]. Rigorous measurement and validation provide the foundation for evidence of inference [49]. In a review of MIS survey research, Newsted and Munro [50] found that very few studies adequately demonstrated validity and reliability. Carmines and Zeller [51] regard measurement as the linking of abstract concepts to empirical determinants. These constructs are often not directly observable, but are believed to be latent in the phenomenon to be studied [47]. Ives and Olson [52] provide a good example of this, defining system success as a latent construct. Flexibility is another example of a latent construct. These constructs are not directly observable or measurable. To understand latent variables, researchers develop measures such as Ives and Olson's "User Information Satisfaction" scale to capture the construct empirically. Measurement models are used to describe how well observed indicators serve as a measurement instrument for latent variables [53]. Measurement models of data are specified by the researcher based on theory. Confirmatory factor analysis (CFA) is used when the research design hypothesizes dimensions a priori. In CFA, precise descriptions of latent variable structure and indicator loadings are specified by the researcher. CFA requires the use of analysis of covariance structure [54]. This technique is the basis of the LISREL approach to data analysis that we use to test the measurement model of software system flexibility. The testing of the validity of the measures of latent theoretical constructs in a measurement model must be explicitly defined and specified. Bagozzi, Yi, and Phillips [55] provide an explicit criteria for measuring the validity of a measurement scheme as an operationalization of a specified theory. This approach is a holistic one, in that the validity of the measurement instrument and the underlying theory are tested simultaneously. The six components of construct validity as prescribed by Bagozzi [47] are theoretical meaningfulness of concepts, observational meaningfulness of concepts, internal consistency of operationalizations, convergent validity, discriminant validity, and nomological validity.
Theoretical and observational meaningfulness of concepts evaluate the internal consistency of language used to describe a construct and the conceptual relationship between a construct and its operationalization. This evaluation does not include any specific statistical tests, but looks at the semantic unity of the construct and operationalization. "The theoretical meaningfulness of a concept refers to the nature and internal consistency of the language used to represent the concept" [47, p. 117]. To be "meaningful", the terminology used to describe a construct must describe the scope or range of the construct. The observational meaningfulness of concepts refers to the relationship between unobserved latent theoretical variables and their observed indicators. Evaluating the criterion of observational meaningfulness of concepts includes assessing whether the questions used as indicators for each construct are clear, unambiguous, and related to the construct. Theoretical and observational meaningfulness of concepts assess the semantic validity and quality of the relationship between the theoretical constructs and observable indicators used to measure those constructs. The internal consistency of operationalizations assesses the homogeneity of indicators. This homogeneity has two components; unidimensionality and reliability. Assessing unidimensionality tests whether all the indicators measure the theoretical construct of interest [55]. When measuring a single construct, multiple indicators must lie on the same dimension. Any multidimensionality present indicates that more than one construct is present and violates construct validity [47]. Reliability is defined as the extent to which measures are free from random error components and yield consistent results. Reliability assesses the proportion of the indicator variance attributable to the underlying construct. The Cronbach alpha coefficient [56] is often used to compute reliability. This statistic computes reliability across a set of indicators of a single theoretical construct and provides a lower bound of reliability. There are, however, some limitations to this method. Cronbach alpha calculations assume that all indicators are equally important [47], which may not be justified in cases when some indicators measure a construct better than others. Joreskog's [57] Analysis of Covariance Structures overcomes this problem by testing for both unidimensionality and reliability. This method is used for assessing reliability in this study rather than Cronbach alpha, since it does not assume the equality of importance across indicators. Convergent validity is the degree to which two or more measures of the same theoretical construct are in agreement. Discriminant validity is the degree that one theoretical construct differs from another. Joreskog [57] provides a procedure that simultaneously assesses convergent and discriminant validity using confirmatory factor analysis. Using CFA, one tests the hypothesis that each indicator loads only onto its associated theoretical construct by fixing the factor loadings of indicators onto other theoretical constructs to zero. The last type of measurement validity is nomological validity. This refers to the drawing of inferences from constructs from the fit between patterns of data, otherwise known as the nomological net [58].
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
7. Testing the technology flexibility measurement
model
The technology flexibility measurement model was tested using the example of software flexibility. The following sections describe the testing procedure. Discovery Phase. The purpose of the discovery phase is to develop and perform preliminary testing of questions and instruments to be used in the testing of technology flexibility, using the dimensions of structure and process. Questions can be developed in two ways. Candidate questions are derived from the flexibility literature in other disciplines. Operationalizations are then examined for relevance to technology, and modified accordingly. Secondly, a series of interviews is conducted with maintainers and users of the technology to be studied. These interviews are conducted to explore the theoretical and observational meaningfulness for the questions drawn from the literature and to generate additional questions. Appendix A contains an instrument used to measure software flexibility that was generated using the process described herein. Generation of Candidate Questions. We have shown that flexibility has been extensively measured in the behavioral psychology literature. Candidate questions used in the software flexibility measurement instrument were developed based on the work of Scott [31] and questions from the California Psychological Inventory [59]. The manufacturing engineering literature provided another source of candidate questions, generated from the work of Gerwin [7], [8] and Upton [9]. The organizational literature was also a source of potential questions. Structural questions are derived from Blau and Schoenherr's [60] number of major subunits questions, and coordination questions were suggested by Georgopoulos and Mann [44]. Based on these questions, a discovery interview instrument was developed. Sample. Respondents in the discovery phase of this type of research should represent the same organizations as the respondents in the testing phase, but be an independent sample. In the case of measuring software flexibility, 28 discovery interviews were completed with respondent’s answers recorded both in written and taped forms. Interview Methodology. The discovery interviews were conducted using open interview techniques with probes [61]. Respondents were asked what characteristics of software systems most supported business processes. Based on the response to this question, probing questions were asked to elicit further attributes specifically concerning structural and process flexibility. The results of these interviews were compared to the questions derived theoretically and were modified to reflect the data gathered from respondents. The result of the discovery interviews was eight to ten questions for each of the six indicators of software system flexibility (change acceptance, consistency, modularity, rate of response, coordination of action, and expertise). Pretest Sort. A pretest sort of the candidate indicators was conducted with three Ph.D. students, two faculty members, and fifteen managers from six participating organizations. A list of questions was presented to subjects
with a separate list of constructs. Subjects were asked to sort questions by construct. Based on the results of these sorts, questions were reworded or deleted from the candidate list. A pilot questionnaire was then developed from the remaining list of indicators. Pilot Instrument. Two to eight respondents representing both users and maintainers from four of the participating organizations filled out the pilot questionnaire. A total of 23 pilot questionnaires were collected. The pilot questionnaire had two sections: structural flexibility and process flexibility. Each question was rated on a sevenpoint Likert scale. After completing the pilot questionnaire, each respondent reviewed each question with the researcher for content, clarity, and meaning. Through these interchanges, candidate questions were further refined and selected for inclusion in the final research instrument. Based on the results of the pilot test, at least four questions for each characteristic of flexibility were retained for the testing phase of the study. The criteria for keeping questions was clarity, meaningfulness, ability to measure the construct, and understandability. Testing Phase. Once instruments were developed, the measurement model of software system flexibility was tested by collecting data was collected from both users and maintainers of software systems, as well as from I/S managers and the user managers supported by the software. The basic design of this research is a crosssectional field study. The sample is a heterogeneous set of representative business processes and software systems. This sample was drawn from twelve organizations that showed an interest in participating in this study. These organizations were chosen for industry diversity, ease of data collection, and availability of metrics information, making this a convenience sample. Data Collection. The primary research instrument for the collection of software system flexibility data was a questionnaire asking about the determinants of software system flexibility. This questionnaire contains multiple questions for each of the six indicators of software system flexibility, all of which were tested on the pilot instruments. This instrument uses a seven point Likert Scale numbered from one to seven. The instrument was administered to groups representing 116 software systems across 12 organizations. For each represented software system, information was collected from expert respondents: one or two maintainers and one or two users of the system. These expert informants each provided knowledge of the nature and role of the software system in use and the purpose and nature of the business process. Indicators of software system flexibility were tested for validity using respondents from both the I/S and user sides of the system. This paper describes a theoretical approach to measuring technology flexibility, using the above described measurement of software flexibility as an applied example. We believe that this approach can be used to measure the flexibility of a variety of emerging technologies, combining the constructs developed with the validation and construction processes described.
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
8. Conclusion The concept of technology flexibility as having two dimensions has theoretical, methodological, and practical implications. Theoretically, this two dimensional approach gives researchers a new way of approaching the problem of flexible technologies. The computer science and organizational approaches can be combined to yield a more complete view of the phenomenon. The antecedents of both structural and process flexibility can be examined more closely for a further understanding of technology flexibility. The primary methodological contribution of this study is the development of a measurement framework for technology flexibility. This framework gives researchers an empirical starting point for examination of technology flexibility and related constructs. The use of discovery interviews, pre-test sorts and pilot tests can result in a strong, empirically supported measurement model of technology flexibility. The dimensions and indicators of this model can be used as foundations for future researchers. From a practical standpoint, this approach provides technology developers with a new perspective of how to build and maintain more flexible technologies. With an understanding of technology flexibility as a two dimensional phenomenon, developers can design technology to incorporate both structural and process flexibility. This will extend the work being done in the areas of structured programming [37} and object oriented design. This theory suggests that management that desires overall technology flexibility but only considers structural flexibility from technology may experience disappointment. Organizations seeking flexibility through technology need to consider it as a system containing the technology application, the people that maintain and support the application, and the management processes that these people use to accomplish their work. By using the two dimensional approach to technology flexibility, organizations can explore several alternatives. Existing technologies can be examined to see if they can be modified for structural flexibility or if personnel can be given increased training to gain process flexibility. Change request processes and coordination between technology maintainers and users can be examined for potential gains in process flexibility. It may be less expensive to gain flexibility by providing training and doing team building for maintainers and users of an existing system than by purchasing or developing new technology. Both types of technology flexibility can be considered when making both cost and strategic decisions. It is possible that many of the disappointments with emerging technologies may be a result of a failure to consider process flexibility. Upton [10], in the case of computer integrated manufacturing, states that, "Not only is computer integration not the panacea for flexibility problems but it also comforts managers with the thought that they are doing something, when all along they should have been doing something else" (pg. 83). This something else , in the case of emerging technologies, is process flexibility. Emerging technologies must be able to change both incrementally and dramatically to keep up with changes in
the business environment. By taking into account the interaction and alignment of structural and process flexibility, designers, maintainers, and users of emerging technologies can use these technologies to contribute to overall organizational success.
References [1] Thompson, J. D., Organizations in Action, McGraw Hill, Inc., New York, N.Y., 1967 [2] Davenport, T. H. and Short, J. E., "The New Industrial Engineering: Information Technology and Business Process Redesign", Sloan Management Review, Summer 1990, pp. 11-27 [3] Harrington, H. James, Business Process Improvement, McGraw-Hill, Inc., New York, NY, 1991 [4] Fiegenbaum, A. and Karnani, A., "Output Flexibility A Compettitve Advantage for Small Firms", Strategic Management Journal, Vol. 12, No. 2, February 1991, pp. 101-115 [5] Hammer, M. and Champy, J., Reengineering the Corporation: A Manifesto for Business Revolution, Harper Collins Publishing, May 1993 [6] Ahituv, N. and Neumann, S., The Principles of Information Systems Management, Third Edition, William C. Brown Publishers,Dubuque, Iowa,1990 [7] Gerwin, D., "An Agenda for Research on the Flexibility of Manufacturing Processes," International Journal of Operations and Production Management, Vol. 7, No. 1, 1987 [8] Gerwin, D., "Manufacturing Flexibility: A Strategic Perspective", Management Science, Vol. 39, No. 4, April 1993, pp. 395-410 [9] Upton, D. M., "The Management of Manufacturing Flexibility", California Management Review, Vol. 36,, No. 2, Winter 1994, pp. 72-89 [10] Upton, D. M., "What Makes Factories Really Flexible?", Harvard Business Review, Vol. 73, No. 4, JulyAugust 1995, pp. 74-86 [11] Schwan, K. and Jones, A. K., "Flexible Software Development for Multiple Computer Systems," IEEE Transactions on Software Engineering, Vol SE-12, No. 3, March 1986, pp.385-401. [12] Gibson, V. R. and Senn, J. A., "System Structure and Software Maintenance Performance", Communications of the ACM, Vol. 32, No. 3, March 1989, pp.347-358 [13] Hedberg, B., and J., Sten, "Designing Semiconfusing Information Systems for Changing Organizations," Data Base, Winter-Spring 1982, pp. 12-25.
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
[14] Huber, G. P. and McDaniel, R. R., "The DecisionMaking Paradigm of Organizational Design", Management Science, Vol. 32, No. 5, May 1986, pp. 572-589. [15] Evans, J. S., "Strategic Flexibility for High Technology Manoeuvres: A Conceptual Framework", Journal of Management Studies, Vol. 28, No. 1, January 1991, pp. 69-89 [16] Davenport, T. H., Process Innovation: Reengineering Work through Information Technology, Harvard Business School Press, Boston MA, 1992 [17] Meyer, A. D., Goes, J. B., and Brooks, G. R., "Organizations Reacting to Hyperturbulence", pages 66111, in Huber, George P. and Glick, William H., Organizational Change and Redesign, Oxford University Press, New York, 1993 [18] Emery, F.E. and Trist E.L., "The Causal Texture of Organizational Environments", Human Relations, Vol. 18, 1965, Pages 21-32 [19] Tushman, M. and Romanelli, E., "Organizational Evolution: A Metamorphosis Model of Convergence and Reorientation", in Cummings, L.L. and Staw, B.M. (Eds.), Research in Organizational Behavior, vol. 7, 1985, JAI Press, Greenwich Conn., pp. 171-222 [20] Gersick, C. J. G., "Revolutionary Change Theories: A Multilevel Exploration of the Punctuated Equilibrium Paradigm", Academy of Management Review, vol. 16, no. 1, 1991, pp. 10-36 [21] Romanelli, E. and Tushman, M. L., “Organizational Transformation as Punctuated Equilibrium: An Empirical Test,” The Academy of Management Journal, Vol. 37, No. 5, October 1994, pp. 1141-1166 [22] Miles, R. E. and Snow, C. C., "Organizations: New Concepts for New Forms," California Management Review, Vol. 18, No. 3, Spring 1986, pp. 62-73 [23] Huber, G.P.,, "The Nature and Design of PostIndustrial Organizations", Management Science, vol. 30, no. 8, 1984, pp. 928-951. [24] Allen, B. R. and Boynton, A. C., "Information Architecture: In Search of Efficient Flexibility", MIS Quarterly, Vol. 15, No. 4, December 1991, pp. 435-446 [25] Robson, G. D., Continuous Process Improvement, The Free Press, New York, NY., 1991 [26] De Groote, X., “The Flexibility of Production Processes”, Management Science, Vol. 40, No. 7, July 1994, pp. 933-945 [27] Silver, M. S. (1), "On the Restrictiveness of Decision Support Systems", Organizational Decision Support Systems, 1988, pp. 259-270
[28] Silver, M. S., "User Perceptions of Decision Support System Restrictiveness: An Experiment", Journal of Management Information Systems, Vol. 5, No. 1, Summer 1988, pp.51-65 [29]Silver, M. S., "Decisional Guidance for ComputerBased Decision Support", MIS Quarterly, March 1991, pp.105-122 [30] Knoll, K. and Jarvenpaa, S. L., "Information Technology Alignment of "Fit" in Highly Turbulent Environments: The Concept of Flexibility", Proceedings of the 1994 SIGCPR Conference, Alexandria, VA, March 2426, 1994, pp. 1-14 [31] Scott, W. A., "Flexibility, Rigidity, and Adaptation: Toward Clarification of Concepts" in Harvey, O.J. (ed.), Expeience, Structure and Adaptability, Springer Publishing Company, Inc., New York, 1966. [32] Garud, R. and Kotha, S., “Using the Brain as a Metaphor to Model Flexible Production Systems,” The Academy ofManagement Review, Vol. 19, No. 4, October 1994, pp. 671-698 [33] Van de Ven, A. H. and Drazin, R., "The Concept of Fit in Contingency Theory", Research in Organizational Behavior, Vol. 7, 1985, pp.333-365 [34] Premelani, W. J. and Blaha, M. R., “An Approach for Reverse Engineering of Relational Databases”, Communications of the ACM, Vol. 37, No. 5, May 1994, pp. 42-53 [35] Haeckel, S. H. and Nolan, R. L., “Managing by Wire,” Harvard Business Review, Vol. 71, No. 5, September-October 1993, pp. 122-133 [36] Yourdon, E., Managing the Structured Techniques: Strategies for Software Development in the 1990's, Yourdan Press/Prentice Hall, New York, 1986 [37] Sommerville, I., Software Engineering, AddisonWesley Publishing Company, Wokingham, England, 1989 [38] Kounin, J. S., "Experimental Studies of Rigidity", Character and Personality, Vol. 9, 1941, pp. 251-282 [39] Banker, R. D., Datar, S., and Kemerer, C., "A Model to Evaluate Variables Impacting the Productivity of Software Maintenance Projects", Management Science, Vol. 37, No. 1, January 1991. [40] Banker, R. D., Datar, S., and Kemerer, C., "Factors Affecting Software Maintenance Productivity: An Exploratory Study", Proceedings of the Eighth ICIS, Pittsburgh, PA, December 6-9, 1987, pp. 160-175. [41] Swanson, E. B. and Beath, C. M., "Departmentalization in Software Development and Maintenance", Communications of the ACM, Vol. 33, No. 6, June 1990, pp. 658-667
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
[42] Wertheimer, M. and Wertheimer, N., "A metabolic interpretation of Individual Differences in Figural Aftereffects", Psychological Review, Vol. 61, 1954, pp. 279-280 [43] Coulter, W. A. and Morrow, H. W., Adaptive Behavior: Concepts and Measurement, Grune and Stratton, New York, 1978 [44] Georgopoulos, B. S. and Mann, F. C., The Community General Hospital, Macmillian, New York, 1962 [45] Cooprider, J. G. and Victor, K. M., "The Contributions of Shared Knowledge to I/S Group Performance,” Proceedings of the Fourteenth ICIS, Orlando, FL, December 15-18, 1993 [46] Bagozzi, R. P., Causal Modelling in Marketing, Wiley & Sons, New York, NY, 1980. [47] Lucas, H. C. Jr., "Methodological Issues in Information Systems Survey Research", in The Information Systems Research Challenge: Survey Research Methods, K. L. Kraemer, (ed.), Harvard Business School Research Colloquium, Boston, MA, 1991 [48] Straub, D. W., "Validiating Instruments in MIS Research," MIS Quarterly, Vol. 13, No. 2, June 1989, pp. 147-165 [49] Newsted, P. R., and Munro, M. C., "Data Acquisition Instruments in Management Information Systems Research", in The Information Systems Research Challenge: Survey Research Methods, K. L. Kraemer, (ed.), Harvard Business School Research Colloquium, Boston, MA, 1991 [50] Carmines, E. G. and Zeller, R. A., Reliability and Validity Assessment, Sage University Press, Beverly Hills, CA, 1979 [51] Bagozzi, R. P., "The Role of Measurement in Theory Construction and Hypothesis Testing: Toward a Holistic Model," in Conceptual and Theoretical Developments in Marketing, American Marketing Association, Chicago, IL, 1979 [52] Ives, B., and Olson, M., "User Involvement and MIS Success: A Review of Research," Management Science, Vol. 30, No. 5, May 1984, pp. 586-603. [53] Joreskog, K. and Sorblom, D., LISREL VI: Analysis of Linear Structure Relationships by Method of Maximum Likelihood, Scientific Software, Mooresville, IN, 1984 [54] Byrne, B. M., A Primer of LISREL: Basic Applications and Programming for Confirmatory Factor Models, Springer-Verlag, New York, 1989 [55] Bagozzi, R. P., Yi, Y. and Phillips, L.W., "Assessing Construct Validity in Organizational Research,"
Administrative Science Quarterly, Vol. 36, 1991, pp. 421458. [56] Cronbach, L. J., "Coefficient Alpha and the Internal Structure of Tests," Psychometrika, Vol. 16, September 1951, pp. 297-334 [57] Joreskog, K., "Analyzing Psychological Data by Structural Analysis of Covariance Matrices," in Contemporary Developments in Mathematical Psychology, Vol. 11, R.C. Atkinson et al., eds., San Francisco, CA 1974 pp. 1-56 [58] Cook, T. D., and Campbell, D. T., QuasiExperimentation: Design & Analysis Issues for Field Settings, Houghton Mifflin Company , Boston, MA, 1979 [59] Megargee, E. I., The California Psychological Inventory Handbook, Jossey-Bass Inc., San Francisco, 1972 [60] Blau, P. M., and Schoenherr, R. A., The Structure of Organizations, New York, Basic Books, 1971 [61] Rossi, P. H., Wright, J. D., and Anderson, A. B., Handbook of Research Methods, Academic Press Inc., Orlando FL, 1983
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
