The Assessment of Information Systems Success

2011 IEEE Symposium on Computers & Informatics

The Assessment of Information Systems Success from the Organizational Context Perspective Syed Azizi Wafa

Zakariya Belkhamza

School of Business and Economics Universiti Malaysia Sabah Kota Kinabalu, Malaysia [email protected]

School of Business and Economics Universiti Malaysia Sabah Kota Kinabalu, Malaysia [email protected] Abstract—The question of why particular information system encountered unanticipated resistance and never met expectation is often justified by the statement that this information systems did not fit the organizational culture. This paper introduces and validates a four-stage sociotechnical model coined as System Behavioral Success Model (SBS), which explains how the behavioral context of an IT organization led by their IT executives would lead to the success of the IS function. This was conducted by exploring the four dimensions: stretch, trust, support and discipline, as the shapers of the behavioral context of the organization, and relates them to the IS success as represented by system performance, information effectiveness and service performance. Data were collected from 316 MSC status companies. The results show good support for the model. Except trust, behavioral context dimensions are proved to be a major factor for the success of information systems in the organization, especially when the organization acts ambidextrously within the information systems context.

. Keywords—Organizational climate, behavioral context, ambidexterity, IS success, system behavioral success model. I.

INTRODUCTION

This paper proposes and tests a model that attempts to provide a new insight of the linkage between behavioral organizational context and IS success. This is performed by exploring the concept of organizational context dimensions suggested by Ghoshal and Bartlett [10], and relates them to the IS research, focusing on the IS success, particularly investigating the role of the managerial action affecting this linkage. The System Behavioral Success Model (SBS) [3] is a socio-technical model describes a four stage process of how the behavioral context of an IT organization led by their IT executives would lead to the success of the IS function. The SBS model is presented in four stages. The first stage illustrates the managerial action of information system managers and IT executives to build a behavioral context in order to fulfill their managerial roles and processes. This will lead to stage two, where the managerial action produces a climate of discipline, trust and support and stretch. Because of the IS context in the organization possesses specific IT capabilities which is manifested from the adaptability and alignment, an organizational change will emerge as illustrated in stage

978-1-61284-691-0/11/$26.00 ©2011 IEEE

three. The final stage four represents the stage where success of IS function occurs as a result of this process. II.

THEORETICAL DEVELOPMENT

Organizational Context and IS Success To test this model, tow main hypotheses have been developed from an in-depth review of the literature and intensive research investigations. IS success constructs suggested above are measured from the IS users’ perspective that form the corporate culture of the organization. Ghoshal and Bartlett suggest that an organization can create and embed in its context a work ethic that induce rational and yet value-oriented actions on the part of its members [10]. This induces the creation of a favorable, supportive organizational context for improved organizational performance, and thus IS effectiveness. This view manifests in the definition of organizational context by Ghoshal and Bartlett [10], as four behavioral attributes, which are created and reinforced by a variety of actions taken by managers in the organization. These attributes are: stretch, discipline, trust and support. Derived from the System Behavioral Success (SBS) Model, the conceptualized framework explains how the interaction of these four key dimensions will result in an organizational context that leads to an improved organizational IS performance. Thus, it is hypothesized that: H1: The more that an organizational context is characterized by stretch, discipline, support, and trust, the higher the level of IS success as captured by system performance, information effectiveness, and service performance. Ambidexterity and Performance The structural context of these four behavioral attributes refers to the establishment of administrative mechanisms that foster certain behaviors in members of the organization, but its emphasis is on relatively tangible system and process such as managerial action. The argument is that the four behavior- framing attributes of discipline, stretch, support, and trust will create the organization context in which an ambidexterity emerges, a concept broadly referred to the organization’s ability to do two different things at the same time [19], [1]. From this definition, the

792

organization is aligned in order to respond efficiently to the demands of its existing customers, and adaptive in order to meet emerging demands as they arise [11]. It is a balance between a pair of hard elements (discipline and stretch) and a pair of soft elements (support and trust). Thus, when a supportive organization context is created, organization members engage in an ambidexterity represented by both exploitation-oriented actions toward alignment, and exploration oriented actions toward adaptability, which results in contextual ambidexterity, and subsequently enhances organizational performance [11]. From this, it is suggested that ambidexterity mediates the relationship between the four attributes of organization context and subsequent IS success used in the organization. That is, the attributes of organizational context influence IS success through the development of ambidexterity. This is because ambidexterity is seen as a meta-capability which is gradually developed over time through the interaction of the various features of an organization context. Ultimately, the four attributes shape individual and collective behaviors that in turn shape business-unit capacity for ambidexterity, and it is the ambidexterity that leads to superior performance of the IS function within the organization. Therefore, it is hypothesized that: H2: Organizational Ambidexterity mediates the organization context as captured by stretch, discipline, support, and trust, on the IS success as captured by system performance, information effectiveness, and service performance.

III.

MEASURE OF CONSTRUCTS

Sixty one items were used to measure the four organizational context constructs, namely stretch, discipline, trust and support. These items were previously developed and validated [3] before their inclusion to this model. The items used to measure the organizational ambidexterity constructs, namely alignment and adaptability, were adapted from Gibson and Birkinshaw [11]. To measure the IS success constructs, six items used to measures system performance were adapted from Chang and King [4], seven items measuring information effectiveness were adapted from DeLone and McLean [6], and four items measuring service performance were adapted from Chang and King [4]. All items of IS success constructs were measured using the scale: 1= to no extent, 2= to a little extent, 3= to some extent, 4= to a great extent, 5= to very great extent. IV.

POPULATION AND SAMPLING

Multimedia Super Corridor (MSC) status companies were selected for this purpose. The MSC is developed specifically to explore the frontiers of information and multimedia technologies, revealing its full potential through the creation and implementation of cyber laws, cutting-edge technologies and excellent infrastructure.

According to MDC [18], there are over 2173 approved MSC-status companies. Following the stratified proportion technique, the final sample drawn is 618 organizations For data collection, web-based survey application called “Winsurvey”, were used. IT managers, CIOs, executive level IT personnel of the MSC status companies will receive an e-mail containing the invitation letter and the link of the website address where the questionnaire is uploaded. Each link holds the unique ID number of that particular organization to keep track on the respondent. The data of that respondent will be uploaded to a MySQL database created in the server. This data will be retrieved and stored in the local machine as a MySQL database with the software interface. V.

RESULTS

Covariance-based SEM technique called Partial Least Squares (PLS) were used in this study because it is primarily intended for causal-predictive analysis in which the explored problems are complex and theoretical knowledge is scarce, and useful in a theory development situation [21], [5]. Following the PLS technique, the test of the model includes two stages: the measurement model, where the psychometric properties of the measures are evaluated in terms of reliability and validity. The second stage is the structural model, which involves the assessment of the path analysis of the theorized model. A. The Measurement Model Three tests were computed to assess the reliability, Cronbach’s Alpha, composite reliability coefficient and average variance extracted (AVE). Hair et al. [13] suggested that both Cronabch’s Alpha and composite coefficient should be equal of greater than 0.70 to represent good reliability, whereby in exploratory studies, a value of .60 may also be accepted [13]. As for AVE, a level greater than .50 is considered acceptable [5], [4]. Results show that all item loadings between an indicator and its posited underlying construct factor were greater than 0.7. Composite reliability of constructs all exceeded the criterion of 0.7, while the average variances extracted were above the recommended threshold of 0.5, adequately demonstrating convergent validity. B. The Structural Model WarpPLS 1.0 [15] was used for the analysis using a bootstrapping of 500 subsamples. WarpPLS software provides the P value, which is more meaningful than T values for hypothesis testing purposes. This is due to the fact that P values reflect not only the strength of the relationship, which is already provided by the path coefficient itself, but also the power of the test, which increases with sample size [15].

793

Two-stage analysis was conducted to analyze the mediation. In the first stage, a fully mediated model (i.e., no direct paths from discipline, support, trust and stretch to system performance, information effectiveness and service performance) was evaluated. This analysis is also to test the second hypothesis of this research (H2). In the second stage, direct paths from the four constructs, discipline, support trust and stretch, were introduced to examine the direct and mediated effects. This analysis is also used to test the first research hypothesis (H1). To statistically investigate the mediating effect, the two stage analysis mentioned above involves the three steps necessary for the mediation function [2], [11]. The first step is to establish that the independent variables (discipline, support, trust and stretch) significantly influence the mediator (organizational ambidexterity). The second step is to demonstrate that the independent variables (discipline, support, trust and stretch) influence the dependent variables (system performance, information effectiveness and service performance). The last step is to demonstrate that the mediator (organizational ambidexterity) influences the dependent variable. If, in this final step, the effect of organizational context constructs (discipline, support, trust and stretch) on IS success constructs is no longer significant when the mediator is introduced to the model, full mediation is indicated. On the other hand, partial mediation is indicated if there is a substantial decrease for the contribution of the independent variable when the mediator is introduced. For full mediation the contribution has to decrease to a not significant level. VI.

DISCUSSION 1

As the results revealed , discipline has direct effect on system performance, information effectiveness and service performance. This study found that when discipline in the organization is high, the system performance and its effectiveness are high, as well as services of the IS function. This finding is consistent with recent research in the occupational commitment for IT professionals [12]. Support was also found to have a direct effect on system performance, information effectiveness and service performance. The finding of this study confirms the findings of previous studies emphasizing the importance of support as a behavioral capability for IT personnel, which lead to an efficient performance of the IS [7]. The findings of this research suggest the importance of trust in success of the IS in the organization. Some researchers examining the DeLone and McLean success model have also noted the importance of trust in the IS success. Jia et al. [14] even used trust as a scale to measure in the service quality construct. This finding also confirms the positive relationships between the quality of expertise trust and IT system success at a general level. When trust exists, the users are prepared to 1

Tables are not provided here due to space limits. Results are provided upon request.

make efforts to ensure that output of the system and its service are interpreted appropriately [8], [9]. Moreover, when stretch exists within IS users in an organization, the IS will more likely be effective in terms of performance, and effectiveness. This is consistent with the findings of Watts and Henderson [20], that the motivation to achieve and strive toward a common organizational goal will enhance the IT innovation in the organization and translate the mutual vision into a technology vision and positively affect the system performance. The findings concerning ambidexterity are also consistent with previous studies that confirm the importance of ambidexterity to the success of the IS projects [16]. Not only that organizational ambidexterity has been associated with higher levels of IS organizational performance [17], but also successful IT managers tend to employ ambidextrous coping strategies to improve the adverse effects IS effectiveness. Looking at the elements of the organizational ambidexterity, alignment activities tends to improve the IS function in the short term, whereas adaptability activities tends to improve the IS function in the long term. VII. IMPLICATIONS

This paper has provided the answer of two main questions often concerns the IT managers and CIOs. The first question is does the behavioral context of the organization members has an effect on the outcome of the IS used in the organization? And if there is an effect, how can organizations enhance the IS effectiveness generated by the behavioral context of its members? These two questions are so important to the extent that IS effectiveness is consistently reported in the top 20 on the list of most important issues that concern the IS executives and managers. The answer to these two questions by presenting the SBS model clearly brings the explanation. For a better performance of the IS functions, managers and IT executives are requested to offer two main things in the organization: firstly, a purposeful action to harness the behavioral collective context in the IT business units. Secondly, facilitate an IT capabilities and infrastructure in which it provides the organization a capacity of change that blends the behavioral context into agility, which is a key factor of a successful IS projects. VIII.

CONCLUSION

For a better performance of the IS functions and a successful systems project, IS managers and IT executives need to offer two main things in the organization: firstly, a purposeful action to harness the behavioral context in the IT business units. Secondly, facilitation of IT capabilities in order to provides the organization with the necessary a capacity of change that blends the behavioral context into agility, which is a key factor of a successful system projects. By providing these two things, IT executives and CIOs tend to solve the explanation of why particular information system encounter unexplained failure and never met expectation

794

and subsequently, a poor organizational performance. This paper integrated the four key dimensions of Goshal and Bartlett that result in an organizational context, with the IS context in order to modify their structure to fit within the business IS aspect, and conceptualize them in a socio-technical model, which can be tested in further research. The System Behavioral Success Model (SBS) may open the door to more intensive research to fill this gap in this research. ACKNOWLEDGMENT

[8]

[9]

[10]

[11]

The authors would like to thank Professor Ned Kock Nick of Texas A&M International University for the provision of WarpPLS software which was used in this research.

[13]

REFERENCES

[14]

[1]

[2]

[3]

[4] [5] [6] [7]

P. Adler, B. Goldoftas, and D. Levine, “Flexibility versus Efficiency? A Case Study of Model Changeovers in the Toyota Production System,” Organization Science, vol. 10, pp.4368,1999. R. Baron, and D. Kenny, “The Moderator-Mediator Variable Distinction in Social Psychological Research: Conceptual, Strategic, and Statistical Considerations,” Journal of Personality and Social Psychology, vol. 51, pp. 1173-1182, 1986. Z., Belkhamza. “Strategic Linkage between Organizational Context and Information Systems Success” Unpublished PhD Dissertation, School of Business and Economics, Universiti Malaysia Sabah. 2010. J.C.-J. Chang, and W. King, “Measuring the Performance of Information Systems: A Functional Scorecard,” Journal of Management Information Systems, vol. 22, pp. 85–115, 2005 W. W. Chin. “Partial Least Squares is to LISREL as Principle Components Analysis is to Common Factor Analysis,” Technology Studies, vol. 2, pp. 315-319, 1995. W. DeLone, and E. McLean, “The DeLone and McLean Model of Information Systems Success: A Ten-year Update,” Journal of Management Information Systems, vol. 19, pp. 9–30, 2003. L. Fink, and S. Neumann, “Gaining Agility through IT Personnel Capabilities: The Mediating Role of IT Infrastructure Capabilities,” Journal of the Association for Information Systems, vol. 8, pp. 440–462, 2007.

[12]

[15] [16] [17]

[18] [19] [20] [21]

795

D. Gefen, “What Makes an ERP Implementation Relationship Worthwhile: Linking Trust Mechanisms and ERP Usefulness,” Journal of Management Information Systems, vol. 21, pp. 263– 288, 2004. D. Gefen, and C.M. Ridings, “Implementation Team Responsiveness and User Evaluation of Customer Relationship Management: A Quasi-Experimental Design Study of Social Exchange Theory,” Journal of Management Information Systems, vol. 19, pp. 47–69, 2002. S. Ghoshal, and C. Bartlett, “Linking Organizational Context and Managerial Action: The Dimensions of Quality of Management,” Strategic Management Journal, vol.15, pp. 91– 112, 1994. C. Gibson, and J. Birkinshaw, “The Antecedents, Consequences and Mediating Role of Organizational Ambidexterity,” The Academy of Management Journal, vol. 47, pp.209–226, 2004. I. Guzman, R. Sharif, A. Kwiatkowska, and Q. Li, “Occupational Culture and Commitment in the IT Profession: A Multi-Country Perspective,” Proceedings of the 12th Americas Conference on Information Systems, Mexico pp.161-171, 2006. J. E. Hair, R. E. Anderson, R. L. Tatham, and W. C. Black.”Multivariate Data Analysis with Readings” Englewood Cliffs, N.J: Prentice Hall. 2010. R. Jia, B.H. Reich, M. Pearson, “IT Service Climate: An Extension to IT Service Quality Research,” Journal of the Association for Information Systems, vol. 9, pp. 294–320, 2008. N. Kock, WarpPLS 1.0 User Manual, Laredo, Texas.2010. G. Lee, W. DeLone, and A. Espinosa, “Ambidextrous Coping Strategies in Globally Distributed Software Development Projects,” Communications of the ACM, vol.49, pp.40. 2006. H. Ling, F. Zhao, and Y. Wang, “Impact of Synergy between IT and Business Process on Organizational Performance: a Perspective of Ambidexterity Theory,” Proceedings of PACIS 2009. MSC Malaysia Status Companies. MDC. http://www.mscmalaysia.my (2008). Retrieved June 2009 M. Tushman, and C. O’Reilly, “Ambidextrous Organizations: Managing Evolutionary and Revolutionary Change,” California Management Review, vol. 38, pp. 8-30, 1996. S. Watts, and J.C. Henderson, “Innovative IT climates: CIO perspectives,” Journal of Strategic Information Systems, vol.15, pp. 125–151, 2006. H. Wold., “Partial Least Squares,” in Encyclopedia of Statistical Sciences, S. Kotz and N. L. Johnson (eds.) Wiley, New York, pp.581-591, 1985.