Representational Decision Support Systems Success ...

3268

Category: Business Information Systems

Representational Decision Support Systems Success Surrogates Roger McHaney Kansas State University, USA

IntroductIon Rapid and frequent organizational change has been a hallmark of business environments in the past two decades. Frequently, technology and new software development are embraced as aspects of complex strategies and tactical plans. Without sufficient analysis, the unforeseen consequences of change can result in unexpected disruptions and the loss of productivity. In order to better control these contingencies, modern managers often employ a variety of decision support aids. One such aid, classified as a representational decision support system, is discrete event simulation (DES).

characteristics of the system. Table 1 lists primary features of a DES application. While the value of DES in organizational settings has been accepted and is evidenced by the varied and growing market of related products, not every DES application is suited for every problem domain. For this reason, information systems researchers have worked to identify salient characteristics of DES and its usage, and then measure the relationship between its application and successful organizational outcomes. These studies have been conducted in different ways with focuses on independent and dependent variables.

IndEPEndEnt VarIaBLE rESEarcH

Background In its purest form, DES is considered to be a branch of applied mathematics. Its considerable popularity is due in part to the availability of computers and improvements in simulation languages and simulator software packages. DES is often the technique of choice when standard analytical or mathematical methodologies become too difficult to develop. Using a computer to imitate the operations of a real-world process requires a set of assumptions taking the form of logical relationships that are shaped into a model. These models assess the impact of randomly occurring events. Experimental designs are developed and the model manipulated to enable the analyst to understand the dynamics of the system. The model is evaluated numerically over a period of time, and output data are gathered to estimate the true characteristics of the actual system. The collected data are interpreted with statistics allowing formulation of inferences as to the true

Much DES research has focused on identifying and evaluating software and project characteristics (independent variables) and then producing recommendations that suggest either how a project can be successfully implemented or how failure can be avoided. A variety of useful practitioner-focused articles have been published in this area (Hlupic & de Vreede, 2005; Swain, 2003). These studies provide recommendations, based in part on first hand experience, of consultants and simulation analysts. In many cases, these recommendations list the packages currently available and focus on the pros and cons of each. Academic studies of independent variables have also been conducted. In one of the first studies using information systems as a basis for DES, McHaney and Cronan (2000) extended a framework of general decision support system (DSS) success factors identified by Guimaraes, Igbaria, and Lu (1992). The developed contingency model was theoreti-

Table 1. DES features Feature

Description

Statistics Collection

Tools that gather data for purposes of inferential statistics about the model

Resource Modeling

A means for the representation of a constrained resource in the model

Transaction

A means for representation of the participants in the simulation model

Simulation Clock

Tools for analysis and step processing of the coded model

Random Number Generators

A means for producing random number streams for randomization of events within the simulation model

Model Frameworks

Generalized frameworks for the rapid development of a model

Copyright © 2009, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.

Representational Decision Support Systems Success Surrogates

Table 2. List of DES success factors Factor 1: Software Characteristics Factor 2: Operational Cost Characteristics Factor 3: Software Environment Characteristics Factor 4: Simulation Software Output Characteristics Factor 5: Organizational Support Characteristics Factor 6: Initial Investment Cost Characteristics Factor 7: Task Characteristics

cally derived from the simulation literature and empirically tested. The results indicated a seven-factor model that was structured as shown in Table 2. Academic research by Robinson (1999) approached the problem from the opposite perspective and identified sources of simulation inaccuracy that may result in project failure. Other studies, such as one by McHaney and White (1998), developed a DES software selection framework that matched salient DES characteristics with software package features. The importance of evaluation and selection of appropriate DES packages were determined in relation to the success of simulation implementation. As might be expected, the choice of the wrong simulation package often correlated with simulation system failure. This study provided a set of criteria to be systematically considered when evaluating DES software. The taxonomy for simulation evaluation together with importance ratings provided by the collective expertise of a large number of DES users were used as guidelines in deciding the relative weighting to give various software package capabilities. Other research (McHaney, White, & Heilman, 2002) attempted to develop an understanding of DES project success by determining which characteristics of a simulation project were more likely to be present in a successful simulation effort. Potential success factors were derived from the simulation literature and used to develop a questionnaire. Based on the findings, projects perceived as failing were often characterized by high costs, model size constraints, and slow software. Successful projects were characterized by teamwork, cooperation, mentoring, effective communication of outputs, high quality vendor documentation, easily understood software syntax, higher levels of analyst experience, and structured approaches to model development. By understanding the simulation process and characteristics of successful simulations, practitioners will find it easier to avoid common mistakes that can ruin a modeling effort.

dEPEndEnt VarIaBLE rESEarcH The studies mentioned in the previous section focused on understanding independent variables defining successful representational decision support system applications. A

problem with research that considers only the independent side of the equation is the lack of objective measures of whether suggested recommendations correlate with desired outcomes. This dilemma has been problematic throughout information systems research in general and has been the subject of academic debate (Delone & McLean, 1992). DES researchers recognize that the identification of a meaningful, reliable, and robust dependent variable is central to being able to conduct accurate comparisons between competing tools, techniques, software implementations, project approaches, and modeling perspectives. A wide variety of dependent variables have been investigated in the broader fields of information systems (Delone & McLean, 1992) and decision support systems. Among these, DES researchers have focused on success surrogates and investigated the possibility of assessing DES success or failure. Representational DSS researchers have extended the use of these surrogates. The first information system success surrogate validated in the context of DES was the end-user computing satisfaction instrument (Doll, Deng, Raghunathan, Torkzadeh, & Xia, 2004; Doll & Torkzadeh, 1988). The EUCS instrument, shown in Figure 1, is of particular interest because most applications of discrete event computer simulation can be categorized as end-user computing. McHaney and Cronan (1998) collected data from 411 participants using a variety of discrete event computer simulation software packages. The analysis indicated the instrument retained its psychometric properties and provided a valid success surrogate for end users beyond the introductory stages of using representational DSSs. The study established the use of information systems surrogate success measures for applied instruments. The results suggest EUCS can reliably and confidently be used in the investigation of competing tools, features, and technologies in the area of DES. Later, EUCS was the subject of a test-retest reliability study that was distributed to users of DES through a mail survey. One month later, follow-up surveys were administered. The original respondents were asked to again evaluate the same system. The two data sets were compared, and the results supported the instrument’s internal consistency and stability (McHaney, Hightower, & White, 1999). Another success surrogate, the technology acceptance model (TAM), was also investigated within the context of representational decision support systems. TAM was developed by Davis (1989) to provide a theoretical explanation of factors influencing technology usage. Davis’ theory is derived from the theory of reasoned action (TRA) model (Fishbein & Ajzen, 1975). The TRA model explains actions by identifying connections between various psychological constructs such as attitudes, beliefs, intentions, and behaviors. TRA posits that an individual’s attitude toward a given behavior is determined by the belief that this behavior will result in a particular outcome. TAM expands upon this framework 3269

R


Figure 1. EUCS instrument

C1

C2 Content

C3

C4

C1:

C4: A1: A2:

Does the simulation system provide the precise information you need? Does the simulation output information content meet your needs? Does the simulation provide reports that seem to be just about exactly what you need? Does the simulation system provide sufficient information? Is the simulation system accurate? Are you satisfied with the accuracy of the simulation system?

A1

C2: C3:

F1: F2:

Do you think simulation output is presented in a useful format? Is the simulation output information clear?

F2

E1: E2:

Is the simulation system user friendly? Is the simulation system easy to use?

E1

T1: T2:

Do you get the output information you need in time? Does the simulation system provide up-to-date information?

Accuracy

A2

F1 Format

EUCS

Ease of Use

E2

T1

Timeliness

T2

to provide a theoretical explanation of factors influencing technology usage. However, instead of relying on purely attitudinal determinants, TAM hypothesizes that perceived ease of use and perceived usefulness influence a person’s intention, which in turn determines actual technology usage. In essence, Davis’ research identifies the external variables that influence attitude toward voluntary use of technology. TAM has been validated and tested in a variety of technology-related studies (Adams, Nelson, & Todd, 1992; Davis, Bagozzi, & Warshaw, 1989; Mathieson, 1991). Figure 2 illustrates the Davis Instrument and TAM. McHaney and Cronan (2001) focused on external validity aspects of TAM when applied to users of DES. The findings suggested that TAM provided a reasonable surrogate measure for success within this domain. An updated version of this study was developed into a book chapter (McHaney & Cronan, 2002).

3270

FuturE trEndS Representational decision support systems will continue to become more important to managers, particularly as computing power increases and costs decrease. Web-enabled DES, applications embedded in enterprise computing systems, visualization, and use of DES to fine-tune manufacturing operations and supply chain operations on the fly will become more commonplace. Although questions have been raised concerning the direction and relevance of DSS research in general (Arnott & Pervan, 2005), representational decision support system research should grow in importance. Areas requiring additional work include the search for better dependent variables and related surrogates that enable competing tools and techniques to be evaluated and compared. User interface design and development environments will also


Figure 2. Davis instrument

R

U1

E1

U2

E2

U3

U4

E3 Perceived Usefulness

Perceived Ease-of-use E4

U5

E5

U6

E1: E2: E3: E4: E5: E6:

Learning to operate simulation would be easy for me. I would find it easy to get simulation to do what I want it to. My interaction with simulation would be clear and understandable. I would find simulation to be flexible to interact with. It would be easy for me to become skillful at using simulation. I would find simulation easy to use.

be important. Mainstream DSS and information system research will continue to contribute methodologies and techniques that can be adopted for use in this growing field, and representational DSS systems will find wider user bases as the capabilities offered by simulation are integrated with popular spreadsheets and ERP systems software.

concLuSIon Representational decision support systems, particularly as manifested in DES, have become an important tool in a modern manager’s decision-making arsenal. In order to determine the effectiveness of these applications, researchers have investigated characteristics of the technology and determined the suitability of various success surrogates to enable comparisons between competing packages, approaches, and methodologies. Future research in this field may determine whether other surrogates are suitable for use with DES and will provide managers with quantitative comparisons to aid in their software selection processes.

E6

U1: U2: U3: U4: U5: U6:

Using simulation in my job would enable me to accomplish tasks more quickly. Using simulation would improve my job performance. Using simulation in my job would increase my productivity. Using simulation would enhance my effectiveness on the job. Using simulation would make it easier to do my job. I would find simulation useful in my job.

rEFErEncES Adams, D. A., Nelson, R. R., & Todd, P.A. (1992). Perceived usefulness, ease of use, and usage of information technology: A replication. MIS Quarterly, 16(2), 227-247. Arnott, D., & Pervan, G. (2005). A critical analysis of decision support systems research. Journal of Information Technology, 20(2), 67-87. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(4), 319-340. Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982-1003. Delone, W. H., & McLean, E. R. (1992). Information success: The quest for the dependent variable. Information Systems Research, 3(1), 60-95.

3271


Doll, W. J., Deng, X., Raghunathan, T. S., Torkzadeh, G., & Xia, W. (2004). The meaning and measurement of user satisfaction: A multigroup invariance analysis of the end-user computing satisfaction instrument. Journal of Management Information Systems, 21(1), 227-262.

Robinson, S. (1999). Three sources of simulation inaccuracy (and how to overcome them). In P. A. Farrington, H. B. Nembhard, D. T. Sturrock, & G. W. Evans (Eds.), Proceedings of the 1999 Winter Simulation Conference (pp. 701-708). Piscataway, NJ: IEEE.

Doll, W. J., & Torkzadeh, G. (1988). The measurement of end-user computing satisfaction. MIS Quarterly, 12(2), 259-274.

Swain, J. J. (2003). Simulation reloaded: Sixth biennial survey of discrete-event software tools. OR/MS Today, 30(4), 46-57.

Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley.

kEy tErMS

Guimaraes, T., Igbaria, M., & Lu, M. (1992). The determinants of DSS success: An integrated model. Decision Sciences, 23(2), 409-430.

Dependent Variable: A value representing the presumed effect or consequence of various states of related independent variables. In other words, a dependent variable is the condition for which an explanation is sought.

Hlupic, V., & de Vreede, G-J. (2005). Business process modelling using discrete-event simulation: Current opportunities and future challenges. International Journal of Simulation & Process Modelling, 1(1/2), 72-81. Mathieson, K. (1991). Predicting user intentions: Comparing the technology acceptance model with the theory of planned behavior. Information Systems Research, 2(3), 173-191. McHaney, R. W., & Cronan, T. P. (1998). Computer simulation success: On the use of the end-user computing satisfaction instrument. Decision Sciences, 29(2), 525-536. McHaney, R. W., & Cronan, T. P. (2000). Toward an empirical understanding of computer simulation implementation success. Information & Management, 37(3), 135-151. McHaney, R. W., & Cronan, T. P. (2001). A comparison of surrogate success measures in on-going representational decision support systems: An extension to simulation technology. Journal of End User Computing, 13(2), 15-25. McHaney, R. W., & Cronan, T. P. (2002). Success surrogates in representational decision support systems. In M. A. Adams (Ed.), Advanced topics in end user computing 1 (pp. 243262). Hershey, PA: Idea Group Publishing. McHaney, R.W., Hightower, R., & White, D. (1999). EUCS test-retest reliability in representational model decision support systems. Information & Management, 36(2), 109119. McHaney, R. W., & White, D. (1998). Discrete event simulation software selection: An empirical framework. Simulation & Gaming, 29(2), 228-250. McHaney, R. W., White, D., & Heilman, G. (2002). Simulation project success and failure: Survey findings. Simulation & Gaming, 33(1), 49-66.

3272

Discrete Event Simulation (DES): Use of a computer to mimic the behavior of a complicated system and thereby gain insight into the performance of that system under a variety of circumstances. Generally the system under investigation is viewed in terms of instantaneous changes due to certain sudden events or occurrences. End-User Computing Satisfaction: A widely accepted information systems success surrogate that measures the degree to which a technology provides the user with a sense of satisfaction that meaningful usage has been affected. Independent Variable: A value representing a presumed cause of a particular outcome. Representational Decision Support System: Computerbased information system that combines models with data in a fashion that closely resembles the system that is being studied. Computer simulation is a form of representational decision support system. Success Surrogate: A proxy for information systems success that takes the form of measurable values. A success surrogate is a dependent variable. Technology Acceptance Model: A theoretical explanation of factors influencing technology usage that hypothesizes perceived ease of use and perceived usefulness influence a person’s intention, which in turn determine actual technology usage. Theory of Reasoned Action (TRA): This theoretical model explains actions by identifying connections between various psychological constructs such as attitudes, beliefs, intentions, and behaviors, then posits that an individual’s attitude toward a given behavior is determined by the belief that this behavior will result in particular outcome.