Exploring User Acceptance of an Electronic Performance Support System Evren Sumuer, PhD, and Soner Yildirim, PhD
W
ell-designed training programs are not This mixed methods research necessarily successful in dealing with study aimed to explain and understand user acceptance of an electronic performance problems in workplaces performance support system (EPSS) due to problems related to forgetting, nontransfer- designed for the Crime Scene Investigaability of learning, and rapid changes in employee tion and Identification Units of the Turkish National Police. Quantitative data needs (Baldwin & Ford, 1988; Hodges, 2002; McKay were collected from 209 police officers & Wager, 2007; Puterbaugh, Rosenberg, & Sofman, through a questionnaire to test relation1989). With the increase in availability and capabili- ships hypothesized in the technology acceptance model. At the same time, ties of computers in workplaces, electronic perfor- qualitative data were collected through mance support systems (EPSSs) are considered to interviews with 15 police officers to be a viable means in addressing several performance acquire an in-depth understanding of perceived usefulness and perceived problems and opportunities in workplaces. These ease of use of the system. This study systems involve a set of computer-based components showed that perceived usefulness, per(e.g., performance support, reference, instruction, ceived ease of use, and attitude toward using were significant determinants of and collaboration tools) that enable employees to the acceptance of the EPSS. The findings perform job-related tasks effectively and efficiently also presented several user personal, (Gery, 1991, 2002; McKay & Wager, 2007). They system, and organizational characteristics that were considered as influencing deliver “just the help a performer needs to do a job, factors for usefulness and ease of use of just when the performer needs it, and in just the the EPSS. form in which he or she needs it” (Carr, 1992, p. 32). These systems may also contain infrastructures that capture, store, and distribute knowledge throughout an organization, which enables learning (Laffey, 1995; Raybould, 1995). EPSSs essentially include any combination of task structuring, knowledge, data, tools, and communication components to perform four supportive functions: learning, doing, referencing, and collaboration (Gery, 2002). Based on the extent to which performance support systems are integrated into users’ work interface or process, there are three fundamental types of performance support systems: intrinsic, extrinsic, and external (Gery, 1995). Intrinsic support is directly integrated with work interface and process, and so users get support while they are performing their tasks without having awareness of using EPSS components. Extrinsic 29 P E R F O R M A N C E I M P R O V E M E N T Q U A R T E R L Y , 2 7 ( 4 ) P P. 2 9 – 4 8 © 2015 International Society for Performance Improvement Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/piq.21178
support is integrated with tasks at hand, but not inherent to primary work interface or actual work flow. Based on tasks and employee situations, this type of support system provides relevant tools and resources that can be invoked by employees or the system. External support is not integrated with either the system or primary workspace. In this respect, employees are forced to leave tasks in order to locate appropriate tools or resources external to the EPSS. According to Gery (1995), an EPSS should be designed to involve almost 80% of performance support as intrinsic, and the remaining part as extrinsic and external with almost equal percentage. EPSSs provide a number of benefits for User acceptance can be employees and organizations. According to Nguyen (2010, 2012), potential advantages of regarded as a pivotal factor using an EPSS include improved performance, for the success of an EPSS in improved attitudes, reduced costs, memory enhancing performance in an support, updated information, and access to a organization. wider range of support content. The design and development process influences the success of an EPSS in an organization. To have considerable benefits for an organization, an EPSS should have a structure and components that are designed and developed in an appropriate way (Milhelm, 1997). A successful EPSS design requires identification of performance problems and the development of appropriate support systems that improve job performance (Barker & Banerji, 1995). Regardless of how well it is designed and developed, it is unlikely for an EPSS to realize intended performance goals and benefits when it is not used at an expected level in an organization (Carliner, 2002; van Schaik, 2010). In general, designers are likely to focus almost entirely on attributes of performance improvement interventions and ignore the human aspects (Spitzer, 1999). Nevertheless, it is important to consider what people bring to the situation. When people reject using a performance improvement intervention, it has no value for the organization (Spitzer, 1999; Stolovitch & Keeps, 2004). Likewise, an EPSS is likely to fail to support job performance when people do not accept its usage on the job (Carliner, 2002; van Schaik, 2010). Therefore, user acceptance can be regarded as a pivotal factor for the success of an EPSS in enhancing performance in an organization. Similar to other information systems, a well-designed EPSS is not readily accepted. Individuals may resist using performance support systems because these systems present considerable changes in the way that they think about technology, work, and training (Rossett, 1996; Milhelm, 1997; Stone & Villachica, 2003; McKay & Wager, 2007). Drawing on the literature, a number of factors have been suggested as playing a role in acceptance of an EPSS (Carliner, 2002; Gery, 1991; Laffey, 1995; Mao, 2004; Maughan, 2005; McKay & Wager, 2007; Nguyen, 2010, 2012; Rossett, 1996; Stevens & Stevens, 1995, Surry & Ely, 2007). These factors can be gathered into the following categories: 30
DOI: 10.1002/piq
Performance Improvement Quarterly
1. User personal characteristics (e.g., attitude toward the work, resistance to innovation, feelings, misconceptions, and preparedness) 2. System characteristics (e.g., usefulness, compatibility, information technology tools, usability, cost, maintenance, legal requirements, and presentation) 3. Organizational characteristics (e.g., organizational structure, communication, training, support, administrative procedures and processes, implementation planning, involving organizational units, incentives, and communication and computer infrastructure) 4. Others (e.g., marketing of the system and expected knowledge outcome) The efforts in explaining and understanding factors influencing the acceptance of a system help designers to identify appropriate interventions or strategies that lead to greater system acceptance and use (Davis, 1993; Venkatesh & Bala, 2008). Therefore, to minimize or overcome user resistance to an EPSS, it is essential to identify and understand factors that influence its acceptance. As advocated by Barker (2010), the technology acceptance model (TAM) (Davis, Bagozzi, & Warshaw, 1989) is a highly validated model that leads to understanding the acceptance of an EPSS. The purpose of the study was to explain and understand user acceptance of an EPSS on the basis of TAM. The study focused on acceptance of an EPSS designed for tasks performed by police officers in the Crime Scene Investigation and Identification Unit of the Turkish National Police. The findings of the study helped to grasp the nature of user acceptance of an EPSS in an organization by using both quantitative and qualitative research methods and also provided a basis for an extended TAM. This study presented valuable information on where designers should invest their efforts to enhance user acceptance and usage. In addition, it laid a groundwork for further studies focusing on the adoption of an EPSS in different settings.
Theoretical Framework TAM (Davis, 1985; Davis et al., 1989) is one of the most leading and successful models that predict and explain user acceptance of an information technology. TAM offers causal relationships between perceived usefulness, perceived ease of use, attitude, intention to use, and actual system usage (see Figure 1) (Davis, 1993; Davis et al., 1989). The relationships in TAM demonstrate a substantial variability across settings (Lee, Kozar, & Larsen, 2003; Legris, Ingham, & Collerette, 2003; Yousafzai, Foxall, & Pallister, 2007). Understanding what influences perceived usefulness and perceived ease of use helps designers formulate interventions or strategies that result in greater acceptance and more effective utilization of an information technology (Davis, 1993; Davis et al., 1989; Venkatesh & Bala, 2008). Yousafzai et al. (2007) claim that “without a better understanding of the Volume 27, Number 4 / 2015
DOI: 10.1002/piq
31
Perceived Usefulness (PU) Attitude toward Using (A)
External Variables
Behavioral Intention to Use (IU)
Actual System Use
Perceived Ease of Use (PEU)
Source. Davis, Bagozzi, and Warshaw, 1989. Reprinted with permission. FIGURE 1.
THE TECHNOLOGY ACCEPTANCE MODEL
antecedents of PU [perceived usefulness] and PEU [perceived ease of use], practitioners are unable to know which levers to pull in order to affect these beliefs and, through them, the use of technology” (p. 268). Several research studies proposed a number of external variables that play determinant roles on perceived usefulness and perceived ease of use of information systems. Based on the synthesis of studies on TAM, Venkatesh and Bala (2008) proposed four different types of determinants of perceived usefulness and perceived ease of use: individual differences, system characteristics, social influence, and facilitating conditions. In addition, Yousafzai et al. (2007) offered four distinct categories for more than 70 external variables influencing perceived usefulness or perceived ease of use: organizational characteristics, system characteristics, user personal characteristics, and other variables. Although some studies have focused on context-dependent determinants of perceptions related to usefulness and ease of use, others have offered general and contextindependent determinants that can be applied across a broad array of settings (Venkatesh, 2000; Venkatesh & Davis, 2000). TAM is regarded as the most appropriate, powerful, robust, and parsimonious theoretical base to investigate user acceptance of an information technology due to specificity for the domain of information technology (Agarwal, 2000; Sharp, 2007); its strong theoretical base and sufficiently validated inventory of psychometric measurement scales (Davis, 1989; Yousafzai et al., 2007); its validity across a wide range of technologies, user populations, situations, cultures, countries, and expertise levels (Chuttur, 2009; Lee et al., 2003; Ma & Liu, 2004; Venkatesh, Davis, & Morris, 2007); and its good predictive validity for technology use (Leong, 2003; Yousafzai et al., 2007).
Hypotheses and Research Questions On the basis of TAM (Davis et al., 1989), perceived usefulness in this study refers to the extent to which police officers believe that using the 32
DOI: 10.1002/piq
Performance Improvement Quarterly
EPSS would improve their performance on work-related tasks. If an EPSS leads to successful job performance, performers are likely to accept using it (Collis & Verwijs, 1995; Laffey, 1995; Nguyen, Klein, & Sullivan, 2005). Prior research studies have confirmed a relationship between perceived usefulness and behavioral intention to use support systems (Habelow, 2000; Hung and Chao, 2007; van Schaik, Pearson, & Barker, 2002). In addition, users’ performance has an effect on their attitudes toward the EPSS (Moore & Orey, 2000). H1: The police officers’ perceptions of usefulness significantly and positively influence their behavioral intentions to use the EPSS. H2: The police officers’ perceptions of usefulness significantly and positively influence their attitudes toward using the EPSS. Based on TAM (Davis et al., 1989), perceived ease of use in this study refers to the extent to which police officers believe that using the EPSS would not require much effort to use. It is asserted that the success of an EPSS depends on its ease of use (Gery, 1991; Milhelm, 1997). The difficulty in using an EPSS tends to impair the potential of the system in improving job performance. Moreover, the interface design of an EPSS influences not only the way people interact with and navigate through an EPSS, but also how they feel about using it (Stevens & Stevens, 1995). Prior research studies supported that perceived ease of using the EPSS was associated with perceived usefulness and attitude toward using computers (Habelow, 2000). H3: The police officers’ perceptions of ease of use significantly and positively influence their perceived usefulness of the EPSS. H4: The police officers’ perceptions of ease of use significantly and positively influence their attitudes toward using the EPSS. TAM implies that users’ evaluative affect associated with a particular system has influence on their likelihood to use it. Rogers (2003) indicates that individuals’ choices to adopt or reject an innovation are usually consistent with their attitudes toward it. Likewise, an earlier research study showed a strong positive relationship between users’ attitudes toward the EPSS and their willingness to use it in the future (Gal & Nachmias, 2012). H5: The police officers’ attitudes toward use significantly and positively influence their behavioral intentions to use the EPSS. As noted earlier, perceived usefulness and perceived ease of use are the fundamental determinants of user acceptance, which can be Volume 27, Number 4 / 2015
DOI: 10.1002/piq
33
manipulated to foster acceptance behavior (Davis, 1989). In the development of an EPSS, an understanding of the determinants of users’ perceptions regarding usefulness and ease of use gives designers valuable information to maximize user acceptance (van Schaik, 2010). Therefore, in the qualitative part of the study, the following two research questions were investigated: RQ1: What do the police officers consider when they judge the ease of use of the EPSS? RQ2: What do the police officers consider when they judge the usefulness of the EPSS?
Method Research Design This study employed a mixed methods research design in which quantitative and qualitative data collection and analysis procedures were conducted concurrently and held approximately equal weight (or emphasis) to fully understand the acceptance of the EPSS. The quantitative and qualitative findings were merged in the interpretation stage of the study. The main rationale behind using a mixed methods research design was to reach a more complete understanding of the acceptance of the EPSS, because quantitative data alone would be limited to measurement of generalized perceptions in TAM and provide abstract and general findings that were an impediment to the identification of practical implementations enhancing the acceptance of the EPSS. The qualitative data collection and analysis enabled an in-depth and detailed understanding of users’ key beliefs associated with the acceptance of the EPSS (i.e., perceived usefulness and perceived ease of use).
Work and Support Systems As stated earlier, the study investigated the acceptance of the EPSS designed for tasks performed by police officers in the Crime Scene Investigation and Identification Units of the Turkish National Police. The basic responsibilities of this unit involve (a) performing judicial duties after an incident or an action defined as a crime, (b) finding, defining, collecting, recording, protecting, and wrapping of real evidence associated with a committed crime, and (c) evaluating and delivering real evidence taken from a crime scene to concerned units (“Crime Scene Investigation”, n.d.). This unit has a hierarchical organization structure consisting of police officers and police chiefs with different ranks. All staffs are experts who perform tasks in a specific expertise area such as crime scene investigation, latent print development, and print identification. 34
DOI: 10.1002/piq
Performance Improvement Quarterly
In the study, the EPSS provided police officers with information, tools, and resources that helped them complete tasks related to a crime scene investigation and identification. It also structured and facilitated the work processes of police officers. This system was integrated into application software, which was concurrently developed to enable police officers to record data related to crimes, evidences, analyses, and results. Police officers could access the performance support systems using mobile personal computers and desktop computers in their offices. The mobile personal computers are the fully ruggedized computers with 2 GB RAM, 64 GB solid-state drive, 16 GHz CPU, Intel GPU, backlit little keyboard, and a 5.6” WSVGA sunlight-viewable touchscreen. Based on the classification of support systems by Gery (1995), Table 1 shows intrinsic, extrinsic, and external performance support systems involved in the EPSS. Participants The participants of the study were police officers in the Crime Scene Investigation and Identification Units of six provinces in Turkey. These units were strategically and purposively selected on the basis of the data related to work load, the number of police officers, information technology awareness, and information technology infrastructure to reflect the general nature of the units in Turkey. Quantitative data were collected from 209 police officers. The majority of the participants (90.9%) were male, which reflected the male-dominated culture of the Turkish Crime Scene Investigation and Identification Units. The age of the participants ranged from 23 to 52, with a mean of 39.07 (SD = 4.81). More than half (53.6%) held an associate degree. The participants’ work experience ranged from 1 to 20 years, with a mean of 10.43 (SD = 4.50). Most of the respondents (65%) anticipated the heavy use of the EPSS on their job. TABLE 1 THE COMPONENTS OF THE EPSS TYPE
COMPONENTS a
Intrinsic supports
MAIN FUNCTION
Wizard
Step-by-step guidance to prepare a crime scene investigation report
Job-task automation tool
Automatic formation of reports prepared for each investigation on evidence
Workflow tool
Regulation of workflow from receiving a denouncement to completing an investigation
Extrinsic supportb
Context-sensitive support system
On-demand access to contextual support information associated with performed tasks
External supportc
Content management system
Access to training documents as well as support information
a
Integrated with work interface and process. Integrated with tasks at hand, but not inherent to primary work interface or actual workflow. c Not integrated with either the system or primary workspace. b
Volume 27, Number 4 / 2015
DOI: 10.1002/piq
35
In the qualitative phase of the study, purposeful sampling with an intensity sampling strategy was employed to select information-rich, not highly unusual, police officers. As a result of observations the researchers made through involvement with police officers in the settings and information obtained from police inspectors or trainers of the system, interviewees consisted of 15 male police officers who had a considerable experience with the EPSS, comprehended its features and capabilities well, and were interested in using it. Instrumentation Questionnaire. In the quantitative phase of the study, a two-section questionnaire was used. The first section contained 19 items to measure four constructs of TAM. Perceived usefulness and perceived ease of use were measured using six Likert-type items for each, adapted from Davis (1989). Attitude toward using was measured using four standard 7-point semantic differential scale items, adapted from Taylor and Todd (1995). Behavioral intention to use was measured using two Likert-type items and one short-answer item, adapted from Venkatesh and Bala (2008). However, the short-answer item (“I plan to use the system in the next months”) was transformed into a Likert-type item (“Given that I have access the system, I plan to use it”). Consistent with the studies from which the scales were adapted, all Likert-type items used 7-point Likert scales, ranging from 1 (strongly disagree) to 7 (strongly agree). The back translation method (Brislin, 1980) was utilized in translating items from English to Turkish in order to ensure the equivalence of the items. The second section of the questionnaire required participants to provide their demographic information. Before the main study, a pilot study was conducted using 149 police officers (145 male, 4 female) in the Turkish Crime Scene Investigation and Identification Units to verify the reliability and validity of the scales. The focus of the pilot study was on the acceptance of computers on the job. A factor analysis using maximum likelihood factor extraction was conducted with direct oblimin rotation. Table 2 shows the results of the factor analysis. One item related to perceived usefulness (PU4) and two items related to perceived ease of use (PEU3 and PEU4) were removed because of not having high loading on the factors. The scales related to perceived usefulness, perceived ease of use, and attitude toward use had high reliabilities, Cronbach’s α = 91, .83, and .88, respectively. However, the reliability of behavioral intention to use scale was not satisfactorily high; Cronbach’s α = .64. According to Hair, Black, Babin, Anderson, and Tatham (2006), for reliability of a scale, although Cronbach`s alpha should be greater than a threshold of .70, a .60 level can be acceptable in exploratory research. Interview Guide. In the qualitative phase of the study, an interview guide was used to obtain rich and in-depth understanding of the 36
DOI: 10.1002/piq
Performance Improvement Quarterly
TABLE 2
SUMMARY OF A FACTOR ANALYSIS FACTORS
ITEM
1
PU1
.98
PU3
.95
PU2
.77
PU6
.70
PU5
.51
2
PEU6
.82
PEU1
.75
PEU5
.72
PEU2
.57
3
A2
.97
A1
.97
A3
.70
A4
.63
4
IU4
.78
IU2
.53
IU1
.37
Note. n = 149; KMO = .85; χ2 (171) = 1653.19, p < .001. Factor 1 = perceived usefulness (PU); Factor 2 = perceived ease of use (PEU); Factor 3 = attitude toward using (A); Factor 4 = behavioral intention to use (IU).
acceptance of the EPSS. It included questions regarding the usefulness of the EPSS, factors influencing the usefulness and ease of use of the EPSS, and facilitating conditions for using the EPSS. The current study particularly focused on the questions related to factors that police officers considered for usefulness and ease of use of the EPSS, which are primary drivers of user acceptance in TAM. To improve the clarity of the interview questions, the guide was independently reviewed by two experts and one police officer. Procedures Before collecting data, official permission was gained from the head of the Crime Scene Investigation and Identification Units through a formal letter. In the quantitative phase of the study, the questionnaire was administered after trainings on the use of the EPSS. The majority of the police officers had the opportunity to have hands-on experience with the EPSS during the trainings. In the qualitative phase of the study, one of the researchers conducted all semistructured interviews after interviewees were identified in each of the Crime Scene Investigation and Identification Units. Prior to the interviews, the researcher tried to spend a considerable amount of time with interviewees in order to establish Volume 27, Number 4 / 2015
DOI: 10.1002/piq
37
trust and a rapport with them. All except one interview was recorded. The Middle East Technical University Human Subjects Ethics Committee approved that this study guarantees the protection of participants. Data Analysis In the analysis of the quantitative data, structural equation modeling (SEM) was used to test the hypotheses offered on the basis of TAM. In the SEM analysis, a two-stage approach was followed in which the measurement model was tested before the structural model (Hair et al., 2006). AMOS 18.0 was used to test these models using the maximum likelihood estimation method. Modification indices were used to modify the model when it was required to improve model fit. Preliminary analyses were conducted to ensure no violation of assumptions of sample size, missing data, outliers, multivariate normality, and multicollinearity. The five cases detected as outliers were removed on the basis of the rule of thumb stated by Byrne (2010). In the analysis of the qualitative data, a content analysis was used following general analytical procedures outlined by Creswell (2007). In the qualitative data analysis process, first, one of the researchers transcribed all records of the interviews; then the researcher read through transcripts and took some memos about initial thoughts, interpretations, and concepts. Later, he chose an interview, and read and searched through the data for the patterns. The researcher identified and assigned a conceptual label into text segments representing incidents, ideas, events, and acts related to purpose of the study. After that, he engaged in aggregating similar codes into categories that represented outstanding issues and matters. Finally, the results were presented in a narrative form. In the qualitative data analysis, peer debriefing was used to enable the researcher to understand his own bias (if any) and to clarify his experiences, perceptions, and interpretations related to the acceptance of the EPSS. Moreover, an audit trail, which consisted of interview records, interview notes, original interview transcripts, data analysis documents, and the interpretation of the data, was kept to present the course of the qualitative data analysis, which contributed to dependability and confirmability of the findings.
Findings The Findings of the Quantitative Phase of the Study The measurement model was assessed using confirmatory factor analysis. The initial assessment of the overall fit of the model revealed a mediocre model fit, χ2 (98) = 270.58, p = .00, CFI = .94, RMSEA = .093. To improve the model fit, the review of the modification indices suggested a covariance between two pairs of error terms, which were justified by 38
DOI: 10.1002/piq
Performance Improvement Quarterly
content overlap between items. After the modifications in the model, the goodness-of-fit indices indicated an acceptable fit, χ2 (96) = 194.58, p = .00, CFI = .97, RMSEA = .071. The Cronbach’s alpha values of the scales of perceived usefulness, perceived ease of use, attitude toward use, and behavioral intention to use were .91, .95, .93, and .91, respectively, indicating high reliabilities. In the assessment of the structural model, first, the overall model fit was assessed. The findings showed that it was acceptable, χ2 (97) = 208.76, p = .00, CFI = .96, RMSEA = .075. Second, the structural parameter estimates were examined for the hypothesized relationships. They indicated that all hypothesized structural relationships were statistically significant in the predicted directions. Figure 2 and Table 3 depict the structural model with parameter estimates. Overall, the model explained 47% of variance in intention to use, 48% in the attitude toward usage, and 55% in perceived usefulness. The Findings of the Qualitative Phase of the Study RQ1: Factors Influencing the Perceived Ease of Use of the EPSS. The analysis of the interviews showed that factors influencing perceived ease of use of the EPSS could be described in three subcategories: (1) user personal characteristics, (2) system characteristics, and (3) organizational characteristics. In terms of user personal characteristics, first, many of the police officers noted that they would use the system more effectively when getting more experience with it. Second, the participants clearly indicated that they should have basic computer knowledge and skill to use the system in an effective and efficient way.
Note. PU = perceived usefulness, PEU = perceived ease of use, A = attitude toward using, IU = behavioral intention to use. * p < .05. ** p < .01. *** p < .001. FIGURE 2.
THE STRUCTURAL MODEL WITH STANDARDIZED STRUCTURAL PARAMETER ESTIMATES
Volume 27, Number 4 / 2015
DOI: 10.1002/piq
39
TABLE 3
SUMMARY OF HYPOTHESIS TESTS
HYPOTHESES
PATH
STANDARDIZED COEFFICIENT
UNSTANDARDIZED COEFFICIENT
SE
RESULTS
H1
PU→IU
.50***
.40***
.07
Supported
H2
PU→A
.47***
.41***
.09
Supported
H3
PEU→PU
.74***
.89***
.09
Supported
H4
PEU→A
.27**
.29**
.10
Supported
H5
A→IU
.24**
.22**
.08
Supported
Note. PU = perceived usefulness, PEU = perceived ease of use, A = attitude toward using, IU = behavioral intention to use. **p < .01. ***p < .001.
Last, the police officers focused on the influence of their voluntariness on the use of the EPSS. One of them stated that when a police officer was not eager to use the system, he did not enjoy using it in the job. With respect to system characteristics, first, most of the police officers stated that simplicity and clarity of the system interface had an influence on the ease of use of the system. Second, the police officers stressed the critical role of user friendliness in the ease of use of the system. They commonly indicated that a user with basic computer knowledge and skill could use the system easily. Third, the interviewees focused on simplified data entry in the system. They noted that the system would enable them to enter data mainly by selecting a value from a specified list of choices, rather than typing. Fourth, the police officers emphasized the importance of relevance of the system to their job. Especially, the police officers who were responsible for crime scene investigation indicated that high relevance of the wizard to their work practices would make their interaction with the system easy. Last, the participants stressed that the low usability of the mobile personal computers might hinder them in using the system easily. Regarding organizational characteristics, first, many of the police officers stressed the demand for the improvement of information technology infrastructure in terms of network, computer, and hardware in the organization. One of them explicitly stated that they had experienced some challenges in using the system because of low configuration of the computers, which impeded ease of use of the system. Second, many of the interviewees focused on training and support. Most of them indicated the importance of training in enhancing their understanding of the system. In addition to training, they also placed emphasis on the influence of help systems, peer support, and technical support on the ease of use of the system. RQ2: Factors Influencing the Perceived Usefulness of the EPSS. Similar to perceived ease of use, the analysis of the interviews indicated that factors influencing perceived usefulness of the EPSS could be covered 40
DOI: 10.1002/piq
Performance Improvement Quarterly
under three subcategories: (1) user personal characteristics, (2) system characteristics, and (3) organizational characteristics. In terms of user personal characteristics influencing perceived usefulness of the EPSS, first, the police officers stated that although many of them had adequate knowledge and skills to use computers, it was necessary to have computer knowledge and skills in order to benefit from the EPSS effectively. Second, many of the police officers addressed that the more experience they had with the system, the more benefit they would get from it. Third, some of the interviewees noted that their anxiety about using the system would limit usefulness of the system. Fourth, some of the police officers stressed the role of their enjoyment with the system use in the usefulness of the EPSS. Last, the police officers indicated that their motivation to use the system was important for the usefulness of the EPSS. They stated that when they were not enthusiastic and willing to use the system, they would not use it effectively. With regard to system characteristics, first, the majority of the police officers thought that performance support facilities such as access to data and information, step-by-step guidance, and automating job-related tasks would make it possible for them to perform the tasks effectively and successfully. Second, the police officers stressed the importance of relevance of the system to their job. For instance, one of the police officers said, “If it is not relevant [to the job], we cannot say it is useful.” Third, some of the police officers noted that some parts of the system overloaded them, and so they could not realize the potential of the system. Fourth, the police officers highlighted the influence of the user interface design (e.g., layout, buttons, data input) on the usefulness of the system. Fifth, the interviewees stressed the importance of the usability of the mobile personal computers. Specifically, they noted that because of their small keyboards, the mobile personal computers made typing or controlling the mouse difficult for them and, in turn, decreased the usability and usefulness of the EPSS. Sixth, the police officers focused on the impact of user friendliness of the system, particularly the learnability of the system, on the perceived usefulness of the EPSS. Last, the interviewees noted that updates should fix problems or bugs, improve existing tools and resources, and add new functions and functionalities to the system. Regarding organizational characteristics, first, the majority of the police officers placed an emphasis on training offered for using the system. One of them clearly said, “When he or she [the police officer] gets stuck, and he or she does not get training support, he or she may think ‘I cannot use it anyway, I will also not be able to use it.’ ” Second, the police officers expressed that information technology infrastructure (e.g., network, computer hardware) should be improved in order for the system to be effective. Third, the interviewees attached importance to end-user support for the usefulness of the EPSS. Last, the police officers underlined the importance of personnel management for effective use of the EPSS. Table 4 summarizes the factors which the police officers considered as influencing the usefulness and ease of use of the EPSS. Volume 27, Number 4 / 2015
DOI: 10.1002/piq
41
TABLE 4
SUMMARY OF THE FACTORS INFLUENCING PERCEIVED USEFULNESS AND PERCEIVED EASE OF USE OF THE EPSS
USER PERSONAL CHARACTERISTICS
SYSTEM CHARACTERISTICS
ORGANIZATIONAL CHARACTERISTICS
Computer literacya,b
Simplified data entryb
Enjoymenta
Performance support facilitiesa
Information technology infrastructurea,b
Experience with the systema,b
Relevance to the joba,b
Personnel managementa
Motivationa
System complexitya
Traininga,b
Anxietya
Updatesa
Support facilitiesa,b
Voluntarinessb
Usability of the devicesa,b User friendlinessa,b User interfacea,b
a b
Considered to influence perceived usefulness. Considered to influence perceived ease of use.
Discussion The purpose of the study was to explore user acceptance of an EPSS by examining the relationships hypothesized in TAM and to understand determinants of perceived usefulness and perceived ease of use of an EPSS. Consistent with TAM (Davis et al., 1989), it was found that perceived ease of use, perceived usefulness, and attitude toward use were important determinants of the acceptance of the EPSS. In addition, this study revealed several user personal, system, and organizational characteristics which influenced users’ perceptions related to usefulness and ease of use of the EPSS. The study found that perceived ease of use had a significant direct influence on both perceived usefulness and attitude toward using the EPSS. TAM points out that perceived ease of use of a system makes possible for a person to accomplish more tasks by making the same effort (Davis et al., 1989). Similarly, when an EPSS is easy to use, people may need less time to find information they require, make fewer errors, and spend less effort to obtain support from the system, which, in turn, improves task performance. With the focus on “keep it simple” as a change strategy, Rossett and Schafer (2007) noted “If performance support is required to use the performance support, abandon all hope” (p. 181). Likewise, this study suggests that the more users find EPSS components easy to use, the more they perceive them as useful. Moreover, the ease of use of a system leads users to have high selfefficacy in using it, which, in turn, influences their attitudes toward it (Davis et al., 1989). Supporting this notion, Stevens and Stevens (1995) stressed that people’s attitudes toward EPSS use are influenced by the interface design of the system. Similarly, this study suggests that users’ beliefs about ease of use of the system components influence their feelings toward using it. 42
DOI: 10.1002/piq
Performance Improvement Quarterly
Consistent with TAM (Davis et al., 1989), the findings also indicated that perceived usefulness of the EPSS largely influenced the acceptance of the system through its significant direct influence on both intention to use and attitude toward using the EPSS. Users’ decisions to use an EPSS depend on whether it helps users to be successThis study revealed several ful (Laffey, 1995; Nguyen et al., 2005). Unless an user personal, system, and EPSS results in successful job performance, peoorganizational characteristics ple are not likely to use it. Furthermore, users’ performance has an effect on their attitudes which influenced users’ toward the EPSS in particular as well as technolperceptions related to ogy in general (Moore & Orey, 2000). usefulness and ease of use of The results also indicated that attitude toward the EPSS. using the EPSS had a significant influence on behavioral intention to use it. In general, attitudes toward a particular behavior play an important role in determining an individual’s intention to perform it (Ajzen & Fishbein, 1980). Likewise, Gal and Nachmias (2012) found that users’ attitudes toward the EPSS were strongly associated with their willingness to use it. Especially in mandatory use environments, attitude appears to have more importance in the acceptance of a system (Brown, Massey, Montoya-Weiss, & Burkman, 2002; Yousafzai et al., 2007). Therefore, if there is a job requirement to use the EPSS, users’ positive or negative attitudes toward it require particular consideration for its acceptance. In this study, although police officers indicated many different factors influencing their beliefs about the usefulness and ease of use of the EPSS, some factors were more important than the others due to having influence on both usefulness and ease of use of the system. Figure 3 illustrates the extension of TAM involving these factors. Based on this model, this study suggests several recommendations to foster the acceptance of the EPSS. In this way, the study addresses the limitation of TAM in
User Personal Characteristics Computer literacy Experience with the system
Perceived Usefulness System Characteristics Relevance to the job User friendliness Usability of the devices User interface
Attitude toward Using
Behavioral Intention to Use
Perceived Ease of Use Organizational Characteristics Information technology infrastructure training Support facilities
FIGURE 3.
Volume 27, Number 4 / 2015
THE EXTENDED TAM
DOI: 10.1002/piq
43
understanding what makes a system useful and easy to use. These recommendations include, but are not limited to, the following. ♦ Users should have adequate computer knowledge and skills, which are necessary to use an EPSS effectively. ♦ Users should have adequate hands-on experience with an EPSS before actual usage. The opportunity to experiment with the system or user involvement through hands-on activity can enable users to obtain experience with an EPSS, which leads to a better understanding of features and functionalities of the system. ♦ Performance support systems should be relevant to tasks which users need assistance in performing on the job. Venkatesh and Bala (2008) suggest that design characteristics, user participants, management support, incentive alignment, training, organizational support, and peer support are interventions that influence employees’ perceptions regarding job relevance of a system. ♦ An EPSS should be user-friendly in terms of navigation, interface design, and basic operations. Based on parameters associated with the user friendliness of an information system (Nielsen, 1993), design processes should aim to develop performance support systems that are easy to learn, efficient to use, easy to remember, error free, and subjectively satisfying. ♦ The devices that provide access to an EPSS should be used in an effective, efficient, and satisfactory manner. Based on the standards of usability offered by the International Organization for Standardization (ISO 9241–11), designers should choose devices which provide an accurate and complete way for users to achieve their goals; require low effort, time, and financial cost to use; and do not cause users to have any discomfort or negative attitudes toward them. ♦ The user interfaces of an EPSS should be designed to make use of screen and system functions easy and effective for users. Performance-centered design, first outlined by Gery (1995) and subsequently revised by Marion (2002), articulates that the user interface of performance support systems should have embedded knowledge; make use of prior learning and physical reality; be compatible with natural work situations; deliver information in an appropriate way; provide helpful visualizations of information; and provide options for the application interface and resources. ♦ Organizations should plan their budgets and resources for hardware, software, and network infrastructures which meet minimum requirements of an EPSS. ♦ Organizations should organize training programs and offer enduser support facilities (e.g., help systems and technical support) to facilitate the utilization of an EPSS without problems. This study suggests that TAM is a valid model to explain users’ intentions to use an EPSS. More important, it provided a much better 44
DOI: 10.1002/piq
Performance Improvement Quarterly
understanding of factors influencing perceived usefulness and perceived ease of use of the EPSS. In the light of these findings, specific interventions should be taken into consideration in order to minimize user resistance, to develop an accurate perception related to usefulness and ease of use of an EPSS, and to encourage the acceptance of systems in organizations. Limitations and Recommendations for Future Studies The findings of the study were based on the EPSS which was specifically designed for job-related tasks in the Turkish Crime Scene Investigation and Identification Units, limiting the generalization of the findings. Therefore, future research should validate the quantitative and qualitative findings beyond specific conditions in the study. Moreover, this study focused on only police officers’ initial decision to use the EPSS. Thus, a future study should explain and understand the continued usage of EPSSs based on TAM. Similar to earlier TAM studies, a longitudinal study may provide empirical data that predict acceptance of an EPSS at different points in time. Furthermore, although the qualitative findings revealed the factors that the police officers considered when judging the usefulness and ease of use of the EPSS, this study did not test their contributions to perceived usefulness and perceived ease of use. Therefore, future quantitative research should investigate how well the factors predict perceived usefulness and perceived ease of use of the EPSS. Also the pattern of the relationships in TAM may be different in a mandatory use context (Brown et al., 2002). In such context, as noted earlier, employees` attitudes toward the technology are likely to matter more for technology usage (Brown et al., 2002; Yousafzai et al., 2007). In addition, the factors that have been found in the study as influencing the acceptance of the EPSS may not be practical for mandatory environments. Therefore, a future study should be carried out to compare acceptance of an EPSS in voluntary and mandatory use environments.
Acknowledgments We would like to express our sincere appreciation to the Scientific and Technological Research Council of Turkey (TUBITAK) for supporting the study.
References Agarwal, R. (2000). Individual acceptance of information technologies. In R. W. Zmud (Ed.), Farming the domains of IT management: Projecting the future through the past (pp. 85–104). Cincinnati, OH: Pinnaflex. Ajzen, I., & Fishbein, M. (1980). Understanding attitudes and predicting social behavior. Englewood Cliffs, NJ: Prentice-Hall. Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for future research. Personnel Journal, 41, 63–105. Barker, P. (2010). Technology perspective. In P. Barker & P. van Schaik (Eds.), Electronic performance support: Using digital technology to enhance human performance (pp. 55–79). Farnham, England: Gower.
Volume 27, Number 4 / 2015
DOI: 10.1002/piq
45
Barker, P., & Banerji, A. (1995). Designing electronic performance support systems. Innovations in Education and International Training, 32(1), 4–12. Brislin, R. W. (1980). Translation and content analysis of oral and written materials. In H. C. Triandis & J. W. Berry (Eds.), Handbook of cross-cultural psychology (Vol. 1, pp. 389–444). Boston, MA: Allyn & Bacon. Brown, S. A., Massey, A. P., Montoya-Weiss, M. M., & Burkman, J. R. (2002). Do I really have to? User acceptance of mandated technology. European Journal of Information Systems, 11(4), 283–295. Byrne, B. M. (2010). Structural equation modeling with AMOS: Basic concepts, applications, and programming. New York: Routledge. Carliner, S. (2002). Choices and challenges: Considerations for designing electronic performance support systems. Technical Communications, 49(4), 411–419. Carr, C. (1992). PSS! Help when you need it. Training & Development, 46(6), 30–38. Chuttur, M. Y. (2009). Overview of the technology acceptance model: Origins, developments and future directions. Sprouts: Working Papers on Information Systems, 9(37). Retrieved from http://sprouts.aisnet.org/9–37 Collis, B., & Verwijs, C. (1995). Evaluating electronic performance support systems: A methodology focused on future use-in-practice. Innovations in Education and Teaching International, 32(1), 23–30. Creswell, J. W. (2007). Qualitative inquiry & research design: Choosing among five approaches (2nd ed.). Thousand Oaks, CA: Sage Publications. Crime Scene Investigation. (n.d.). Retrieved November 24, 2014, from http://www. kpl. pol.tr/EN/Sayfalar/Olay.aspx Davis, F. (1985). A technology acceptance model for empirically testing new end-user information systems: Theory and results (Unpublished doctoral dissertation). Massachusetts Institute of Technology, Cambridge, MA. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user accaptance of information technology. MIS Quarterly, 13(3), 319–340. Davis, F. D. (1993). User acceptance of information technology: System characteristics, user perceptions and behavioral impacts. International Journal of Man-Machine Studies, 38, 475–487. Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982–1003. Gal, E., & Nachmias, R. (2012). The effect of users’ attitudes on electronic performance support systems implementation. Performance Improvement, 51(5), 22–31. Gery, G. (1991). Electronic performance support systems: How and why to remake the workplace through the strategic application of technology. Cambridge, MA: Ziff Communications Company. Gery, G. (1995). Attributes and behaviors of performance-centered systems. Performance Improvement Quarterly, 8(1), 47–93. Gery, G. (2002). Achieving performance and learning through performance-centered systems. Advances in Developing Human Resources, 4(4), 464–478. Habelow, E. M. (2000). Factors related to the use of an electronic performance support system (EPSS). (Doctoral dissertation). Available from ProQuest Dissertations & Theses database. (UMI No. 9969889). Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Multivariate data analysis (7th ed.). Upper Saddle River, NJ: Pearson Education. Hodges, T. K. (2002). Linking learning and performance. Boston, MA: Butterworth– Heinemann. Hung, W., & Chao, C. (2007). Integrating advance organizers and multidimensional information display in electronic performance support systems. Innovations in Education and Teaching International, 44(2), 181–198. Laffey, J. (1995). Dynamism in electronic performance support system. Performance Improvement Quarterly, 8(1), 31–46. Lee, Y., Kozar, K. A., & Larsen, K. R. (2003). The technology acceptance model: Past, present, and future. Communications of the Association for Information Systems, 12, 752–780.
46
DOI: 10.1002/piq
Performance Improvement Quarterly
Legris, P., Ingham, J., & Collerette, P. (2003). Why do people use information technology? A critical review of the technology acceptance model. Information & Management, 40(3), 191–204. Leong, L. (2003). Theoretical models in IS research and the technology acceptance model (TAM). In C. Davis (Ed.), Technologies and methodologies for evaluating information technology in business (pp. 1–31). Hershey, PA: IRM Press. Ma, Q., & Liu, L. (2004). The technology acceptance model: A meta-analysis of empirical findings. Journal of Organizational and End User Computing, 16(1), 59–72. Mao, J. (2004). Electronic performance support: An end-user training perspective. Journal of Information Technology Theory and Application, 5(4), 51–67. Marion, C. (2002). Attributes of performance-centered systems: What can we learn from five years of EPSS/PCD competition award winners? Technical Communication, 49(4), 428–443. Maughan, G. R. (2005). Electronic performance support systems and technological literacy. The Journal of Technology Studies, 31(1), 49–56. McKay, J., & Wager, W. W. (2007). Electronic performance support systems: Visions and viewpoints. In R. A. Reiser & J. V. Dempsey (Eds.), Trends and issues in instructional design and technology (2nd ed., pp. 147–155). Upper Saddle River, NJ: Pearson Education. Milhelm, W. (1997). Instructional design issues for electronic performance support systems. British Journal of Educational Technology, 28(2), 103–110. Moore, J. L., & Orey, M. A. (2000). The implementation of an electronic performance support system for teachers: An examination of usage, performance, and attitudes. Performance Improvement Quarterly, 14(1), 26–56. Nguyen, F. (2010). Electronic performance support systems. In R. Watkins, & D. Leigh (Eds.), Handbook of improving performance in the workplace, volume two: Selecting and implementing performance interventions (pp. 325–343). San Francisco, CA: Pfeiffer. Ngyuen, F. (2012). Performance support. In R. A. Reiser, & J. V. Dempsey (Eds.), Trends and issues in instructional design and technology (3rd ed., pp. 147–157). Boston, MA: Pearson. Nguyen, F., Klein, J. D., & Sullivan, H. (2005). A comparative study of electronic performance support systems. Performance Improvement Quarterly, 18(4), 71–86. Nielsen, J. (1993). Usability engineering. San Diego, CA: Academic Press. Puterbaugh, G., Rosenberg, M., & Sofman, R. (1989). Performance support tools: A step beyond training. Performance & Instruction, 28(10), 1–5. Raybould, B. (1995). Performance support engineering: An emerging development methodology for enabling organizational learning. Performance Improvement Quarterly, 8(1), 7–22. Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press. Rossett, A. (1996). Job aids and electronic performance support systems. In R. L. Craig (Ed.), The ASTD training and development handbook (pp. 554–578). New York, NY: McGraw-Hill. Rossett, A., & Schafer, L. (2007). Job aids and performance support: Moving from knowledge in the classroom to knowledge everywhere. San Francisco, CA: Pfeiffer. Sharp, J. H. (2007). Development, extension, and application: A review of the technology acceptance model. Information Systems Education Journal, 5(9), 1–11. Spitzer, D. R. (1999). The design and development of high-impact interventions. In H. D. Stolovitch & E. J. Keeps (Eds.), Handbook of human performance technology: Improving individual and organizational performance worldwide (2nd ed., pp. 163–184). San Francisco, CA: Jossey-Bass/Pfeiffer. Stevens, G. H., & Stevens, E. F. (1995). Designing electronic performance support tools: Improving workplace performance with hypertext, hypermedia, and multimedia. Englewood Cliffs, NJ: Educational Technology Publications. Stolovitch, H. D., & Keeps, E. J. (2004). Training ain’t performance. Alexandria, VA: ASTD Press.
Volume 27, Number 4 / 2015
DOI: 10.1002/piq
47
Stone, D. L., & Villachica, S. W. (2003). And then a miracle occurs! Ensuring the successful implementation of enterprisewide EPSS and e-learning from day one. Performance Improvement, 42(3), 42–51. Surry, D. W., & Ely, D. P. (2007). Adoption, diffusion, implementation, and institutionalization of instructional innovations. In R. A. Reiser & J. V. Dempsey (Eds.), Trends and issues in instructional design and technology (2nd ed., pp. 104–111). Upper Saddle River, NJ: Pearson Education. Taylor, S., & Todd, P. A. (1995). Understanding information technology usage? A test of competing models. Information Systems Research, 6(2), 144–176. van Schaik, P. (2010). Psychological perspective. In P. Barker & P. V. Schaik (Eds.), Electronic performance support: Using digital technology to enhance human ability (pp. 31–54). Farnham, England: Gower Publishing. van Schaik, P., Pearson, R., & Barker, P. (2002). Designing electronic performance support systems to facilitate learning. Innovations in Education and Teaching International, 39(4), 289–306. Venkatesh, V. (2000). Determinants of perceived ease of use: Integrating control, intrinsic motivation, and emotion into the technology acceptance model. Information System Research, 11(4), 342–365. Venkatesh, V., & Bala, H. (2008). Technology acceptance model 3 and a research agenda on interventions. Decision Sciences, 39(2), 273–315. Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186–204. Venkatesh, V., Davis, F. D., & Morris, M. G. (2007). Dead or alive? The development, trajectory and future of technology adoption research. Journal of the Association for Information Systems, 8(4), 267–286. Yousafzai, S. Y., Foxall, G. R., & Pallister, J. G. (2007). Technology acceptance: A metaanalysis of the TAM: Part 1. Journal of Modelling in Management, 2(3), 251–280.
EVREN SUMUER Evren Sumuer received his PhD from the Department of Computer Education and Instructional Technology at Middle East Technical University, Turkey, in 2012. He is currently a faculty member in the Department of Computer Education and Instructional Technology at Kocaeli University, Turkey. His research focuses on pre- and in-service teachers’ technology training, use of information and communication technologies in education, technology adoption, electronic performance support systems, educational use of social network sites, and instructional design. E-mail:
[email protected] SONER YILDIRIM Soner Yildirim, PhD, is currently a professor in the Department of Computer Education and Instructional Technology at Middle East Technical University, Turkey. He graduated with a PhD degree in instructional technology from the University of Southern California in 1997. His research interests include teachers’ technology training and integration, electronic performance support systems, learning objects, and the use of digital storytelling in preschool education. E-mail:
[email protected] 48
DOI: 10.1002/piq
Performance Improvement Quarterly
