Studying Customer Evaluations of Electronic Commerce ... - CiteSeerX

Proceedings of the 36th Hawaii International Conference on System Sciences - 2003

Studying Customer Evaluations of Electronic Commerce Applications: A Review and Adaptation of the Task-Technology Fit Perspective John D. Wells, [email protected] Saonee Sarker, [email protected] Andrew Urbaczewski, [email protected] Suprateek Sarker, [email protected] Washington State University Abstract The advantages arising from the emergence of electronic commerce (EC) are manifold. From automating inventory replenishment to replacing traditional sales channels with web-based interfaces, the concept of electronic commerce presents a number of research challenges and opportunities. A key prerequisite for leveraging EC applications is a thorough understanding of how customers interact with these applications. A primary challenge for the successful design and implementation of these applications is managing not only the increasingly heterogeneous nature of the task performed in the context of EC, but the wide variety of interaction devices (e.g., wireless phones, PDAs, etc.) that customers use to execute these tasks. The significance and primary contribution of this research effort is to adapt and apply Task-Technology Fit (TTF) Theory to an EC domain. Using the techniques outlined by Goodhue [14], TTF constructs for an EC task domain are derived by applying factors/principles from web usability research. The TTF determinants of task, technology, and individual characteristics are modified to fit an EC domain. An electronic commerce TTF instrument is presented along with a research model for better understanding the relationships between independent and dependent variables. Finally, plans and implications for future research are discussed.

1. Introduction One of the distinguishing characteristics of Electronic Commerce, hereafter referred to as ‘EC’, is that the customer has an unprecedented degree of control over its interaction with the organization [39]. This control is mediated via the use of information technology and its related applications. A key prerequisite for leveraging EC applications is a thorough understanding of how customers interact with these applications. A primary challenge for the successful design and implementation of these

applications is managing not only the increasingly heterogeneous nature of the task performed in the context of EC, but the wide variety of interaction devices (e.g., wireless phones, PDAs, etc.) that customers use to execute these tasks. The significance and primary contribution of this research effort is to adapt and apply Task-Technology Fit (TTF) Theory to an EC domain. While there has been considerable research in the area of web usability, it has typically been limited to a particular technology (e.g., browser-based web sites) [35, 42, 56]. Given the use of TTF as a theoretical framework, it is reasonable to expect that the evaluation constructs and measurement instrument created in this study possess a high degree of generalizability (i.e., can be applied to a number of EC domains). It also places a much needed emphasis on the emerging heterogeneity of interaction devices and interfaces that, up to this point, have been a non-issue due to the relative standardized nature of equipment that dedicated end-users utilized when carrying out their job-related tasks in an organizational context. The next section of this paper reviews the existing literature on individuals’ evaluation of information technology and, more specifically, TTF theory. In section 3, using the techniques outlined by Goodhue [14], TTF constructs for an EC task domain are derived by applying factors/principles from web usability research. In addition, research propositions are generated to examine customer evaluations of TTF. Section four provides an overview of the research model and the development of a TTF instrument for an EC domain. Section five discusses the results of an initial exploratory factor analysis (EFA) of the EC TTF constructs. Finally, the paper concludes by discussing the implications for applying this instrument and plans for future research.

2. Literature review The topic of individual evaluation of information technology (IT) and its related applications has been

0-7695-1874-5/03 $17.00 (C) 2003 IEEE

1


covered extensively in academic literature. Issues of interest range from user attitudes [23], accuracy [2], satisfaction [10], usability [9], and performance [28, 35]. These issues (e.g., usability, attitudes) have been explored using theoretical models such as Technology Acceptance (i.e., TAM) [9], Cognitive Fit [50, 51], and Task-Technology Fit (i.e., TTF) [14, 15, 16]. Of these theoretical models, the concept of ‘fit’ has been quite effective in terms of explaining the issues surrounding users and their interactions with IT, primarily from the perspective of a cognitive cost-benefit framework [36, 46]. The concept of ‘fit’ as an effective means for assessing user performance is supported by the following propositions: a) an individual’s performance is affected by how well technology options ‘fit’ his or her task requirements, b) ‘fit’ operates through its impact on task processes, c) individuals can evaluate fit and choose technologies on that basis. [14, p. 1830] Vessey [50] and Vessey/Galletta [51] explored the concept of ‘fit’ by focusing on the incongruence between ‘problem representations’ and ‘problem solving tasks’ (i.e., Cognitive Fit). Goodhue [14, 15] and Goodhue & Thompson [16] extended the concept of Cognitive Fit to a more generalized model (i.e., Task-Technology Fit) that took into account a wider range of factors that potentially affect how users evaluate IT’s ability to support their assigned tasks. However, the constructs, and subsequent measurement instruments, that were derived using the TTF model were designed for end-users operating in an organizational context [14, 15]. More specifically, the process in which these end-users processed quantitative information to perform their job-related tasks [8, 12, 41] was used to derive the TTF constructs. It has been noted by past researchers that user evaluations of TTF are context-specific, meaning that it “requires applying the perspective to a specific task domain” [14, p. 1840]. Therefore, while an organizational context makes sense for understanding dedicated end-users, the issues affecting how customers interact with an EC domain are very different [28, 29, 35]. The focus of this research is to apply the TTF theoretical model (see Figure 1) to derive the factors that are applicable to an EC domain and, subsequently, design/validate a measurement instrument for assessing how customers evaluate the usability of EC applications. The rationale for adapting TTF to an EC domain is based on the distinguishing characteristics of the tasks, technology, and customers that affect this new environment.

Task Characteristics

Technology Characteristics

Task-Technology Fit

Performance Impacts

Individual Characteristics Figure 1: Task-Technology Fit [Goodhue 1995, 1998]

These fundamental differences can be illustrated across the determinants of TTF: Task, Technology, and Individual characteristics [16]. As compared to dedicated end-users in an organizational setting, Tasks performed by customers vary from broad information gathering to specific business transactions [28]. From a Technology perspective, differences exist for both physical and logical interfaces. Simply put, a higher degree of heterogeneity exists for EC customers because of the different types of interaction devices (e.g., cell phones, personal digital assistants (PDA), laptops, desktops) as well as the logical interfaces that can come with these devices (WAP-based, Palm-based, Web-based). Both the physical and logical limitations of these instances of IT have significant implications for how customers evaluate EC applications, including the much researched issues of graphical vs. tabular information presentation [27, 30, 48, 53], information overload [21], and cognitive capacity [31]. Finally, Individual characteristics differ for customers when compared to dedicated end-users. Because fundamental differences exist between job-related and consumerrelated tasks, the reasons that customers adopt EC applications diverge from the rationale used by organizational end-users. Based on this key difference, interfaces for EC applications have to take into account varying levels of customer IT expertise.

3. Applying task-technology fit to an electronic commerce domain The preceding discussion makes the argument that organizational and EC domains are distinct. Thus, justification is provided for the use of TTF as a means for understanding customer interaction with EC applications. To strengthen such an assertion, the TTF model must be operationalized and validated within a specific EC domain. Therefore, the task domain for this study is the customer decision process for conducting EC transactions, which includes all the prepurchase, purchase, and post-purchase subtasks that makeup this process [34].

0-7695-1874-5/03 $17.00 (C) 2003 IEEE

2


Table 1: Assessing Web usability within the customer decision process Customer Decision Process Need Recognition

Content

Navigation

Interactivity

•Targeted Product Offerings •Profile Information – Accurate and Relevant •Accurate (real-time) and Complete

•Accessible Offering

•Intelligent Agents •Contact Means (Email, SMS, etc) •Customized Search Mechanism •Accessible information •Configuration •Customization

Information Search

Evaluation Purchase

Post-Purchase

•Information Presentation (Accurate, Relevant, Understandable) •Profile Information (Payment, Shipping Info, etc.) •Accurate (real-time) and Relevant •Profile Information (Order History)

•Application Design/Structure •Information Organization •Ease of Use (HW/SW) •Ease of Use (HW/SW) •Efficient Checkout •Shopping Cart •FAQ •Returns •Service Support

The foundation for using the customer decision process is grounded in basic marketing literature [38], yet has been used specifically to understand the relationship between information technology and customer interaction [19]. The customer service life cycle [25] has been used as a framework for exploring IT-supported customer service [26] and understanding the early growth of EC [22]. As research in EC has continued to evolve, a number of studies have focused on web site usability within the context of the customer decision process [1, 4, 42].

3.1 Web site usability Using web site usability principles as a guiding foundation, specific TTF constructs are derived from a variety of sources. First, because some overlap exists between the constructs within the organizational TTF domain and those that exist within an EC domain, Goodhue’s [14] constructs are applied to our instrument where appropriate. Second, existing literature on web usability and performance is analyzed and used to derive constructs using the customer decision process as a guiding framework (Table 1). Because we are using previous web usability research as the foundation for deriving our constructs, it is logical to frame our discussion with commonly accepted evaluation parameters. Scharl and Bauer [42] proposed three criteria for evaluating commercial web sites: content, navigation, and interactivity. Palmer [35] validated a web site usability instrument that included these criteria as well as two additional factors: download delay and responsiveness. Within the scope of our research, we consider the download delay and responsiveness constructs as part of interactivity criterion, as both factors directly influence the interaction between the customer and the organization.

•Reconciliation •Authorization •Security •Contact Means (Email, SMS, etc)

These criteria are used to organize the constructs for the EC domain TTF instrument. 3.1.1 Content. When viewing content from within the context of the customer decision process (see Table 1), several web usability issues come to the forefront. Relevant Information [1, 56] is particularly important when supporting the need recognition, evaluation, and post-purchase processes. Accurate Information [1, 56] applies to the entire customer decision process, but is heavily emphasized in the information search and post-purchase processes. Complete Information [35, 56] is necessary for adequately supporting a customer’s ability to search for information and providing effective post-purchase support. Finally, Understandable Information [1, 56] is critical when customers evaluate products online. Table 2 presents a summary of the constructs that apply to the content evaluation parameter. Table 2: Constructs for the Content Evaluation Parameter EC TTF Web Usability Constructs Construct Relevant Relevant Information [56] Information Relevance [1] Accurate Accuracy [56] Information Current and Timely Information [1] Complete Completeness [56] Information Information/Content [35] Understandable Media Use [1] Information Readability [56]

3.1.2 Navigation. When viewing navigation from a customer decision process perspective (see Table 1), several web usability issues apply to this evaluation parameter. Process Efficiency [29] is important when considering how an interface should support the evaluation, purchase, and post-purchase steps in the customer decision process. Information Organization [1, 35] applies to the entire customer decision process, but is heavily emphasized in the information search,

0-7695-1874-5/03 $17.00 (C) 2003 IEEE

3


purchase, and post-purchase processes. The Logical Interface [33, 43] is a fundamental requirement particularly for information intensive activities such as information search and product/service evaluation. The Physical Interface [56] becomes an important consideration as customers move towards an increasingly heterogeneous set of interaction devices (PDAs, cell phones, etc.) to support a variety of customer decision processes. Table 3 presents a summary of the constructs that apply to the navigation evaluation parameter. Table 3: Constructs for the Navigation Evaluation Parameter EC TTF Web Usability Constructs Construct Process Ease of Use [1] Efficiency Information Navigation/Organization [35] Organization Structure [1] Logical Interface Application Structure/Navigation [33] Application Design [43] Physical Interface Technical Features [56]

3.1.3 Interactivity. When viewing interactivity from a customer decision process perspective (see Table 1), several web usability issues pertain to this evaluation parameter. Customization [1, 35] is an important consideration for providing customers with effective need recognition and product/service evaluation. Security [45, 56] is particularly applicable when interacting with customers during customized need recognition and facilitating purchase activities. System Performance [18, 35] is a primitive requirement that supports all of the steps within the customer decision process. Finally, Appropriate Contact/Interaction [24, 29] presents implications when customers are contacted prior to a purchase (i.e., need recognition) or after a purchase (i.e., post-purchase). Table 4 presents a summary of the constructs that apply to the interactivity evaluation parameter. Table 4: Constructs for the Interactivity Evaluation Parameter EC TTF Web Usability Constructs Construct Customization Interaction/Customization [35] Personalization [1] Security Individual Security [45] Web Site Security [56] System Web Site Downtime [18] Performance Download Delay [35] Appropriate Perceived Control [29] Contact/Interaction Unethical Customer Contact [24]

After defining the TTF constructs for the EC domain, the next step is to qualify the determinants of TTF evaluation. The following section discusses Task, Technology and Individual Characteristics, respectively. Subsequently, research propositions are presented that are intended to investigate both main

and interaction effects of each of these determinants on customers’ evaluation of TTF.

3.2 The relationship between customer tasks and TTF In general terms, tasks have been categorized across three dimensions: variety, difficulty, and interdependence [13]. However, there has been some debate as to whether a discernable difference exists between variety and difficulty [37], an issue that has been recognized by past TTF research [14]. Before tasks can be thoroughly understood, one must examine what constitutes a task as well as the process for making such a determination. Task analysis is typically a hierarchical decomposition process that breaks autonomous tasks into lower-level subtasks, eventually creating sequential task lists [20]. The nature of an autonomous task (i.e., collection of related subtasks) dictates the degree of variety, difficulty, and interdependence that a certain task possesses. 3.2.1 Task Structure. One can argue that task variety is directly related to homogeneity/heterogeneity of the task structure. When considering the structure of tasks as they relate to IT, Simon [44] distinguished between “programmed” and “non-programmed” decisions. The former consisted of repetitive/routine decision tasks while the latter were decision tasks that were novel and unstructured in nature. Gorry and Scott Morton [17] extended Simon’s representation of decision-making to a broader context of problem-solving using “structured” and “unstructured” as polar extremes. Within an EC domain, the relative structure of a customer’s task is distinguished by his/her purchase motivation [32] or as other researchers have categorized as goal-directed vs. experiential shopping [54]. Because of its nature, a goal directed task (e.g., purchasing a specific product) consists of a higher degree of structure than an experiential task (e.g., online browsing). 3.2.2 Task Complexity. A further distinction can be made between task variety and difficulty when one considers the relationship between task difficulty and complexity. We have argued that variety represents the relative structure of a task. However, whether a task is structured or unstructured does not indicate the relative complexity of the task. It is reasonable to infer that a simple task can be either structured (e.g., parking in your garage) or unstructured (e.g., finding a parking space before a football game) in nature. The number of distinct acts/information cues, the nature of relationships between task inputs and task products, and dynamic changes in the relationships between task inputs and task products determine task complexity [55,

0-7695-1874-5/03 $17.00 (C) 2003 IEEE

4


pp.66-71]. A task in an EC domain increases in complexity in proportion to the number of discriminating, specialized attributes that are associated with a customer’s transaction (i.e., task). For example, consider the difference in relative complexity between purchasing a book on Amazon.com and purchasing a customized computer on Dell.com. Because the Dell.com task requires that the customer manage a larger number of distinct acts and information cues as compared to the book purchase on Amazon.com, the complexity of the computer purchase task is significantly higher. Unstructured

Executing a repetitive, routine task using a relatively small, homogenous series of steps Example: Checking a stock quote

Executing a repetitive, routine task using a relatively large, heterogeneous series of steps Example: Purchasing a computer

Simple

Executing a novel, nonroutine task using a relatively large, heterogeneous series of steps Example: Researching a product/service with multiple discriminating, specialized attributes.

Complex

Executing a novel, nonroutine task using a relatively small, homogenous series of steps Example: Domain name search via search engine.

Structured Figure 2: Task Complexity vs. Task Structure

Within the EC domain, we have decided to address the issues and variety and difficulty by placing our focus on task structure and complexity (see Figure 2). It is not our intention to discount interdependence as an important consideration. However, we perceive interdependence, from an EC perspective, to be considered part of task complexity. From the TTF organization perspective, interdependence was a major issue because of the various data sources and integration issues [14]. However, because a primary goal of EC is effective customer interaction, organizations are motivated to facilitate interdependence issues. An obvious example of this type of facilitation is the handoff that occurs between Amazon.com and its distributors (e.g. UPS). IT enabled features such as hypermedia are well suited for reducing the effect of independence issues that were much more prevalent for internal organizational decision-making. Therefore, taking into account the issues of task structure and complexity, we offer the following research proposition:

P1: Task characteristics will affect a customer’s evaluation of Task-Technology Fit for electronic commerce applications.

3.3 The relationship technology and TTF

between

customer

The initial focus on the organizational end-user and TTF dealt with technology characteristics that were appropriate for that particular domain. Such issues included integration of data, access to hardware/software, and training [14]. Because the customer is the primary end-user in an EC domain, the technology characteristics are very different. While TTF from an organizational domain perspective could assume a relative static interaction device (i.e., computer terminal with 3270 emulation and standard keyboard), the devices that customers use to complete tasks in an EC domain possess a high degree of heterogeneity, with an increasing emphasis being placed on mobile devices [7]. The dynamic nature of interaction devices for EC has implications for both logical and physical interface design, which is evident from the following statement by web usability expert Jakob Nielsen: “The recommended way of dealing with device diversity on the Web is to separate presentation and content and encode the presentation-specific instruction in style sheets that can be optimized for each platform” [33, p. 68] 3.3.1 Logical Interface. In theory, the logical interface is a software-mediated means for facilitating humancomputer interaction and is supposed to be decoupled from the physical interface [43]. However, certain aspects of logical design are inherently tied to the capabilities of the physical device – otherwise known as device-dependency [5]. For example, a graphical user interface (GUI) is inherently dependent on a pointing device (e.g., mouse, pen). When designed effectively, a logical design can overcome the limitations that are often imposed by physical devices. Features such as menus, interactive icons, and GUI controls (e.g., radio buttons) can enhance user productivity, accuracy, and satisfaction when interacting with a logical interface [43]. Menus, for example, can enhance productivity by allowing customers to quickly navigate through an organization’s website. In lieu of such features, customer tasks become unnecessarily too unstructured and/or complex to complete in a reasonable amount of time. Therefore, a key challenge for the eventual success of EC is the development of logical interfaces that can maximize the potential of the physical device,

0-7695-1874-5/03 $17.00 (C) 2003 IEEE

5


particularly as customers move towards smaller and more mobile devices [7; 52].

3.4 The Relationship differences and TTF

3.3.2 Physical Interface. While addressing the issues associated with logical interface design is an important consideration, one cannot avoid the implications and constraints imposed by physical interaction devices, which is a hardware-mediated means for facilitating human-computer interaction. Any task, at a certain level, is dependent on the physical device [5, 6], which creates a distinction between external and internal tasks. External tasks, otherwise referred to as goals, possess a conceptual orientation and, from Benyon’s perspective, are independent of the physical device. Internal tasks are the actual instance of a task being executed and drive the selection of the physical device. For example, suppose someone is faced with the task of writing a two-page memo. The physical capabilities of the interaction device dictate which one is selected. A palm-based PDA, with its limited input capabilities, would be a counter-productive choice when compared to a desktop PC with an extended keyboard. However, physical interaction capabilities can be augmented to overcome some of these limitations. A common example of this phenomenon is the recent emergence of keyboard attachments for PDAs.

As supported by earlier TTF research, the ability of an end-user to complete certain tasks may differ based on his/her individual characteristics [14]. When examining TTF from an EC perspective, the characteristics that are specific to customers’ interacting in this particular domain should be considered. A popular characteristic for gauging Internet usage is computer literacy. Bellman et al. [3] allude to this by speculating that the most likely online shoppers have a “wired” lifestyle and further justify this position by stating that members of this group “work on the Internet in their offices every week, and they agree that the Internet and other developments in communication technology have improved their productivity at work” [p. 35]. Recent research suggests three characteristics for Internet shoppers: socioeconomic, motivational, and attitudinal [11]. Socioeconomic factors that were observed to be significant were age and income. Significant motivational factors were importance of convenience, need for innovativeness, lower risk aversion, higher impulsiveness, and a need for variety. Significant attitudinal factors included positive feelings towards direct marketing and online advertising. Therefore, taking into account the issues associated with individual customer characteristics, we offer the following research proposition:

3.3.3 Mobility. The move towards a customer’s desire for mobility has sparked the emergence of a wide variety of interaction devices. With these devices come both opportunities and challenges. For instance, these mobile devices present customers with an unprecedented ability to complete tasks when and where it is convenient for them. However, the uncertainty that comes with these capabilities leaves one to question what sorts of tasks are realistically eligible to be processed via these interaction devices. This dynamic environment presents challenges for both logical and physical interface design. To move EC closer to mass adoption, an effective concert must be achieved between logical and physical interfaces, particularly in light of the rapid movement towards smaller, mobile interaction devices. Simply put, by placing an emphasis on a data-centric design (i.e., decoupling of presentation and content) [33], the selection of an interaction device can be based on its ability to support a given task rather than have certain tasks drive the design of the device [6]. Therefore, taking into account the issues associated with logical and physical interfaces, we offer the following research proposition: P2: Technology characteristics will affect a customer’s evaluation of Task-Technology Fit for electronic commerce applications.

between

individual

P3: Individual characteristics will affect a customer’s evaluation of Task-Technology Fit for electronic commerce applications.

3.5 The interaction effects between task/individual and technology characteristics on TTF Tasks, when ready to be executed, will produce different evaluations of TTF when mixed with different technology characteristics. For instance, take the issue of device-dependency. Benyon [5] pointed to the relationship between task and technology characteristics that directly affects how well a user processes tasks. Tasks, at a pragmatic level, dictate which interaction devices are most effective from a productivity, accuracy, and satisfaction perspective [20]. For example, a customer may find a wireless PDA an acceptable means for checking a stock quote, but a poor fit for creating an extensive stock portfolio. The potential interaction between task and technology characteristics suggest the following research proposition:

0-7695-1874-5/03 $17.00 (C) 2003 IEEE

6


P4: The interaction between Task and Technology characteristics will affect a customer’s evaluation of Task-Technology Fit for electronic commerce applications. Individual characteristics, when mixed with different technology characteristics, will affect a customer’s overall evaluation of TTF. For example, a highly computer-literate customer perceives a much greater TTF when using certain types of technology as compared to a less technically proficient customer. Similar implications exist for other customer characteristics such as socioeconomic, motivational, and attitudinal. Therefore, potential interactions between individual and technology characteristics represent our last research proposition: P5: The interaction between Individual and Technology characteristics will affect a customer’s evaluation of Task-Technology Fit for electronic commerce applications.

4. Research development

model

and

instrument

Based on the constructs identified in the previous discussion along with the relevant determinants for TTF in an EC domain, we propose the following research model (see Figure 3). This model is consistent with prior TTF research and falls into a fit as moderation category, based on Venkataman’s [49] analysis of the various types of strategic fit. The independent variables consist of task, technology, and individual characteristics. The dependent variable is customer evaluations of TTF. Also, performance impact (denoted by dashed lines) is included in this model as a downstream dependent variable, but is not addressed within this paper. The following discussion outlines how each of the independent variables is operationalized and discusses the development of the TTF instrument for EC customer evaluations.

4.1 Task characteristics Task Characteristics Task Structure Task Complexity

Individual Characteristics Socioeconomic Motivational Attitudinal Internet Proficiency

4.2 Technology characteristics The Technology Characteristics variable represents the capabilities of both the logical and physical interfaces. The logical interface is qualified across a number of different criteria including: 1) GUI vs. Textbased interface, 2) Types of navigation controls (e.g., menus, radio buttons, etc.), 3) Information organization techniques (e.g., filtering). The physical interface consists of the features of the interaction device being used by the customer. Relevant features include: 1) I/O devices (e.g., keyboard, mouse, pen), 2) Screen size, 3) device attributes (e.g., size, weight, etc.). The latter of these features, size, is directly related to an important aspect of the physical device – mobility. The Mobility of a device is determined by not only its size and ease of use, but also its usefulness (e.g., the ability to establish a network connection).

4.3 Individual characteristics The Individual Characteristics variable is gauged by a set of attributes (e.g., age, aversion to risk, etc.) that have been observed to affect a customer’s EC shopping behavior [11]. Also, the customer’s Internet proficiency is assessed. This consists of a customer’s ability to effectively interact with Internet-based applications and is considered to be tangential to Rockart and Flannery’s [40] classification of end-users, but with a customer orientation. A set of questions has been developed to qualify the individual differences of customers interacting in an EC domain.

P1

Technology Characteristics Logical Interface Physical Interface Mobility

The Task Characteristics variable determines both the structure and complexity of the customer tasks. First, tasks are categorized by structure. Tasks are labeled as repetitive/routine or novel/nonroutine. From an EC perspective, task structure is dictated by whether the subject’s task is goal-directed (e.g., shopping for something specific) or experiential (e.g., browsing) [32]. Subsequently, task complexity is the product of an objective task analysis. In hierarchical fashion, a task structure is decomposed into subtasks until it reaches a primitive level [20]. From these primitive subtasks, the number of steps contained within autonomous tasks is calculated and used to ascertain the task’s relative complexity.

P4

4.4 Customer evaluations of TTF P2

P5

Customer Evaluations Of Task-Technology Fit

Performance Impacts

P3

Figure 3: Electronic Commerce Task-Technology Fit Research Model

As stated earlier, using the customer decision process as the task domain, web usability principles were applied to derive the EC TTF constructs. 36 questions have been developed to measure customer evaluations of TTF (NOTE: Due to space limitations, we could not include the instrument in the conference

0-7695-1874-5/03 $17.00 (C) 2003 IEEE

7


proceedings – it is available upon request). The questions are organized by Scharl and Bauer’s [42] criteria for evaluating commercial web sites: content, navigation, and interactivity, with each content evaluation parameter consisting of 12 questions, respectively.

5. Instrument validation Item reliabilities (see Table 5) and an exploratory factor analysis (EFA) were conducted as an important first step towards validating the EC TTF constructs. 49 subjects participated in the instrument validation. The results for the constructs within each evaluation parameter (e.g., content) will be discussed.

5.1 Content constructs For the content evaluation parameter, we identified 4 potential constructs (refer to Table 2). The factor loadings produced 3 components. The Complete Information, Understandable Information, and Accurate Information items loaded reasonably well. The Relevant Information construct cross-loaded with other factors and needs to be refined.

5.2 Navigation constructs For the navigation evaluation parameter, we identified 4 potential constructs (refer to Table 3). The factor loadings produced 2 components. The most important observation was that the Process Efficiency, Information Organization, and Logical Interface items loaded together. Upon further examination, this dynamic made a certain degree of sense as the items were geared towards how well the user interacted with the logical interface. Further validation is necessary to determine whether these constructs should be grouped into a single construct or refined to better isolate the each individual construct. The Physical Interface construct loaded very well.

5.3 Interactivity constructs For the interactivity evaluation parameter, we identified 4 potential constructs (refer to Table 4). The factor loadings produced 4 components, although one of them only had a strong loading for 1 item. The Security and System Performance constructs loaded well. However, Customization and Appropriate Content/Interaction did not load as well. Upon further examination, it became apparent that some of the items within each construct were focusing on customization while other items were focusing on customer control. We’ve concluded that Appropriate Content/Interaction will be replaced with Customer Control.

6. Conclusions

The benefits of a validated EC TTF instrument are manifold. Organizations can objectively analyze the tasks that their customers are executing and determine if the current technology is providing adequate support. If customers are having difficulty executing certain tasks, an organization has a couple of options. First, it can simplify the task by taking out unnecessary steps or by aggregating two or more steps. Second, and more realistically, the organization can adjust the technology to make it more conducive to customer tasks. Such adjustments can include enhancements to the logical and/or physical interfaces. Potential logical interface improvements include enhanced browser functionality, better navigation controls, and effective information organization/presentation. Depending on the customer’s set of tasks, physical interfaces can be improved by selecting an effective interaction device or by augmenting a current device (e.g., keyboard attachment for PDAs). The purpose of this paper is three-fold. First, existing TTF literature is reviewed and adapted to an EC domain. Second, research propositions are identified that explore customer evaluations of TTF for EC applications. Third, an initial instrument consisting of TTF constructs for an EC domain is presented. By gaining insight into the relationship between customer tasks and the technology used to execute such tasks, the opportunity to deploy more effective EC applications increases along with an organization’s ability to satisfy its customers.

Table 5: Reliabilities of the Constructs Content Construct Relevant Information Accurate Information Complete Information Understandable Information

Reliability (Cronbach’s Alpha) .65 .65 .65 .79

Navigation Construct Process Efficiency Information Organization Logical Interface Physical Interface

Reliability (Cronbach’s Alpha) .81

Interactivity Construct Customization

Reliability (Cronbach’s Alpha) .38

.74

Security

.76

.85

System performance Appropriate Contact/ Interaction

.69

.72

0-7695-1874-5/03 $17.00 (C) 2003 IEEE

.23

8


7. References [1] Agarwal, R. and Venkatesh, V. “Assessing a Firm’s Web Presence: A Heuristic Evaluation Procedure for the Measurement of Usability,” Information Systems Research, (13:2), June 2002, pp. 168-186. [2] Bailey, J.E. and Pearson, S.W. “Development of a Tool Measuring and Analyzing Computer User Satisfaction,” Management Science, (29:5), pp. 530-544. [3] Bellman, S., Lohse, G.L., and Johnson, E.J. “Predictors of Online Buying Behavior,” Communications of the ACM, (42:12), December 1999, pp. 32-38. [4] Benbunan-Fich, R. “Using Protocol Analysis to Evaluate the Usability of a Commercial Web Site,” Information & Management, (39:2), 2001, pp. 151-163. [5] Benyon, D.R. “The Role of Task Analysis in Systems Design,” Interacting with Computers, (4:1), 1992a, pp. 102121. [6] Benyon, D.R. “Task Analysis and Systems Design: The Discipline of Data,” Interacting with Computers, (4:2), 1992b, pp. 246-259. [7] Billsus, D., Brunk, C.A., Evans, C., Gladish, B, and Pazzani, M. “Adaptive Interfaces for Ubiquitous Web Access,” Communications of the ACM (45:5), May 2002, pp. 34-38. [8] Cowan, D.A. “Developing a Process model of Problem Recognition,” Academy of Management Review, (11:4), 1986, pp. 763-776. [9] Davis, F.D. “Perceived Usefulness, Ease of Use, and User Acceptance of Information Technology, “MIS Quarterly, September 1989, pp. 319-339. [10] Doll, W.J. and Torkzadeh, G. “The Measurement of End-user Computing Satisfaction,” MIS Quarterly, (12:2), June 1988, pp. 259-276. [11] Donthu, N. and Garcia, A. “The Internet Shopper,” Journal of Advertising Research, (39:3), May/June 1999, pp. 52-59. [12] Ference, T.P. “Organizational Communications Systems and the Decision Process,” Management Science, (17:2), October 1970, pp. B83-B96. [13] Fry, L.W. and Slocum, J.W. “Technology, Structure, and Workgroup Effectiveness: A Test of a Contingency Model,” Academy of Management Journal, (27:2), 1984, pp. 221-246. [14] Goodhue, D.L. “Understanding User Evaluations of Information Systems,” Management Science, (41:12), December 1995, pp. 1827-1844.

[15] Goodhue, D.L. “Development and Measurement Validity of a Task-Technology Fit Instrument for User Evaluations of Information Systems,” Decision Sciences, (29:1), Winter 1998, pp. 105-138. [16] Goodhue, D.L. and Thompson, R.L. “Task-Technology Fit and Individual Performance.” MIS Quarterly, June 1995, pp. 213-236. [17] Gorry, G.A. and Scott Morton, M.S. “A Framework for Management Information Systems,” Sloan Management Review, (30:3), Spring 1989, pp. 49-61. [18] Graham, A. “Don’t Let Downtime Be the Downfall of Your Web Site,” Journal of Business Strategy, (22:2), March/April 2001, pp. 10-12. [19] Grenci, R.T. and Todd, P.A. “Solutions-Driven Marketing,” Communications of the ACM, (45:3), 2002, pp. 64-71. [20] Hackos, J.T. and Redish, J.C. User and Task Analysis for Interface Design, (New York: Wiley Computer Publishing, 1998): pp. 51-90. [21] Hahn, M., Lawson, R. and Lee, Y.G. “The Effects of Time Pressure and Information Load on Decision Quality,” Psychology and Marketing, (9:5), 1992, pp. 365-378. [22] Haley, B.J., Carte, TA, and Watson, R.T. “Commerce on the Web: How is it Growing?,” Proceedings of Second Americas Conference on Information Systems, Phoenix, 1996, pp. 1-3. [23] Hartwick, J. and Barki, H. “Explaining the Role of User Participation in Information System Use,” Management Science, (40:4), 1994, pp. 440-465. [24] Huang, J. “Consumer Evaluations of Unethical Behaviors of Web Sites: A Cross-Cultural Comparison,” Journal of International Consumer Marketing, (13:4), 2001, pp. 51-71. [25] Ives, B. and Learmonth, G.P. “The Information Systems as a Competitive Weapon,” Communications of the ACM, (27:12), December 1984, pp. 1193-1201. [26] Ives, B. and Mason, R.O. “Can Information Technology Revitalize Your Customer Service?” Academy of Management Executive, (4:4), 1990, pp. 52-69. [27] Jarvenpaa, S.L. “The Effect of Task Demands and Graphical Format on Information Processing Strategies,” Management Science, (35:3), March 1989, pp. 285-303. [28] Jarvenpaa, S.L. and Todd, P.A. “Consumer Reactions to Electronic Shopping on the World Wide Web,” International Journal of Electronic Commerce, (1:2), Winter 1996-97, pp. 59-88.

0-7695-1874-5/03 $17.00 (C) 2003 IEEE

9


[29] Koufaris, M. “Applying the Technology Acceptance Model and Flow Theory to Online Consumer Behavior,” Information Systems Research, June 2002, (13:2), June 2002, pp. 205-223. [30] Lohse, G.L. “A Cognitive Model for Understanding Graphical Perception,” Human Computer Interaction, (8), 1993, pp. 353-388. [31] Miller, G.A. “The Magical Number Seven: Plus or Minus Two: Some Limits on our Capacity for Processing Information,” Psychological Review, (63), March 1956, pp. 81-97. [32] Moe, W.W. and Fader, P.S. “Uncovering Patterns in Cybershopping,” California Management Review, (43:4), Summer 2001, pp. 106-117. [33] Nielsen, J. “User Interface Directions for the Web,” Communications of the ACM, (42:1), January 1999, pp. 6572. [34] O’Keefe, R. and McEachern, T. “Web-Based Customer Decision Support Systems,” Communications of the ACM, (41:3), 1998, pp. 71-75. [35] Palmer, J.W. “Web Site Usability, Design, and Performance Metrics,” Information Systems Research, June 2002, (13:2), June 2002, pp. 151-167.

[44] Simon, H.A. The New Science of Management Decision (New York: Harper & Row, 1960), pp. 5-6. [45] Smith, H.J., Milberg, S.J., and Burke, S.J. “Information Privacy: Measuring Individuals’ Concerns About Organizational Practices,” MIS Quarterly, June 1996, pp. 167-195. [46] Smith, J.F., Mitchell, T.R., and Beach, L.R. “A CostBenefit Mechanism for Selecting Problem-Solving Strategies: Some Extensions and Empirical Tests,” Organizational Behavior and Human Performance, (29), 1982, pp. 370-396. [47] Torkzadeh, G. and Dhillon, G. “Measuring Factors that Influence the Success of Internet Commerce,” Information Systems Research, June 2002, (13:2), June 2002, pp. 187204. [48] Tractinsky, N. and Meyer, J. “Chartjunk or Goldgraph? Effect of Presentation Objectives and Content Desirability on Information Presentation,” MIS Quarterly, September 1999, pp. 397-420. [49] Venkataman, N. “The Concept of Fit in Strategy Research: Toward Verbal and Statistical Correspondence,” Academy of Management Review, (14:3), 1989, pp. 423-444.

[36] Payne, J.W. “Contingent Decision Behavior,” Psychological Bulletin, (92:2), 1982, pp. 382-402.

[50] Vessey, I. “Cognitive Fit: A Theory-Based Analysis of the Graphs versus Tables Literature,” Decision Sciences, (22:2), Spring 1991, pp. 219-241.

[37] Perrow, C. “A Framework for the Comparative Analysis of Organizations,” American Sociological Review, (32:2), 1967, pp. 194-208.

[51] Vessey, I. and Galletta, D., “Cognitive Fit: An Empirical Study of Information Acquisition,” Information Systems Research, (2:1), 1991, pp. 63-84.

[38] Pride, W.M. and Ferrell, O.C., Marketing: Concepts and Strategies, (Boston, Houghton Mifflin Co., 1995): p. 542.

[52] Urbaczewski, A., Wells, J.D., Sarker, S., and Koivisto, M. “Cultural Differences in Creating the Global Mobile Internet: A Theoretical Basis and Program of Research.” Proceedings of the Thirty-Fifth Hawaii International Conference on System Sciences, 2002.

[39] Rayport , J.F. and Jaworski, B.F. e-Commerce (New York: McGraw-Hill, 2001). [40] Rockart, J.F. and Flannery, L.S. “The Management of End User Computing,” Communications of the ACM, (26:10), October 1983. [41] Saunders, C. and Jones, J.W. “Temporal Sequences in Information Acquisition for Decision Making: A Focus on Source and Medium,” Academy of Management Review, (15:1), 1990, pp. 29-46. [42] Scharl, A. and Bauer, C. “Explorative Analysis and Evaluation of Commercial Web Information Systems,” Proceedings of the Twentieth International Conference on Information Systems, December 1999, pp. 534-539. [43] Shneiderman, B. Designing the User Interface: Strategies for Effective Human-Computer Interaction, (Reading, Massachusetts: Addison-Wesley Publishers, 1998).

[53] Washburne, J.N. “An Experimental Study of Various Graphic, Tabular, and Textual Methods of Presenting Quantitative Material,” Journal of Educational Psychology, June 1927, pp. 361-376. [54] Wolfinbarger, M. and Gilly, M.C. “Shopping Online for Freedom, Control, and Fun,” California Management Review, (43:2), Winter 2001, pp. 34-55. [55] Wood, R.E. “Task Complexity: Definition of the Construct,” Organizational Behavior and Human Decision Processes, (37:1), February 1986, pp. 60-83. [56] Zhang, P. and von Dran, G.M. “User Expectations and Rankings of Quality Factors in Different Web Site Domains,” International Journal of Electronic Commerce, (6:2), Winter 2001-2002, pp. 9-33.

0-7695-1874-5/03 $17.00 (C) 2003 IEEE

10