Network effects in technology acceptance: Laboratory experimental ...

G Model JJIM-907;

No. of Pages 10

ARTICLE IN PRESS International Journal of Information Management xxx (2009) xxx–xxx

Contents lists available at ScienceDirect

International Journal of Information Management journal homepage: www.elsevier.com/locate/ijinfomgt

Network effects in technology acceptance: Laboratory experimental evidence夽 Andrea Pontiggia a , Francesco Virili b,∗ a b

Ca’ Foscari University of Venice, Department of Business Economics and Management, San Giobbe, Cannaregio 873-30121 (VE), Italy Università degli Studi di Cassino, OrgLab – Dipartimento Impresa, Ambiente e Management – Facoltà di Economia Via S. Angelo, sn – 03043 Cassino (FR), Italy

a r t i c l e

i n f o

Article history: Available online xxx Keywords: Technology acceptance Network effects Network externalities Laboratory experiment

a b s t r a c t This research analyzes network effects in technology acceptance. The hypothesis is that the size of the user network affects technology acceptance. Even today, empirical measurement of network effects is challenging and there is a lack of experimental evidence. In order to investigate and measure the relationship between network size (number of adopters) and user acceptance, technology acceptance research needs to broaden its scope and approaches. To overcome this limitation we reproduce a particular type of technology acceptance process in a laboratory experiment, controlling for user network size and testing its influence on user perceptions and, ultimately, on acceptance decisions. We measured user perceptions and analyzed the data using consolidated and tested technology acceptance models. The results confirm our hypothesis, showing a significant effect of user network size on user perceptions. Finally, we discuss the theoretical and managerial implications of our approach and findings. © 2009 Elsevier Ltd. All rights reserved.

1. Introduction This empirical study investigates the influence of network effects on technology acceptance. Network effects (or network externalities1 ) occur when users are directly or indirectly connected in a network of relationships, experiencing growing benefits as the number of connections in the network increases. Technology acceptance is basically a choice among different alternative technologies/tools (such as software applications or computer systems) to accomplish user tasks. Many studies focus on this fundamental choice. The most tested theoretical approach is the so-called “Technology Acceptance Model” (TAM: Davis, Bagozzi, & Warshaw, 1989). Many different versions of the original model have since been developed in an attempt to explain user decisions and acceptance behaviour (Venkatesh, Morris, Davis, & Davis, 2003). Our present research study is based on the expectation that the candidate technology with a larger user network could be favoured

夽 A previous version of the paper was presented at the International Conference of Information Systems, ICIS, Paris 2008, with the title “Network effects in technology acceptance: Laboratory evidence”. ∗ Corresponding author. E-mail addresses: [email protected] (A. Pontiggia), [email protected] (F. Virili). 1 Network externalities are a particular type of network effect. The differences between network externalities and network effects, and their implications, are discussed in (Liebowitz & Margolis, 1994). We are taking into account all types of network effects, not just network externalities; an explicit discussion of the distinction is irrelevant to our purposes.

in comparison with the candidate technology with a smaller user network, because users may experience greater benefits with an increasing user network size, as predicted by economic studies on network effects. To our knowledge, this issue has never been directly addressed within technology acceptance models. Different reasons may have determined this lack of theory testing: first, network effects have been mainly investigated in Economics, at the macro-economic level; whereas technology acceptance processes have been investigated by behavioural studies in Information Systems, at the individual level; second, the empirical measurement of network effects is difficult to accomplish on the field; third, technology acceptance models were first proposed in the late 1980s, when network technologies (and their effects) were much less developed and recognized. This study aims at covering this empirical gap, proposing for the first time a direct measurement of the influence of user network size on technology acceptance processes using TAM. In particular, the following research issues will be discussed: • Which types of technology acceptance processes may be expected to show network effects, and why? • How can network effects be operationalized and measured? • How can the influence of network effects on the selected type of technology acceptance processes be empirically tested? The intended contribution of this study is theory testing more than theory building, with no explicit focus on proposing new theoretical explanations. The main objective is to verify whether user network size may affect certain technology acceptance processes, showing a need to explain and theorize the underlying reasons.

0268-4012/$ – see front matter © 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.ijinfomgt.2009.07.001

Please cite this article in press as: Pontiggia, A., & Virili, F. Network effects in technology acceptance: Laboratory experimental evidence. International Journal of Information Management (2009), doi:10.1016/j.ijinfomgt.2009.07.001

G Model JJIM-907; 2

No. of Pages 10

ARTICLE IN PRESS A. Pontiggia, F. Virili / International Journal of Information Management xxx (2009) xxx–xxx

Nevertheless, directions for theoretical development are advanced both in the literature analysis and in the discussion of the outcomes. The investigation described here is based on data from laboratory experiments, reproducing the technology acceptance process under network effects of different intensity. The methodological choice of the laboratory experiment is not only a way to overcome the well known difficulties in detecting and measuring different user network sizes; it is also useful to introduce a clear empirical distinction between network effects and other relational factors well known in technology acceptance, such as social norms, normative pressures, and social influence. The results confirm the role of network effects in user choice and show some unexpected outcomes. This empirical study rests on two main theoretical pillars: the theory of technology acceptance, and the studies on network effects/externalities. These contributions are discussed in the following section. The subsequent sections describe the experimental setting, the data collection process, and the empirical analysis, with a discussion and interpretation of the outcomes. Final remarks on the theoretical and managerial implications conclude the paper. 2. Relevant literature and theoretical focus 2.1. The Technology Acceptance Model (TAM) Understanding why people accept or reject technologies has proven to be one of the most challenging issues in Information Systems (IS) research: in the last 20 years technology acceptance has been among the most investigated topics in Information Systems. The theory of technology acceptance goes back to 1989, with the introduction of the Technology Acceptance Model (Davis, 1989; Davis et al., 1989), which soon emerged as one of the most widely cited and replicated studies in Information Systems as a whole. The numerous TAM-based empirical studies have been the subject of several critical reviews and meta-analyses, including (Lee, Kozar, & Larsen, 2003; Legris, Ingham, & Collerette, 2003; Hirschheim, 2007; King & He, 2006). Two aspects of the TAM-related theories and models are probably at the origin of their success in the academic world: their simplicity and their potential (King & He, 2006). TAM is simple: TAM sheds some light on a complex phenomenon (technology acceptance) on the basis of just two fundamental factors: the perceived “ease of use” and “usefulness” of the system. A TAM investigation is natural and immediate: through standard questionnaires and statistical analysis tools, researchers can easily make rigorous measurements of individual perceptions to build standard structural equation models. TAM has a wide potential for application: a good model of technology acceptance may result in an invaluable aid for making better systems, which would be more promptly and easily accepted by potential users. To recall the words of Davis’ group, “computer systems cannot improve organizational performance if they aren’t used” (Davis et al., 1989, p. 982). In (Lee et al., 2003) a simple graphical map of the original TAM and its extensions is depicted, with four main groups of additional factors, visualized here in Fig. 1. Besides the original TAM core (in the center), it is possible to distinguish (1) prior factors; (2) factors suggested by other theories; (3) contextual factors; and (4) consequent factors. TAM research has covered quite a lot of ground since it was first introduced. TAM is often taken as a solid empirical and methodological base for studies on user perceptions about technology (e.g. Ghorab (1997) and Lin (2006)), with applications to various technological domains, including Web systems (e.g. Chuan-Chuan Lin & Lu, 2000). The most significant contributions were recently integrated in a unified theory of acceptance, including constructs such

Fig. 1. The original TAM model and four categories of modifications. Source: King and He (2006, p. 741).

as “attitude towards technology”, “social influence”, “facilitating conditions”, “self-efficacy”, and “anxiety” (cf. Venkatesh et al., 2003, and references therein). To our knowledge, TAM research has not yet investigated whether bigger user networks may actually push potential new users toward acceptance. Conversely, a similar phenomenon has been extensively studied in Economics. In what follows, some of the main contributions of this research path are briefly recalled. 2.2. Economic studies on network effects Network effects and network externalities have been widely debated and investigated. The most cited and comprehensive literature reviews published in the last 20 years in this area include (David & Greenstein, 1990; Economides, 1996; Farrell & Klemperer, 2007; Stango, 2004). As an indication of the wealth of studies in this area, Farrell and Klemperer (2007) alone take into account over 470 different contributions. One of the most influential early studies on network externalities is Katz and Shapiro (1985). Their opening sentence is often quoted as a definition of network externality: “There are many products for which the utility that a user derives from consumption of the good increases with the number of other agents consuming the good” (p. 424). Katz and Shapiro propose a formal economic model of oligopolistic markets in presence of network externalities, showing two main results: first, the role of consumer expectations for the selection of the dominant seller “if consumers expect a seller to be dominant, then consumers will be willing to pay more for the firm’s product, and it will, in fact, be dominant” (p. 425); second, the need for social incentives in order to achieve product compatibility “we find that in our model the firms’ joint incentives for product compatibility are lower than the social incentives” (p. 425). Another well known account of the economic issues related to network externalities is given in Farrell and Saloner (1985). Many studies followed these two seminal papers. Network effects theories and related issues were popularized by Brian Arthur’s widely-cited account in Scientific American (Arthur, 1990). More recently, Katz and Shapiro (1994) took into account the so-called “systems markets”, involving products intimately related and working together, such as hardware and software within a standard system architecture (e.g. PC versus Mac software). The success of a new product is actually bound to the success of the entire system/architecture, with network effects playing an important role. In particular, three orders of decision are influenced by network effects: technology adoption decisions, product selection decisions and compatibility decisions. Again, the analysis is purely theoretical, based on existing studies and findings, but with no direct empirical support.


G Model JJIM-907;

No. of Pages 10


While many economic studies have been proposed for theoretical development, empirical evidence on network effects is much more scattered, and based on indirect approaches to measurement. Schilling (2002), for example, by using survey data addressing multiple products and industries, showed that the installed base (among other factors such as availability of complementary goods, a firm’s learning orientation and timing of entry) can play a significant role in market success. This study was preceded by a few other empirical studies limited to a single product category; as the author herself notes: “owing to the difficulty of gathering suitable data, most of the empirical work on network externalities has focused on a single product category; Gandal’s (1994) and Brynjolfsson and Kemerer’s (1996) studies of spreadsheet software, Wade’s (1995) study of microprocessors, and Shurmer’s (1993) work on prepackaged PC software are examples” (Schilling, 2002, p. 388). However, an accurate measurement of network effects would involve an accurate measurement of: • the “installed base” (number of product users) • some proxy for “consumer utility”, or perceived user benefits. None of the preceding studies could actually provide a definitive solution to these two measurement issues: for example, Schilling (2002) uses a 7-point Likert scale for the installed base (ranging from 1 = very large to 7 = small installed base). She does not measure user benefits at all, giving just a raw indication of “product success/failure”. To give another example, Brynjolfsson and Kemerer (1996) investigate the relationship between market prices and installed base in the spreadsheet market, finding significant, but only indirect, empirical evidence of network effects: “our model suggests that the positive network externality effects [. . .] are approximately as important as any of the intrinsic product features” (p. 1644). Their measure of installed base is based on “unit sales”, but as a proxy for “consumer utility” they can give only a very indirect (and potentially biased) estimate using product prices. All in all, the existing literature on network effects shows a rich tradition of sophisticated economic models and simulations demonstrating the theoretical relevance of user network size in the market diffusion of competing technologies and systems under various different assumptions and initial conditions. Such works are quite inspiring for the investigator interested in technology acceptance and suggest that network effects could actually matter. 2.3. Theoretical model As discussed earlier, the economic studies on network effects do not attempt to formulate formal theoretical explanations of the reasons why a growing user network size may produce user benefits. They also give only scattered and indirect empirical evidence of the actual existence of network effects. The main focus of their attention is on how the economic equilibrium and its social efficiency may be affected by a growing user network size for competing products or technologies. In this study, the focus of the research is limited to showing empirically whether network effects may influence technology acceptance. Further research might address the development of a theory explaining how and why this influence might be exerted. Nevertheless, theoretical insights might help to better motivate and position this contribution. In general, the user’s decision to accept or reject a particular type of technology may be affected by other users (i.e. through their user network) in two different ways, with different theoretical justifications. The first potential source of influence may be due to the social relationships with other users, and justified by the theories of social influence. The second source may be due to the size of the market for the economic transactions

3

to which the system is devoted, and this is taken into account in the economic theories on network effects. 2.3.1. Social effects Personal (or group) pressure may facilitate, convince, or actually push the user towards acceptance: for a team member the decision to adopt a particular type of project management software may be positively influenced by the attitudes and suggestions of the team leader and of the other team members already using that software. This type of effect may be related to the dimension of the user network: the bigger the user network, the stronger the pressures towards adoption. To a certain degree, the concept and construct of “subjective norm”, present in the behavioural theories that originally inspired TAM, such as the Theory of Reasoned Action (Fishbein & Ajzen, 1975) and the Theory of Planned Behaviour (Ajzen, 1991) may capture this type of inter-subjective or group influence. These types of issue have been widely discussed and tested in TAM literature, e.g. leading to the introduction of the “social influence” construct in some of the latest formulations of TAM (Venkatesh et al., 2003). This direction of investigation is promising: research is in progress to show how concepts and constructs such as the “informational social influence” (Salancik & Pfeffer, 1978) may actually intervene in the processes of attitude change (Bagozzi & Lee, 2002; Kelman, 1958) pushing users towards acceptance and use (Magni, Angst, & Agarwal, 2007; Magni & Pennarola, 2008). The focus of this investigation, however, is not on this type of effect. 2.3.2. Market size effects Economic studies suggest that there could be a second way, substantially different from social influence and similar constructs, in which a growing user network size may affect technology acceptance. Typical examples are the choice of PC/Windows versus Apple Macintosh systems for personal computing, or the choice of eBay versus Yahoo auction systems: it may happen that, ceteris paribus, user acceptance choices could be influenced in favour of those systems, such as PC/Windows and eBay, with the biggest installed bases and user networks, because of the important benefits associated with the biggest “markets”, e.g. a wider potential software application portfolio or a higher probability of finding a good buyer/seller at a decent auction price. The economic studies on network effects reviewed in the preceding section are actually (often implicitly) based on a similar assumption of perceived user economies due to an expanding market size for user transactions: these economies may positively affect user attitudes towards accepting the transactional system associated with the bigger market. This second type of user benefits, which increase with the dimension of the user network, are actually the focus of this study. This specific phenomenon under observation here (i.e. “market size” network effects) is not necessarily general: there are classes of systems/tasks, in which the existence of a market associated with a user network is not relevant for user acceptance: for example, the choice of using or rejecting a system internally developed in a factory, say, for controlling specific production machinery is going to be heavily influenced by the specific technical features of the system itself (e.g. effects on productivity, quality, failure rate, etc.); conversely, in such a case the existence of a big network of external users is probably not going to play a decisive role in system choice. An exemplary taxonomy of tasks significantly influenced by user network size follows: • Transactional, market-exchange tasks, with benefits generated by availability of a growing market size for transactions (e.g. emarketplaces, electronic commerce, electronic trading, banking, etc.). • Communication tasks, with benefits generated by an increasing number of actors and information available to communicate (e.g.


G Model JJIM-907;

No. of Pages 10

4


making telephone calls, sending fax messages, sending emails, etc.). • Learning tasks where an exchange of knowledge is required, with benefits generated by wider availability of knowledge and learning opportunities (e.g. consulting within a community of practice/professionals). • Secondary tasks where an exchange of goods/information/ knowledge is often desirable as a complement to the main task, with benefits generated by opportunities for market transactions, communication, or learning (e.g. writing documents or spreadsheets and exchanging files with other users, using a computer systems with a wide third-party/software application market, etc.).

a big user community may help and facilitate system adoption and usage in many ways, including more effective learning and knowledge sharing (e.g. Katz & Shapiro, 1994). On the other hand, when a system is used in interaction with a large network of users, actual system usage could be more complex and difficult, due to transaction costs including interaction, search, negotiation and similar activities. In this regard, negative network effects on the perceived ease of use of the system could be expected. The overall influence is therefore uncertain. As with most previous studies on network effects, we suggest here that the prevailing pattern would be positive, ultimately facilitating system acceptance:

On the other hand, user tasks of different nature may not show network effects, such as, for example, simple, independent and well-known information processing tasks (math, accounting, computer graphics, application software development), not requiring communication/exchange with other users.2 In this contribution, the methodological choice of the laboratory experiment allows us to expressly design a few model “marketlike” tasks with some of the characteristics described above.

Network effects may also have a direct positive influence on the behavioural intention to use the system, not mediated by perceived usefulness and perceived ease of use. This hypothesis would take into account the possibility that the user benefits due to an increased user network size, predicted by the economic studies on network effects, would positively affect the user behavioural intention of acceptance, without affecting user perceptions of the usefulness and ease of use of the system. This possibility cannot be excluded by the existing economic literature and it may therefore be tested for:

3. Hypothesis development No universal claim on technology acceptance is advanced here. We are not going to show that network effects always influence technology acceptance. What we are going to test is whether network effects could influence some acceptance processes, involving “market-like” tasks. Our general line of inquiry, therefore, implies that some specific technology acceptance processes might be influenced by network effects. This line of inquiry is articulated here in a set of hypotheses on the fundamental TAM constructs: perceived usefulness, perceived ease of use, and the behavioural intention of adoption. The first expectation is that user network size may have a positive impact on perceived usefulness. In systems targeted to market transactions (e.g. eBay), the probability of finding and getting the right item at the right price may increase with the number of users available in the market, i.e. with the network effects due to the increased size of the user network. In turn, the user’s perceived usefulness of the system may be enhanced by the improved chances of obtaining better deals. This expectation is in line with most of the research works on network effects reviewed in Section 2, evidencing or assuming growing perceived user benefits with an increased user network size (e.g. Economides, 1996). Therefore: Hypothesis 1. Perceived USEFULNESS can be positively influenced by network effects. The influence of network effects on perceived ease of use has contrasting aspects. Previous works on network effects show how

2 The existing literature is not very helpful for characterizing a general user task in terms of its “market degree”. There is a long tradition of organizational studies on task analysis, especially with regard to task complexity (Campbell, 1988; Wood, 1986). Task complexity has been theoretically related to Group Support Systems (GSS) (Zigurs & Buckland, 1998), showing how GSSs may give different basic types of group support (communication, information processing, and process structuring) in correspondence with different categories of task complexity (simple tasks, problem tasks, decision tasks, judgment tasks, fuzzy tasks). Some studies as (Mennecke, Valacich, & Wheeler, 2000; Wageman, 1995) take into account user and/or task interdependencies in a potentially useful way for our aim. But these concepts would require a much deeper theoretical elaboration, with multiple levels of analysis, possibly along the lines pointed out by the enlightening and deep conceptualization recently proposed in (Burton-Jones, 2005; Burton-Jones and Gallivan, 2007). Such an effort is here devoted to further research.

Hypothesis 2. Perceived EASE OF USE can be positively influenced by network effects.

Hypothesis 3. BEHAVIOURAL INTENTION of acceptance can be positively influenced by network effects. 4. Experimental design: task and system description As mentioned above, an empirical investigation on network effects in technology acceptance should deal with the following aspects: first, selecting the right class of technology acceptance processes, i.e. selecting a system appropriate for “market-like” tasks; second using a direct empirical measure for user network size; third, empirically testing for network effects in technology acceptance. A simple strategy for producing and testing empirical evidence for network effects in technology acceptance could be based on testing whether, ceteris paribus, user benefits in terms of changes in “perceived usefulness” and “perceived ease of use” would be determined by a growing user network size. To this aim, a laboratory experiment may be a powerful methodological choice to control for network effects and isolate them from contextual factors: the basic “ingredients” of system usage and technology acceptance — a user, a user task, different candidate systems/technologies — (Burton-Jones & Straub, 2006, p. 233) can be carefully replicated with different user network sizes, in order to observe and investigate network effects. The laboratory experiment is well known in IS research: see, for example, Silver (1988), Vance Wilson and Zigurs (1999). Some TAM research was also based on lab experiments: In a selection of 101 TAM studies, 86 were recently classified as field studies, 12 as laboratory experiments and four as qualitative studies (Lee et al., 2003). To simplify the experimental setting, a decision was taken to build and compare very simple “paper and pencil” systems instead of traditional hardware/software systems such as word processing or spreadsheet software applications. The use of a paper and pencil system is not new in IS research: for example in (Bailey & Konstan, 2003) paper and pencil design tools are compared with multimedia design tools. The underlying idea is that basic features for information codification, elaboration, and communication, may be present also in paper and pencil systems. In our specific application, the benefits of a growing user network are actually much more related to the nature of the user task than to the technicalities of the under-


G Model JJIM-907;

No. of Pages 10


lying information system. Even the simplest system (like paper and pencil) may show quite important network effects if the underlying task requires a market exchange, like children having to swap their double cards with friends: the dimension of the group of friends is relevant for the perceived benefits, more than the system (IT-based or not) used to negotiate and exchange cards. In consequence, the rationale for an experimental design is straightforward: one of the simplest possible “market-like” tasks could actually be inspired by card games in which users have to exchange doubles with other users. A system where a (sub)task, such as “exchanging cards”, is present could be a good basis to design an exemplary acceptance process including “market-like” tasks. We designed an experiment where an effort had to be exerted in performing the task of composing an image, by using different alternative systems, i.e. different simple “mosaics”. Our experimental design is meant to test the research hypotheses formulated above. First, we want to explore the effect of large versus small groups (i.e. user network sizes) on perceived usefulness and perceived ease of use. Moreover, we test whether variation of user network size affects the behavioural intention of acceptance. 4.1. System description: mosaic Each “mosaic” was prepared by cutting a colour image into 16 equal-sized, square paperboard tiles (to be used as cards) within a regular 4 × 4 grid. The task of each user is to recompose the image individually by positioning the cards as mosaic tiles on the grid. The initial 16-card user kit contains eight missing cards and eight multiple cards. In more detail, each kit is composed of six double, one triple and one single card. In consequence, only eight single cards can be directly used for image composition (12/2 + 3/3 + 1/1); the remaining eight cards have to be exchanged. Users are free to exchange cards with other users, but can do so only one at a time. The “user network” is defined by the group of users who are allowed to exchange cards with each other. The missing cards in each kit have to be found and exchanged with other users. 4.2. Task description: reconstructing an image using cards The user has to get the eight missing cards by exchanging the eight multiples with other users in the network. Then, they must position the available cards on the table grid. Five points are scored for each correctly positioned card.3 The objective is to achieve the maximum score (130) with a fully reconstructed image in the minimum time. The task is divided into two sub-tasks: • locating and obtaining all the 16 necessary cards, by exchanging the eight multiple cards with the other users in the network; • composing the image using the 16 cards. As already mentioned, different types of mosaics, i.e. different technologies/systems, have been tested by users for comparative evaluation and final acceptance. A simple feature, i.e. using numbered cards (where card number indicates the correct card position on the table grid), may lower task complexity by making cards easier to distinguish/recognize, to name and find when exchanging,

3 Specifically, five points were attributed to each card (5 × 16 = 80 points), plus a bonus for the completed picture of 50 points, for a total maximum score of 130 points. In preliminary experiments, we also tested a different scoring system attributing four points to each card (4 × 16 = 64 points) plus a 40 points bonus, for a total maximum score of 104 points. The idea (confirmed by the preliminary experiments) was that a typical user would attribute higher usefulness to the system enabling them to better accomplish their task, i.e. to maximize the final score.

5

and to arrange on the table grid. The natural expectation is that a system based on numbered cards with a lower task complexity should be perceived as easier to use. On the other hand, systems with higher scores for each card would improve the final user score, resulting in higher levels of expected perceived usefulness. 4.3. Controlling the user network size As discussed above, the task was designed to be highly influenced by network effects: at start, users were given eight multiple cards. To be able to correctly recompose the original image, they had to exchange their multiple cards with other users. We could control the user network size with two different configurations: a 20-node network versus four independent 5-node networks. In the first configuration all the 20 users were able to exchange cards with no limits in a fully connected network. In the second configuration the 20 users were divided into four independent sub-networks of five users each. In this case, each user could exchange cards only with the four colleagues in their sub-network. Notice that the cards missing in each user kit (randomly assigned) are certainly present within the 20-node network, but after splitting the network, there is no guarantee that a user will find their missing cards in their own sub-network. 4.4. Tuning and initial testing A first round of experiments aimed to tune and validate the experimental design. In a correctly designed experiment, user perceptions of usefulness and ease of use should be in line with what was expected “by design” for the different systems. The tuning and initial testing experiments confirmed our expectations: the results are not shown here, for reasons of space; for more details see Pontiggia and Virili (2005). In this phase also an appropriate time interval for the execution of all the experiments was tested and defined. 5. Data collection Users were students who volunteered at different locations (Bachelor students at the University of Cassino and at the University of Lugano). User groups, each with 20 participants, were formed by random assignment. We measured and collected user perceptions using standard TAM (Davis et al., 1989) and UTAUT (Venkatesh et al., 2003) questionnaires. In particular, in a preliminary set of experiments, we used the full 31-item questionnaire of the “Unified Theory of Acceptance and Use of Technology” (UTAUT) as from Venkatesh et al. (2003, p. 460).4 In the following set of experiments, to simplify and quicken the data collection and to facilitate the user comparison of different systems, we reduced the number of items in the questionnaire, using just the standard TAM constructs “performance expectancy” (four items), “effort expectancy” (four items), “behavioural intention to use the system” (three items), from Venkatesh et al. (2003, Table 16). The questionnaire is shown in Fig. 3. Each item of the standard TAM questionnaire was presented in two adjacent columns, in order to allow an easy comparative evaluation of the same system with different levels of network effects. The measurement scale was the same as that adopted in the original

4 As discussed in Section 2.3, the experimental system/task was designed to be influenced by network effects due to “market size effects”, and not due to “social effects” (i.e. social influence). An exploratory analysis of the data collected for the “social influence” construct with the full 31-item UTAUT questionnaire confirmed this expectation.


ARTICLE IN PRESS

G Model JJIM-907;

No. of Pages 10

6

A. Pontiggia, F. Virili / International Journal of Information Management xxx (2009) xxx–xxx

sources: a Likert scale with numeric values ranging from 1 (strong agreement) to 7 (strong disagreement). To measure the influence of network effects, the same card system was tested by the same 20 users for two consecutive times. The 20 users were put in a room, in four rows of five users, with tables and chairs. Each user was given a user kit with a mosaic and 16 cards. Everybody could move and talk around the room. In the first round, all the 20 users could freely exchange cards with all the other ones, in a fully connected 20-node network. In the second round, each user could exchange cards only with the other four colleagues in the same row. The original user network was then split into four smaller 5-node networks. The two rounds, lasting a few minutes each, were sequential. At the end of the second round, each of the 20 users filled in the questionnaire, introducing comparative evaluations for the “systems” tested in the first round and in the second round. We accepted the risk of introducing minor bias in the experiment by allowing each user to play twice in sequence, because we wanted to explicitly model the acceptance choice as a comparative evaluation between alternative systems for accomplishing a given task. This aspect is novel in TAM research, and its potential implications are described in the discussion. The meaning of the evaluation was briefly explained to the users before distributing the questionnaires. With the first eight question items they had to evaluate usefulness and ease of use of the two alternative “ways” of accomplishing their task (maximizing scores) experienced in the two rounds. In the last three question items they had to formulate an intention of acceptance for each of the two “systems”. We clearly explained in the TAM questionnaire what “system” and “task” meant with reference to the experiment. We did not suggest any specific meaning for “usefulness” and “ease of use”. In the preliminary round of experiments we used three different systems: P1, P2, and F: for details see Pontiggia and Virili (2005). For each system we did three experiments with high network effects (a single 20-node user network). We collected 56 (out of 60 = 20 × 3) valid questionnaires for F and P1, 55 for P2. We then conducted an additional experiment with system F and low network effects (four 5-node user networks), resulting in 56 valid questionnaires. In total we collected 56 × 3 + 55 = 223 valid questionnaires in the preliminary phase. We collected 40 valid questionnaires in the second set of experiments with system F and high network effects, and 39 additional valid cases with low network effects, for a grand total of (223 + 79) = 302 valid cases. 6. Data analysis and results The expected results were that different levels of network effects could influence user perceptions and their behavioural

intention of acceptance. As explained above, each user had to test the same “system” (mosaic) twice: the first time with the possibility of freely exchanging cards within a network of 20 users (high network effects). The second time, the exchange was limited to four separate “sub-networks” of only five users each (low network effects). After two successive rounds of experiments, each user could express in the comparative TAM questionnaire the perceived values of “usefulness” and “ease of use” for the two alternative “systems” and formulate an intention of acceptance for each one. The collected data were then checked and analyzed according to well-established statistical methods based on structural equation models (SEM). In our case the two structural equation models were built and analyzed using the commercial software package AMOS rel. 4 (Arbuckle, 1999). 6.1. Hypothesis testing In order to test for the influence of network effects on technology acceptance we introduced in TAM an external variable accounting for user network size and we checked for its statistical relationships with the TAM constructs, according to our three research hypotheses. We used the entire available dataset collected during several sessions of experiments with three different system types, with two different network sizes (big: 20 users and small: five users), accounting for 303 valid cases. A TAM was computed using this dataset, including “net effect”, a dummy variable set to its upper value (1) in experiments with bigger user networks, and to its lower value (0) with smaller networks. The statistical relationships of “net effects” with the three TAM constructs were estimated and tested, according to H1–H3. Table 1 below shows the main regression weights with the respective statistics. All the traditional TAM relationships are significant and have the correct sign. Though with a low absolute value and a quite high standard error, the path coefficient between “ease of use” and “behav intention” is now also highly significant (p = 0.03). The relationship between “network effects” and “behav intention” has a coefficient close to 0 (−0.05) and nonsignificant (p = 0.4). All the other path coefficients and factor loadings are highly significant (p < 0.01). The relative standard errors are extremely low in absolute value (