engagement with online media

ENGAGEMENT WITH ONLINE MEDIA Rachel Davis Mersey Edward C. Malthouse Bobby J. Calder Northwestern University

ABSTRACT Engagement has emerged as an important concept for news organizations. Yet lack of agreement within the industry and the academy on the definition of engagement has left news companies vulnerable to the definitions dictated by advertisers, who focus on brand placement not news content. This paper relies on quantitative and qualitative methodologies to define online engagement as a collection of experiences, illustrates that there are actually two important types of engagement (personal and social-interactive engagement) for media companies, and demonstrates their predictive validity by showing both are associated with readership. KEY WORDS: audiences, users, engagement, experience, digital media

In 1970 the Beatles’ last album was released, floppy disks were invented, construction of the World Trade Center was completed, and for the first time since mass circulation launched the medium, total U.S. daily newspaper circulation fell below the total number of U.S. households (Bogart, 1989). Forty years later, readers’ continued abandonment of the print newspaper, combined with falling advertising revenue, forced the doors closed at the Rocky Mountain News. Some traditional print newspapers that remain open are wounded, perhaps mortally so. The newsroom staff at the Los Angeles Times was sliced by half before its parent company, Tribune, declared bankruptcy in December 2008; 45 percent of the editorial staff at the (NJ) Star-Ledger took buyouts in October 2008; and the Detroit Free Press and Detroit News were the first major metropolitan newspapers in the country to trim home delivery to three days a week, a move that started in 2009. Newsrooms are debating how to build and retain readership, and publishers are trying to figure out how to monetize that readership. Copyright © 2010 Journal of Media Business Studies Rachel Davis Mersey, Edward C. Malthouse, and Bobby J. Calder, “Engagement with Online Media,” 7(2):39-56 (2010).

40

Journal of Media Business Studies

These are important journalistic questions, and we believe questions of engagement. The fact is that journalists are increasingly interested in issues of reader engagement. Advertisers, of course, also want engagement (Grusell, 2007; Perez-Latre, 2007). As evidence, the Advertising Research Foundation (ARF, 2006) provides this definition: “Media engagement is turning on a prospect to a brand idea enhanced by surrounding context.” However, this advertising-based definition demotes media content, even that of a well-respected newspaper such as the New York Times, to “surrounding context.” This is a proposition that should not be tolerated by journalism practitioners or scholars, but the failure of these groups to assert a clear definition of engagement has left them vulnerable to the impositions of others such as the ARF. Practitioners and scholars do not agree on a definition of engagement. If we cannot define engagement, we cannot measure it. Further, without systematic and comparable measures, media organizations cannot track relevant short- and long-term trends, nor can they show advertisers evidence that their audiences are engaged. In this article we will define engagement as the collective experiences that readers or viewer have with a media brand.1 An experience is a specific set of beliefs that consumers have about how some media brand fits into their lives (Calder & Malthouse, 2008a, 2008b). As we will show, people can have many different experiences with a brand, including utilitarian, hedonic and social-psychological ones. For example, some media brands play a utilitarian role in the lives of their consumers by providing useful advice, tips and ideas. Other media are more hedonic, transporting consumers to another time, place or state of mind by helping them relax and escape the pressures of daily life. We contend that understanding which experiences an organization intends to create is the essential first step in developing and positioning a successful media brand (Calder, 2010; Malthouse & Peck, 2010). It is a clear articulation the intended experiences that will focus the creation of content and communicate the nature of engagement to potential advertisers (Calder, et al., 2009). As Picard (2009) points out, standing on the hallowed ground of equating journalism to “moral or even sacred terms” and not realizing that “journalism must innovate and create new means of gathering, processing, and distributing information so it provides content and services that readers, listeners, and viewers cannot receive elsewhere” is a fatal mistake. Understanding engagement and experiences enables the journalistic organization to create such value for their consumers and differentiate their content from others. The purpose of this research is therefore to define engagement, explore the range of experiences that consumers have with on-line media, propose measures of engagement 1 We shall use the term media brand instead of channel-centered terms such as newspapers, magazines, TV programs, or Web sites. Increasingly, media brands such as ESPN and the New York Times distribute content over multiple channels.

Mersey, Malthouse, and Calder—Online Engagement

41

and experiences, and test the predictive validity of our measures. Applications of these concepts are discussed further in the concluding section.

Media Engagement and Experiences To understand how a reader engages with a publication, we must begin by understanding how a media brand fits into its readers’ lives. This premise makes the task of defining engagement difficult because the relationships between publications and their readers are complex and multidimensional. Consider a brand that distributes content through a newspaper and Web site. Of course, readers may be engaged with it because they feel it connects them with the community or they enjoy participating in its discussion groups. They may also feel engaged with it because it serves as filter for all the news and information that is “out there”—readers want to “keep up” with certain topics and believe that this newspaper will provide them with the information they need to be up-to-date. Alternatively, readers may view reading this newspaper as a way to relax and unwind after a long day. They may also engage with it because it makes them more interesting by providing them with topics for conversation. Reading certain media can even reinforce their readers’ identities by affirming their membership in a certain group, e.g., the Wall Street Journal and the business class. The fact is that when people are engaged with a media brand they are having experiences with or because of that product. The media brand is playing integral roles in that person’s life. Therefore, we will simply refer to each these reasons for being engaged as experiences. People who read a publication to relax and unwind have a timeout experience. Those who read it for things to talk about are having a social-facilitation experience. In general, an experience is a set of thoughts and beliefs that readers have about how a publication fits into their lives. Engagement is thus a qualitative, holistic feeling of readers, which we cannot measure directly; readers know about the presence of engagement via thoughts and beliefs, which we can measure more readily. The empirical section below will discuss additional experiences and show how they can be measured as latent variables using factor analysis. Beliefs such as “This newspaper helps me relax” and “I like to kick back and wind down this newspaper” will become questionnaire items that load on a (first-order) social-facilitation experience factor. We can now define engagement as the collection of experiences that readers have with a publication (see Figure 1). The engagement of readers of the New York Times is characterized by the experiences they have with it. Experiences, in turn, are the thoughts and beliefs the readers have about how it fits into their lives. Under this definition, different publications need not deliver the same experiences to be engaging. Some could be engaging because they provide high levels of a

42

Journal of Media Business Studies

Figure 1: Experiences, Engagement, and their Consequences

utilitarian experience—providing useful advice and tips—while others could be engaging because they are intrinsically enjoyable. Experiences are not necessarily mutually exclusive and some content could engender high levels of multiple experiences. It is essential to realize that there is more than one path to engagement and that the different paths are realized by offering different experiences, enabling a brand to differentiate itself from others. Consider, for example, the travel section of www.nytimes.com. Some articles could engage readers by creating a utilitarian experience, where the reader believes the articles give useful advice about what to do and where to stay at certain destinations. Other articles could be engaging because they offer intrinsic enjoyment. A narrative story about some travel adventure could relax readers and “transport” them to a different place and not provide utilitarian “how-to” detail. To the extent that readers have experiences that lead to engagement, they should have higher levels of readership. As depicted in Figure 1, we therefore hypothesize that engagement causes readership: Hypothesis: Engagement is positively associated with readership. To construct measures of engagement as we have conceptualized it, we will need to draw a sample of experiences from its construct domain. Before drawing such a sample, we must first establish the boundaries of this domain. For this, we turn to the uses and gratifications (U&G) framework, which has been used by mass communication researchers to suggest that individuals define their needs and control of the mediaseeking process in an attempt to gratify those needs. From its inception, the U&G framework has relied on the active-audience assumption; that is, people are deliberate and discriminating in choosing media. Therefore an important theoretical contribution made by U&G research came in

Mersey, Malthouse, and Calder—Online Engagement

43

the form of the qualification that it “proceeds from the assumption that the social and psychological attributes of individuals and groups shape their use of mass media rather than vice versa” (Katz & Foulkes, 1962). Despite detractors who argue that audiences are not sufficiently selfaware to understand why they use media, Stone (1987) suggests that U&G does have face validity: “People will not attend to messages that have no perceived interest value for them. They will choose among media content offering those items they deem valuable, even if that value is only momentary enjoyment” (p. 129). Further, by our definition of engagement, experiences are exactly what people do understand about their behavior and feelings. U&G has a long history that evolved from a media-first methodology, focused on asserting media’s functions, identifying audience motives by listing functions served by the content or medium and then categorizing them (Berelson, 1949; Cantril & Allport, 1935; Herzog, 1941; Lazarsfeld & Stanton, 1942; Lazarsfeld & Stanton, 1944; Waples, Berelson, & Bradshaw, 1940; Wolf & Fiske, 1949). Then, in the 1960s, scholars directed research toward “[identifying] and [operationalizing] many social and psychological variables that were presumed to be the precursors of different patterns of consumption of gratifications” (Ruggiero, 2000, p. 15). U&G research was bolstered by Katz, Haas, and Gurevitch’s (1973) attempt to synthesize a comprehensive typology of the audience uses, and social and psychological gratifications. However, one of the most widely cited typologies of U&G came years later from McQuail (1983): Information—finding out about relevant events and conditions in immediate surroundings, society and the world; seeking advice on practical matters or opinion and decision choices; satisfying curiosity and general interest; learning, self-education; gaining a sense of security through knowledge. Personal Identity—finding reinforcement for personal values; finding models of behaviour; identifying with valued others (in the media); gaining insight into one’s self. Integration and Social Interaction—gaining insight into the circumstances of others; social empathy; identifying with others and gaining a sense of belonging; finding a basis for conversation and social interaction; having a substitute for real-life companionship; helping to carry out social roles; enabling one to connect with family, friends and society. Entertainment—escaping, or being diverted, from problems; relaxing; getting intrinsic cultural or aesthetic enjoyment; filling time; emotional release; sexual arousal (pp. 82-83). McQuail’s assertion, of course, preceded the Internet, and there is a general sense that online U&G are in some ways different than those of traditional print products. These differences are highlighted by the

44

Journal of Media Business Studies

concept of interactivity. According to Ruggiero (2000), “interactivity significantly strengthens the core [U&G] notion of the active user” (p. 15). Further, it highly differentiates online media from its predecessors. We shall adopt McQuail’s typology, plus Ruggiero’s suggestion that there are additional interactive components, as the construct domain, which we will call the extended U&G framework. The different roles specified by this framework are experiences. In the language of measurement models, experiences are first-order constructs while engagement is a second-order construct. We shall use the term experience whenever we refer to a specific set of consumer beliefs about a vehicle such as utilitarian or intrinsic enjoyment, and the term engagement whenever we refer to the overall experiences of a vehicle. Therefore, the aim of this research is to examine the following: Research question: How do the various experiences under the extended U&G framework factor? Our definition of engagement as a collection of experiences subsumes other definitions that have been advanced. Representing one perspective, digital editor of the Telegraph Media Group (TMG) Edward Roussel, is focused on action. Roussel (2008) explains, advising newspapers to “create functionality [online] that encourages readers to share eyewitness accounts of breaking news, rate services such as restaurants and hotels, and get into discussions and debates” (p. 10). The value of the contribution approach is echoed by others including John Byrne (2008), executive editor of BusinessWeek and editor-in-chief of BusinessWeek.com, and Bob Giles (2006), Nieman Foundation curator and former editor and publisher of The Detroit News. We shall call this the “participating and socializing experience.” Even more widely used among journalists and journalism scholars is an alternative approach to engagement in terms of community connection or participation (Mersey, 2009a and 2009b; Cuillier, 2008; Kwak, Shah, & Holbert, 2004; Kwak, Williams, Wang, & Lee, 2005; McLeod, Kosicki, & McLeod, 2002; McLeod, Scheufele, & Moy, 1999; Moy & Scheufele, 2000; Moy, Scheufele, & Holbert, 1999; Norris, 1996; Rothenbuhler, Mullen, DeLaurell, & Ryu, 1996; Shah, 1998; Shah, McLeod, & Yoon, 2001; Shah, Schmierbach, Hawkins, Espino, & Donovan, 2002; Stamm, Emig, & Hesse, 1997). We will call this the “civic experience.” This perspective, brought to the forefront by Putnam (2000), prizes attention to the connection individuals have with one another and their geographic communities. Putnam’s concerns are commonly cited by researchers of traditional news products but this citizen model is also being used by researchers of news online. According to Singer (2006), “… despite high hopes for the potential of an ‘electronic republic’ to empower citizens, a decade of Web use has led many to conclude that a ‘virtual sphere’ is just a dream. … observers emphasize that although the medium offers a useful place for political discussion and may encourage

Mersey, Malthouse, and Calder—Online Engagement

45

greater pluralism, technology alone cannot foster democracy by engendering political interest or engagement” (p. 266). The “participating and socializing” and “civic” experiences are certainly ways of being engaged, and we regard them as manifestations of a second-order engagement construct. But as U&G scholars have known for years, there are many other utilitarian, hedonic and socialpsychological motives. Our conceptualization of engagement as a secondorder construct enables measurement and allows for additional forms of engagement, which is essential given the heterogeneous nature of existing media brands.

METHOD AND RESULTS Our methodology consists of several steps, which are discussed in the subsections below. Selecting Experience Scales The first step was to select measurement scales for a set of online experiences that capture aspects of the extended U&G framework and are indicators of engagement. Ideally these scales should produce an acceptable fit in a measurement model and have good psychometric properties such as acceptable reliability and convergent and discriminant validity. We use the Calder-Malthouse (CM) set of media experiences because they provide measurement scales for a broad range of experiences (Calder & Malthouse, 2004; Calder & Malthouse, 2005; Malthouse, Calder, & Tamhane, 2007). We briefly summarize the CM methodology and argue that these experiences span the engagement domain. CM conducted more than 400 hour-long, in-depth interviews with consumers about the role that specific Web sites, newspapers, magazines and TV news programs play in their lives. They analyzed the transcripts for common themes and created hundreds of Likert-scale items. The items were included on surveys of media users. Exploratory factor analysis identified 22 online experiences, 44 newspaper experiences, 39 magazine experiences and 12 TV news experiences. The values of coefficient alpha suggested that most of the scales were reliable (some had weak reliability because of too few items). None of the original CM studies estimated confirmatory factor analysis (CFA) models. For this confirmatory study, we had to select eight of the 22 CM experiences, due to constraints on survey length and respondent fatigue. Requiring experience measures to have an acceptable fit in a measurement model also limits the number of experiences.2 In reviewing 2 Hatcher (1994, p. 260) recommends using “a maximum of 20-30 indicator variables” in a measurement model and we will use 37.

46

Journal of Media Business Studies

the original 22 “experiences,” we decided that some did not fit in the construct domain because they describe the site itself rather than how the site fits into the consumer’s life. For example, one of the “experiences” was about credibility of the site and another was about the site being easy to use. Several experiences were also dropped because they were specifically about the advertising on the site. The eight experiences and their items are displayed in Table 1. They were selected with a stratified sampling procedure from the remaining experiences so that there would be at least one from each of McQuail’s four U&G categories (the strata) that characterize more traditional media, and others, such “community” and “participation & socializing,” that are particularly relevant to online media. We tried to avoid picking too many experiences from any single McQuail U&G category. For example, two of the remaining experiences fit under McQuail’s information category: “makes me smarter,” which is about keeping people up-to-date on issues that concern them, and “utilitarian,” which is more about advice and “how-to” information. Using the flip of a coin we decided to include utilitarian. Likewise, the original CM experiences “intrinsic enjoyment,” “entertains and absorbs me” and “a way to fill my time” all fit under McQuail’s entertainment category and we selected the first (at random). The “social facilitation” experience was selected as a representative of McQuail’s integration and social interaction category. The “self-esteem and civic mindedness” experience represents the personal identity category. Table 1: Question Wording and Parameter Estimates from Confirmatory Factor Analysis Measurement Model Stand. Loading

Experience

Item

Stimulation & inspiration (α=0.88)

It inspires me in my own life.

0.85

This site makes me think of things in new ways.

0.84

This site stimulates my thinking about lots of different topics. This site makes me a more interesting person.

0.78

Social facilitation (α=0.88)

0.79

Some stories on this site touch me deep down.

0.71

I bring up things I have seen on this site in conversations with many other people. This site often gives me something to talk about.

0.85

I use things from this site in discussions or arguments with people I know.

0.81

0.85

Mersey, Malthouse, and Calder—Online Engagement

47

Table 1 continued Experience

Item

Temporal (α=0.90)

It's part of my routine. This is one of the sites I always go to anytime I am surfing the Web. I use it as a big part of getting my news for the day. It helps me to get my day started in the morning. Using this site makes me feel like a better citizen. Using this site makes a difference in my life. This site reflects my values. It makes me more a part of my community. I am a better person for using this site. It's a treat for me. Going to this site improves my mood, makes me happier. I like to kick back and wind down with it. I like to go to this site when I am eating or taking a break. While I am on this site, I don't think about other sites I might go to. This site helps me make good purchase decisions. You learn how to improve yourself from this site. This site provides information that helps me make important decisions. This site helps me better manage my money. I give advice and tips to people I know based on things I've read on this site. I do quite a bit of socializing on this site. I contribute to the conversation on this site. I often feel guilty about the amount of time I spend on this site socializing. I should probably cut back on the amount of time I spend on this site socializing. I'm as interested in input from other users as I am in the regular content on this site. A big reason I like this site is what I get from other users. This site does a good job of getting its visitors to contribute or provide feedback. I'd like to meet other people who regularly visit this site. I've gotten interested in things I otherwise wouldn't have because of others on this site. Overall, the visitors to this site are pretty knowledgeable about the topics it covers so you can learn from them.

Self-esteem & civic-mindedness (α=0.91)

Intrinsic enjoyment (α=0.87)

Utilitarian (α=0.88)

Participation socializing (α=0.88)

Community (α=0.88)

&

Stand. Loading 0.85 0.83 0.84 0.80 0.86 0.85 0.76 0.75 0.88 0.83 0.85 0.82 0.65 0.71 0.81 0.83 0.76 0.81 0.74 0.86 0.77 0.82 0.78 0.84 0.85 0.59 0.80 0.73 0.66

We claim that these eight experiences are representative of the engagement construct domain. Of course, other sets could also represent the domain, but our approach is entirely consistent with our objective of developing indicators of engagement. For example, we would not expect our engagement measure to change in a substantive way if we had used “makes me smarter” rather than “utilitarian” from the information category.

48

Journal of Media Business Studies

Survey Methodology The second step was to sample users of media Web sites. Eleven online media Web sites were used in the present confirmatory study.3 These sites represent a convenience sample, but include broad range of different types of media sites including those with national reputations (e.g., Reuters.com and Washingtonpost.com), special interest sites (e.g., about.com) and local sites (e.g., king5.com). The target population, identified with a screening question, was people who used the site at least once a month. Subjects were recruited from the visitors on the particular sites, who were redirected to an online survey. The sample sizes for the 11 sites ranged from n = 35 to n = 1,289, with a median sample size of n=542 and a total sample size of n=5,942. Respondents were asked about their usage and experiences with the particular site. Measurement Models for Experiences and Engagement The third step was to develop measures of online engagement as follows: (1) estimate a confirmatory factor analysis measurement model to study the psychometric properties of our experience measures; (2) develop second-order engagement factors by applying exploratory factor analysis to the eight experiences; and (3) fit a second-order CFA model. We estimated a measurement model for the experiences, allowing each possible pair of experiences to be correlated. Fit statistics are provided in Table 2. Question wording, factor loadings, and the values of coefficient alpha are provided in Table 1. There were 37 items used to measure the 8 experiences. All eight scales are highly reliable, with coefficient alpha ranging from 0.87 to 0.91. GFI, CFI, and NNFI all exceed 0.90, indicating an acceptable fit. Convergent and discriminant validity were also supported.4

3 The sites are About.com, Washingtonpost.com, PalmBeachPost.com, Reuters.com, DallasNews.com, Projo.com, King5.com, AZFamily.com, WFAA.com, KHOU.com, and PE.com. 4 Convergent validity was assessed by comparing the t-values of the factor loadings,

computed as the ratio of the loading to the standard error of the item. The minimum t-value was 48.2>2, providing evidence in support of the convergent validity of the indicators. We assess discriminant validity with the chi-square difference test. For each of the 28 pairs of experiences we estimated a separate measurement model identical to the one shown in Table 2, except that the covariance between the pair is fixed at 1. The chi-square statistics between the models were computed, and range from 4,132 to 12,073. The differences have chi-square distributions with 1df, and are very highly significant, supporting discriminant validity.

 

Mersey, Malthouse, and Calder—Online Engagement

49

Table 2: Summary of Confirmatory Factor Analysis Modela Measurement 102 0.9155 0.9482 0.9426 0.0472

Parameters GFI CFI NNFI RMSEA a n=5942 with 37 items

Second-order 87 0.9029 0.9392 0.9343 0.0505

Pearson correlations between the experiences are provided in Table 3 and follow a pattern that suggests the possibility of second-order factors. The first six experiences are moderately correlated with each other, with values between 0.42 and 0.72. Participation and socializing (7) is substantially less correlated with the first six, but moderately correlated with the community experience (8). Community is somewhat less correlated with the first six experiences. This correlation structure suggests that there is a higher-order factor structure generating the data. Table 3: Correlation Matrixa Pearson correlation Experience

1

2

3

4

5

6

7

1

Stimulation & inspiration

2

Social facilitation

.54

3

Temporal

.51

.53

4

.65

.57

.47

5

Self-esteem & mindedness Intrinsic enjoyment

.66

.51

.64

6

Utilitarian

.60

.53

.41

.72

.57

7

Participation & socializing

.24

.19

.19

.29

.33

.35

8

Community

.48

.41

.34

.53

.52

.57

.56

civic-

8

9

10

.63

Engagement 9

Personal engagement

.83

.77

.77

.82

.84

.76

.32

.58

10

Interactive engagement

.58

.49

.43

.69

.65

.74

.77

.90

.73

.26

.28

.48

.22

.33

.22

.15

.23

.39

Readership .27

a

All correlations are significantly different from 0 at the 0.0001 level

Therefore the next step in developing the measurement model is to identify the second-order engagement factors using both exploratory and confirmatory factor analysis. We performed an exploratory factor analysis with a varimax rotation on the first-order experiences and found two eigenvalues greater than 1. The rotated factor loadings are provided in Table 4 and show two interpretable factors, hereafter called personal

50

Journal of Media Business Studies

engagement and social-interactive engagement. The first six experiences from the correlation matrix have the largest loadings on personal engagement, although community also has a cross-loading greater than .3. Participation and socializing as well as community have the largest loadings on social-interactive engagement, but several other experiences have sizable cross-loadings. The utilitarian experience likely cross-loads on social-interactive engagement because much of the advice and tips could be coming from the community of users rather than from content created by employees of the site itself. Self-esteem likely cross-loads because contributing to an online conversation could contribute to one’s self-esteem. Table 4: Exploratory Factor Analysis Loadings of First-Order Experiencesa Experience

Factor 1 Personal engagement

Factor 2 Social-interactive

Social facilitation

0.768

Temporal

0.753

Stimulation & inspiration

0.744

Self-Esteem & civic Mindedness Intrinsic enjoyment

0.710

0.375

0.701

0.366

Utilitarian

0.612

0.472

Participation & socializing Community

0.881 0.361

0.755

a

Loadings less than 0.3 were omitted

We then estimated a second-order confirmatory factor model to test whether it is plausible that the personal and social-interactive engagement latent variables generate the observed correlation structure between the experiences and items. Personal and social-interactive engagement will be used in the subsequent analyses of readership. Fit statistics are shown in Table 1, with CFI, GFI, and NNFI all greater than 0.9 suggesting a good fit. Figure 1 shows the parameter estimates of the second-order factor structure.5

5 The loadings for the 37 items were very similar to those from the measurement model above and have been omitted. Note that the second-order factor model finds a significant correlation between the two engagement latent variables. In the analyses that follow, we estimate the two engagement factors using a weighted average of the experiences, with the factor loadings as weights.

 

Mersey, Malthouse, and Calder—Online Engagement

51

Figure 2: Second-Order Engagement Factor Structure  

Stimulation & Inspiration Social Facilitation .85 .73 Personal Engagement

.42

.68 .79

Temporal

.78 .69

Self-Esteem & Civic-Mindedness

.29

Intrinsic Enjoyment

.14 Social-Interactive Engagement

.25 .76 .74

Utilitarian Participation & Socializing Community

Personal engagement is manifested in experiences that are similar to those that people have with print newspapers and magazines. For example, experience items such as “This site makes me think of things in new ways” or “This site often gives me something to talk about” could also apply to print. Social-interactive engagement, however, is more specific to Web sites. Items such as “I do quite a bit of socializing on this site” and “I contribute to the conversation on this site” would not characterize print. While social-interactive engagement is more closely associated with the Web, aspects of it can be found for other media. For example, “A big reason I like this site is what I get from other users” could also apply to the letters-to-the-editor page of a daily. The utilitarian experience is a manifestation of both forms of engagement. Service oriented Web sites (e.g., bhg.com-Better Homes and Gardens) will have a prominent utilitarian component as will user-contributed advice sites (e.g., Yahoo!Answers or chowhound.com). In sum, the measurement model and values of coefficient alpha have shown that the eight experiences have been measured reliably and support the convergent and discriminant validity of the scales. The second-order analysis shows two engagement factors, personal engagement and social-interactive engagement. Personal engagement is manifested in experiences that have counterparts in magazines and newspapers while social-interactive engagement is more specific to Web sites. As reflected the loadings in Figure 2, with personal engagement users seek stimulation and inspiration from the site, they want to use the site to facilitate their interactions with other people, they feel the site affirms their self-worth, they get a sense of intrinsic enjoyment in using

52

Journal of Media Business Studies

the site itself, they feel it is useful for achieving goals, and they value input from other users. With social-interactive engagement, users experience some of the same things in terms of intrinsic enjoyment, utilitarian worth, and valuing the input from the larger community of users but in a way that links to a sense of participating with others and socializing on the site. Thus social-interactive engagement is motivated both intrinsically and extrinsically but in this case it is the social relevance of these, rather than their personal or individual quality, that is associated with the larger engagement experience. And it is the valuing of input from the community and sense of participating with others and socializing that gives social-interactive engagement its dominant character. Validation of Engagement We now test the predictive validity of our measures by studying whether engagement is associated with readership. We measure readership with the Site Usage Measure (SUM) and relate it to the two types of engagement with two methods. SUM is an average of the number of visits and the total time spent on a Web site. First, Table 3 shows that personal and social-interactive engagement have positive, highly significant correlations with readership. Higher levels of either type of engagement are associated with higher levels of readership. Simple correlations, however, have several limitations, which will be addressed in the next analysis: (1) they do not allow for different correlations for different sites; and (2) they do not account for the correlation between the two types of engagement. The second analysis uses two separate hierarchical linear models (HLM) to predict RBS from the two types of engagement (Table 5), including a random intercept and random slopes for engagement at the site level. The slope for both personal and social-interactive engagement is positive and highly significant. None of the random effects are significant indicating that we do not have evidence to conclude that the relationships vary across sites. We conclude that our engagement measures predict readership. Table 5: Regression of Readership on Engagement with a Hierarchical Linear Model Effect Intercept Personal Intercept Interactive

Slope 0.6394 0.7378 1.5588 0.5504

Fixed effects Std. error 0.08087 0.02473 0.08097 0.02655

P-value