International Journal on Media Management
ISSN: 1424-1277 (Print) 1424-1250 (Online) Journal homepage: http://www.tandfonline.com/loi/hijm20
Using Reader Preferences to Optimize News Content: A Method and a Case Study Vamsi K. Kanuri, Esther Thorson & Murali K. Mantrala To cite this article: Vamsi K. Kanuri, Esther Thorson & Murali K. Mantrala (2014) Using Reader Preferences to Optimize News Content: A Method and a Case Study, International Journal on Media Management, 16:2, 55-75, DOI: 10.1080/14241277.2014.943898 To link to this article: http://dx.doi.org/10.1080/14241277.2014.943898
Published online: 21 Aug 2014.
Submit your article to this journal
Article views: 120
View related articles
View Crossmark data
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=hijm20 Download by: [University of Miami]
Date: 17 September 2015, At: 04:39
The International Journal on Media Management, 16:55–75, 2014 Copyright © Institute for Media and Communications Management ISSN: 1424-1277 print/1424-1250 online DOI: 10.1080/14241277.2014.943898
Using Reader Preferences to Optimize News Content: A Method and a Case Study VAMSI K. KANURI Robert J. Trulaske, Sr. College of Business, University of Missouri, USA
Downloaded by [University of Miami] at 04:39 17 September 2015
ESTHER THORSON Missouri School of Journalism, University of Missouri, USA
MURALI K. MANTRALA Robert J. Trulaske, Sr. College of Business, University of Missouri, USA
This article introduces the use of choice-based conjoint analysis to measure the content preferences of newspaper readers. Its importance lies in its role as part of an economic theory of newspaper finance that connects various measures of news quality to circulation and to circulation revenue. In addition, choice-based conjoint provides another way to operationalize “quality” in terms of how much readers like various kinds of content and the relative amounts of that liking. Content preferences were obtained from a large sample of readers of a newspaper of about 100,000 in circulation from the Southeastern United States using the proposed methodology and subsequently, the results were used by the newspaper to modify its content accordingly. Four months after the content changes were made, the publisher of the newspaper reported a significant maintenance of circulation even as print price was increased by 75%. The findings are consistent with the economic model of newspaper demand, and suggests that preferences measured using choicebased conjoint provide impetus for making key management decisions.
Address correspondence to Esther Thorson, Missouri School of Journalism, University of Missouri, Columbia, MO 65211. E-mail:
[email protected] 55
56
V. K. Kanuri et al.
Downloaded by [University of Miami] at 04:39 17 September 2015
BACKGROUND AND THEORETICAL CONTEXT Introduced nearly 25 years ago, the economic model of news demand (Lacy, 1989) provides foundational thinking about the relationship of news quality to circulation. A central tenet of the model is that the number of readers is positively related to product (newspaper) quality, where quality is defined as how well the product serves the wants and needs of the aggregate of users across time (Lacy, 1989, p. 44). A product that satisfies a consumer’s needs and wants better than another provides higher utility than the other. Consumers choose the product that provides them the highest utility and the summation of the choices in favor of a particular product yields its aggregate demand. In particular, a newspaper’s users have needs and wants, expressible as preferences, with respect to its content, for example, its coverage mix of various topics that is, news, politics, sports, etc. The highest quality or utility-maximizing newspaper for a consumer is the one whose content best aligns with consumer preferences for coverage of news topics subject, of course, to their budget constraint. Thus, the Lacy (1989) framework highlights not only the positive connection between news quality and circulation, but also how enhancement of news quality critically depends on careful assessment of news readers’ content mix preferences. Over the years, news publishers and scholars have employed various ways to measure news quality including approaches that involve direct assessment of readers’ content mix preferences. However, the common approach of direct elicitation of preferences suffers from some serious shortcomings. A more sophisticated approach that overcomes these weaknesses is choice-based conjoint (CBC) analysis (e.g., Louviere & Woodworth, 1983). CBC is an advanced preference measurement technique, well-known and widely used for product development in many consumer industries which, however, has hardly been applied in the newspaper publishing world. In this article, our objective is to demonstrate how CBC works and is useful in assessing reader content preferences in a case study at a large, metropolitan newspaper in the Southeastern United States. In this case study, we show how CBC is superior to the common direct elicitation of preferences approach and helped the newspaper determine the news quality-enhancing changes in content based on assessments of readers’ preferences for various news topics and coverage (newspaper space allocations). The rest of this article is organized as follows. In the next section, we briefly review how media economists and social scientists have traditionally attempted to measure news topic preferences and the strengths and weaknesses of these methods. We then introduce and explain the benefits of the CBC analysis approach followed by a description of its application at a southeastern U.S. newspaper concerned with increasing newspaper quality to enhance circulation. A summary of results, limitations, and directions for future research are presented in the concluding section.
Using Reader Preferences to Optimize News Content
57
Literature Review: Alternative Approaches to News Content Preference Measurement
Downloaded by [University of Miami] at 04:39 17 September 2015
DIRECT
ELICITATION OF PREFERENCES
In this approach, typically, respondents are asked to think about and state what newspaper sections they are most likely to read. For example, Leo Bogart examined news content preferences of U.S. citizens in this manner in 1977 and again in 1987 (Bogart, 1989). Replications of these studies in succeeding decades, that is, by Stone and Boudreau (1995) and the Pew Center (Robinson, 2007), showed that these preferences were consistent over the years. The question used by the latter was “thinking about your own reading and use of the newspaper, how often do you read these parts of ?” It should be noted that, using this technique of content measurement, the choices (e.g., local sports, national politics, state government) mixed geographic categories of news (local, state, national, international) with specific topics (e.g., sports, business, entertainment, health, science). A second direct elicitation approach is to evaluate what might be called “attributes of news stories.” These are generally conceived of as adjectives that are related to the quality of the news product, including such dimensions as: accuracy, believability, bias, interestingness, clarity, coherence, comprehensiveness, concise, fairness, and timeliness. Gladney (1996) examined whether readers and editors agreed on the importance of features like these. Sundar (1999) attempted to reduce features like these down into a simple factor structure that he suggested involved only four dimensions: credibility, liking, quality, and representativeness. Aside from readers, professionals like editors and reporters can also be asked directly for their preferences. For example, Stone, Stone, and Trotter (1981) had experts rate 124 newspapers on various dimensions of quality. They then showed that these measures were significantly correlated with circulation (after controlling for number of households, owner occupied households, and number of local radio stations). INDIRECT
ASSESSMENTS OF NEWS QUALITY NOT INVOLVING SELF - REPORTS
A second stream of research uses techniques that indirectly measure content preferences. For example, one could use indicators of “quality,” based on secondary data-based metrics that is, high ratios of staff-written to wire service-based copy, news interpretations and back-grounders to spot news, or non-advertising to advertising content. Unlike the first group of approaches, this method involves no self-report. Lacy and Fico (1991), for example, content-analyzed measures like these in a sample of 114 newspapers. They found that values of these metrics in 1984 predicted 22% of the
58
V. K. Kanuri et al.
Downloaded by [University of Miami] at 04:39 17 September 2015
variance in the newspapers’ 1985 circulation, controlling for population in 1985. Similarly, Lacy and Sohn (1990) looked at the amount of coverage and circulation in the suburbs of Denver and Detroit. Content was coded in terms of news coverage of suburban government, sports, business, police and fire, cultural and social events, and editorials. The total was measured in terms of square inches of suburb-related copy. For metropolitan daily newspapers in Denver and Detroit, the amount of coverage of the suburbs was strongly correlated with their circulation in those suburbs. The amount of suburban local advertising in the metros was also correlated with circulation in those suburbs. FINANCIAL
COMMITMENT - BASED ANALYSES
A third way to index reader preferences is by using Lacy’s (1992) financial commitment model-based analysis. In this model, financial investment in the newsroom leads to higher content quality, greater utility for readers and hence, higher circulation for the newspaper. The model has been validated by Blankenburg (1989) who examined Inland Press data about expenditures on newsroom functions, and found a 0.92 correlation between this measure and circulation. Rosenstiel and Mitchell (2004) also showed with 1987 Inland Press data that total newspaper revenue was related to four financial investment indicators of newsroom quality. Pardue (2004) showed in a case study of the Arkansas Democrat-Gazette that large financial investment in the paper over time resulted in the newspaper having the highest penetration level of any American newspaper (71%). Chen, Thorson, and Lacy (2005) showed with Inland Press financial data for newspapers with less than 85,000 circulation that there was a positive relationship between newsroom investment and circulation revenue per copy, advertising revenue per copy, total revenue per copy, and gross profit per copy. Tang, Sridhar, Thorson, and Mantrala (2011) updated this finding by doing econometric modeling of 12 years of data from a single newspaper and found that investment in the newsroom predicted online advertising revenue. In a more comprehensive analysis that considered both the reader and advertiser sides of one newspaper firm, Sridhar, Mantrala, Naik, and Thorson (2011) found that both investments in advertising and newsroom departments were important predictors of the firm’s revenues, but newsroom investment had a 50% greater impact than that in the advertising department. The above demand model of news and the various ways it has been tested are important because the findings suggest that when a newspaper wants to shore up its circulation, it needs to determine what consumers want, then invest in satisfying those desires, that is investing in its newsroom in a way that provides the desired news content. Thus, determining consumers’
Using Reader Preferences to Optimize News Content
59
preferences is key. Next, we turn to a more detailed explaination of the limitations of direct elicitation approaches in eliciting consumer preferences, and how the proposed CBC approach overcomes those limitations.
Downloaded by [University of Miami] at 04:39 17 September 2015
Limitations of Direct Elicitation Methods There are two specific problems with direct elicitation (or self-stated ratings or rankings) of preferences (or importances) of different attributes, for example, news topic categories. The first is the problem of ambiguous ratings. For example, content on the topic of sports news may be rated as a very important attribute when considering newspapers that differ noticeably in the coverage of sports news, but may be quite unimportant if the consideration set comprises of newspapers that happen to provide the same level of coverage. Similarly, national politics coverage may be regarded as an extremely important topic for any newspaper to cover in general, but may not matter in the reader’s choice if all newspapers provide the same level of coverage. Importance judgments are therefore ambiguous unless discussed in a highly specific context. The second problem is non-discriminating ratings. Overstatement of interest, importance, liking, etc., in various news topics—perhaps as a result of social norms about the value of news topics—is likely in direct elicitation measures. For example, when directly asked about the ideal levels or liking for various content topics in choosing which television channel to view or subscribe to, there is a consistent finding (e.g., Berry, 1983) that respondents’ ratings tend to be high for all topics, for example, the ratings indicate little discrimination between one topic and another. Respondents, it seems, want “more” of everything. Now, if there were no limits on the sizes of newsrooms or the cost of ink, newsprint, and delivery, providing more of everything would be fine. But, of course, that is not practically possible—indeed, in the current world of local print journalism, “less of everything” is the reality.
PRESENT RESEARCH GOALS Against the above backdrop, the present research focuses on the following general problem: In a world of limits, where the reader cannot have everything,1 how may a newspaper quantitatively measure readers’ preferences for coverage of areas, for example, local versus national, within news topic categories, for example, sports, hard news, etc., as well as page coverage of categories, that is, amount of newspaper space allocated to each category, in order to evaluate different options and determine the optimal (readers’ utility-maximizing) content mix and coverage subject to the newspaper’s total page-space and resource constraints? More specifically, we address the
60
V. K. Kanuri et al.
following research questions within the context of our empirical case study involving an actual print newspaper: RQ1:
Downloaded by [University of Miami] at 04:39 17 September 2015
RQ2:
Can the CBC approach be effectively applied to reveal newspaper readers’ preference weights (or “part-worths”), for topic emphasis areas and alternative page-space allocations by topic, that accurately predict their choices from alternative news content bundles? How can the estimated CBC analysis-based preference weights be utilized to quantify the relative importances of news content attributes (including page-space allocations) and determine the reader utilitymaximizing news content mix and space allocation of a fixed number of newspaper pages?
To meaningfully answer these research questions, we need information about reader trade-offs that is, how readers value various levels of each news topic category (e.g., ratios of local to national sports news coverage) and the extent to which they would forego a high level of coverage for one topic (e.g., sports) to achieve a higher level of another (e.g., politics). The method we propose is based on the premise that each reader’s choice behavior is governed by such trade-off values and that, although he/she is unable to articulate them unambiguously or discriminatingly, they may be revealed by his/her discrete choices among alternative news content packages that vary in terms of news topic and coverage characteristics in systematic ways. This method belongs to the category of preference measurement techniques known as conjoint analysis, which has found numerous applications in fields that is, marketing, transportation, environmental, and healthcare management (e.g., Green & Srinivasan, 1978; Johnson, 1974; Ryan & Farrar, 2000), but has been sparsely used to date in journalism and news media studies. In the next sections, we will describe and demonstrate the use and benefits of one advanced form of conjoint analysis technique, namely, CBC analysis.
Proposed Methodology: CBC Analysis In fields outside journalism, CBC analysis has been used to make crucial management decisions that is, product and service design, supplier selection, pricing, and operations capacity (Green, Kreiger, & Wind, 2001). In general, CBC leverages the fact that most decisions in our lives involve trade-offs in making a choice. In the process of making such trade-offs, we subliminally evaluate products by placing weights on various attributes of a product, for example, categories of topics covered in a newspaper. Subsequently,
Using Reader Preferences to Optimize News Content
61
we pick the product that gives us the maximum utility (or is most attractive). CBC thrives on explicating such weights that individuals place on different product attributes.
Downloaded by [University of Miami] at 04:39 17 September 2015
A CASE STUDY In this case study, our first order of business is to answer RQ1, for example, investigate whether CBC analysis can be used to derive newspaper readers’ preference weights for news content topics and space allocations that do a good job of predicting their choices from alternative news content mix offerings. Our guiding hypothesis is that CBC analysis can indeed achieve these results in an environment where readers have to choose between alternative news content bundles that involve trade-offs. We test this hypothesis in the following case study that involves a major daily newspaper in Southeast United States. The newspaper operates in 17 counties with a daily print circulation of over 100,000 and a Sunday print circulation of over 150,000. Its readers included all major demographic segments. The necessary data were collected via an online survey of the newspapers’ readers that included a CBC experiment.
CBC Design The conjoint experiment consisted of a series of choice tasks. Each choice task comprised four alternative newspaper content bundles or configurations. Each configuration had five key attributes that shaped the newspaper including: four news topic categories, namely, hard news, features, sports news, advertising, and a relative space attribute reflecting proportion of total newspaper space allocated (e.g., newspaper column-inches or pages) to each content type. A group of editors iteratively reviewed and modified news categories and their corresponding definitions until they reached consensus. These editors defined hard news as the coverage of events and issues, that is, government and politics, crime, and business; features as stories about people, arts and entertainment, health, and family; sports news as news pertaining to high school, college, pro, and participant sports; and advertising as display ads and wants ads, including auto, real estate, jobs, and services. Finally, an example of the relative space allocation could be 20% of the newspaper’s total content space for hard news, 30% for features, 40% for sports news, and 10% for all advertisements. Next, news categories (attributes) were assumed to have multiple levels. Specifically, the “hard news” attribute had five possible “domain emphasis” levels: business news, education news, state government news, technology news, and crime, and court news. The domain emphasis levels (numbers in parentheses) for the other three content categories were as follows: “features” (4): arts and entertainment news, people features, health news, and home
Downloaded by [University of Miami] at 04:39 17 September 2015
62
V. K. Kanuri et al.
FIGURE 1 Sample choice task in a choice-based conjoint experiment.
or real estate news; “sports news” (4): high school sports news, college sports news, pro sports news, and participatory sports news; “advertising” (6): retail, real estate, automobile, want, jobs, and services advertisements. Although these categorizations might differ at different newspapers, these choices were determined by the managers of the newspaper itself. Finally, the “space attribute” included four different levels, for example, alternative allocations of space to hard news, features, sports news, and advertising: (10, 20, 30, and 40%), (20, 30, 40, and 10%), (30, 40, 10, and 20%), and (40, 10, 20, and 30%). Face validity of the five attributes and corresponding levels associated with each attribute was established by testing our CBC design on 32 toplevel executives of the collaborating newspaper firm. Some changes with respect to the wording of the attributes and levels were suggested, which were subsequently incorporated in the design. A sample CBC choice task is shown in Figure 1.
Step-by-Step Execution of the CBC Study The execution of a CBC study entails three key steps, as described here. STEP 1: DETERMINING
THE NUMBER OF CHOICE TASKS FOR RESPONDENTS
Fundamentally, the number of choice tasks presented to a respondent influences the preference values obtained at the end of the CBC. The total number of possible combinations of levels (content mixes and space allocations) is 1920 (5 × 4 × 4 × 6 × 4). Therefore, the total number of possible choice tasks, taking four or three at a time, is enormous (1920 C 3 or 1920 C 4 ). Because it is impractical to ask the respondent to complete a huge number of choice
63
Using Reader Preferences to Optimize News Content
tasks, we relied on orthogonal design to obtain a feasible set of choice tasks. Orthogonal design ensures that every level of every attribute is randomly repeated the same number of times in the choice experiment. Using the OPTEX macro in SAS, we derived a saturated orthogonal design that consisted of 12 choice tasks with 4 newspaper configurations in each task (Kuhfeld, 1997).
Downloaded by [University of Miami] at 04:39 17 September 2015
STEP 2: LAUNCHING
AN ONLINE SURVEY
Our online survey included three broad sections. The first section consisted of direct elicitation questions where we asked the respondents to rate each attribute level (23 in total) on a 9-point scale ranging from “not at all attractive” to “very attractive.” The second section consisted of the 12 choice tasks obtained from the OPTEX macro2 and finally, the third section consisted of standard demographics questions. The survey was hosted on a remote server location for seven days and the survey link was emailed to 4,000 print subscribers of the collaborating newspaper firm who had been subscribing to the newspaper for more than a year. To maximize response rate, we requested the editor of the newspaper firm to send an email to subscribers detailing the importance of the study along with the survey link.3 At the end of the survey, we received 1,131 completed responses (28.27% response rate). On average, respondents had subscribed to the newspaper for 1.84 years, had a four-year bachelor’s degree, were earning $75,000–$99,000 and were mostly White or Caucasian (see Table 1 for sample descriptives). The newspaper firm confirmed that this respondent profile was largely representative of their print readers. STEP 3: ESTIMATING
UTILITIES :
HIERARCHICAL BAYESIAN (HB)
APPROACH
To estimate the part-worths (values) of every attribute level for each individual from his/her conjoint task data, we utilized the HB approach within the CBC analysis software provided by Sawtooth Software (Sawtooth, 2009). The technical details pertaining to the implementation of HB model are provided in Appendix B.
Findings Regarding Research Question (RQ) 1 In the CBC analysis, the primary outcome is a list of estimated part-worths of levels of each attribute. Part-worths are quantitative assessments of the preference weights placed by respondents on different news category levels when choosing between various newspaper configurations. They are interval data centered at zero within each attribute. Consequently, some levels of an attribute exhibit negative part-worths and others exhibit positive part-worths.
64
V. K. Kanuri et al.
TABLE 1 Respondent Sample Descriptives (N = 1131)
Gender Education Race Income
Minimum
Maximum
Mean
Standard Deviation
1 2 1 1
2 8 8 8
1.45 N/A N/A 5.64
0.49 N/A N/A 1.84
Note. Survey scales are provided in the appendix; N/A: not applicable.
Downloaded by [University of Miami] at 04:39 17 September 2015
TABLE 2 Model Diagnostics: HB CBC Value
Definition
Percentage certain
60.20%
RLH
57.60%
Indicates how much better the solution is than chance (25% in this case), as compared to a “perfect” solution The nth root of the likelihood, where n is the total number of choices made by all respondents in all tasks The average of the current estimate of the variances of part-worths, across respondents The root mean square of all part-worth estimates, across all part-worths and over all respondents
Percent variance
1.37
Parameter RMS
1.32
It is worthwhile to note that negative part-worths do not indicate that the corresponding attribute levels are unwanted. Rather, they simply indicate that these levels are less attractive compared to other levels within an attribute that have positive part-worths. A total of 23 part-worths were estimated using 12 choice tasks for each individual. As shown in Table 2, the estimating HB algorithm converged and values of several goodness-of-fit metric such as percentage certainty of 60.2%, root likelihood of 57.6%, and average variance of 1.37, indicate that the model with estimated part-worths fits the data well (Hauser, 1978). ROBUSTNESS
OF
CBC/HB
RESULTS
To validate the part-worths obtained using CBC/HB, we performed a holdout analysis. That is, we randomly divided the choice tasks into two groups: estimation (or “training”) sample of ten choice tasks, and a validation (or “holdout”) sample of two choice tasks. We then estimated the part-worths using the respondent selections from the “training” choice tasks and used these to predict each individual’s actual choices in the two holdout choice tasks. If the prediction matches the actual choice, we term it a hit; otherwise it’s a miss. As indicated in Table 3, we obtained a hit rate of over 54% for the first holdout task and a hit rate of about 61% for the second hold-out task in run 1 (in contrast to the random hit benchmark rate of 25%). To rule out the possibility that the holdout sample influenced the hit rates, we repeated this
65
Using Reader Preferences to Optimize News Content TABLE 3 Robustness Check: Predictive Analysis
Run 1
Holdout Holdout Holdout Holdout Holdout Holdout
Run 2 Run 3
1 2 1 2 1 2
Hit Rate
Average Hit Rate
54.38% 60.83% 59.33% 63.12% 52.88% 59.91%
57.61% 61.23% 56.40%
Downloaded by [University of Miami] at 04:39 17 September 2015
Note. Compare hit rates to a random probability of 25% in predicting each holdout task.
test multiple times for various combinations of holdout and validation tasks. The hit rates ranged between 50 and 60% (results of three such runs are provided in Table 3). Consequently, our answer to RQ1 is affirmative, that is, we can effectively apply CBC/HB analysis to obtain estimates of newspapers readers’ news topic preferences that are robust and valid.
Answers to Research Question (RQ) 2 Next, Figure 2 summarizes our CBC/HB estimation results. Specifically, among the hard news categories, respondents derived maximum utility from state government news (M = 45.27) followed by business news (M = 3.9), crime and courts news (M = −10.9), technology news (M = −13.84), and education news (M = −24.43). Similarly, among features news Hard News
Features
50.00
20
40.00
45.27
10
15.87
5.12
30.00 10.00
2.98
0
20.00
3.90
–10
0.00
–13.84
–10.00
–10.90
–24.43
–20.00
–30
Arts and People Features Entertainment News
–30.00
Business News
–23.97
–20
Education State and Technology Crime and News Local News News Courts News
Sports News 23.21
24.64
10 0
–15.96
–10 –20 –30
–31.89
–40
High School Sports News
Home and Real Estate News
Advertisements
30 20
Health News
College Sports Pro Sports News Participartory News Sports News
14.00 12.00 10.00 8.00 6.00 4.00 2.00 0.00 –2.00 –4.00 –6.00 –8.00
11.16
–0.43
–0.66
–4.24
–0.10
–5.73
Retail Real Estate Auto mobile Want Jobs Services Advertisements Advertisements Advertisements Advertisements Advertisements Advertisements
FIGURE 2 Part-worths of domain emphasis levels using HB CBC technique.
Downloaded by [University of Miami] at 04:39 17 September 2015
66
V. K. Kanuri et al.
categories, respondents derive maximum utility from arts and entertainment news (M = 15.87) followed by people features (M = 5.12), health news (M = 2.98), and home and real estate news (M = −23.97). Next, among sports news categories, respondents derive maximum utility from pro sports news (M = 24.64) followed by college sports news (M = 23.21), participatory sports news (M = −15.96), and high school sports news (M = −31.89). Among the advertising news categories, retail advertisements provide respondents with maximum utility (M = 11.16) followed by services advertisements (M = −0.10), want advertisements (M = −0.43), real estate advertisements (M = −0.66), and automobile advertisements (M = −5.73). Finally, among the four different space allocation combinations, the 30-40-10-20 split over the four content categories (i.e., hard news, features, sports, and advertisements) was found to be the most appealing among readers (M = 35.70) followed by 40-10-20-30 (M = 30.08), 20-30-40-10 (M = 11.94), and 10-20-30-40 (M = −77.72) splits. Next, we turn to respondent preferences across news categories. The importance of a news category can be determined by understanding how much difference it can make in the total utility of a newspaper. This is indicated by the range (maximum minus minimum estimated values of part-worths among the attribute levels) in the utilities of the attribute’s levels. For example, the range of the hard news category is 69.70 (45.27 – [−24.43]). Multiplying the ratio of each attribute’s utility range to the sum of all attributes’ utility ranges with 100 and comparing the results, we can determine the relative importances of the attributes. As shown in Figure 3, we find that space allocation is the most important attribute (M = 33.76%) followed by hard news (M = 24.28%), sports news (M = 19.37%), features (M = 12.19%), and advertisements (M = 10.39%).4 DIFFERENCES
BETWEEN PREFERENCES DERIVED USING
CBC
AND DIRECT
ELICITATION
Finally, we compare preference estimates derived using CBC and direct elicitation techniques. Using the direct elicitation technique, Figure 4, shows that state government news (M = 7.38), health news (M = 6.88), pro sports news (M = 6.92), and retail advertisements (M = 5.70) were the highest rated content categories in hard news, features, sports news, and advertisements attributes, respectively. These observations are largely consistent with the CBC analysis findings (with the exception of the features attribute where CBC found arts and entertainment news was the most valued). However, the direct elicitation technique does not as effectively discriminate and inform us of which is the most preferred attribute as CBC analysis does. CBC analysis clearly shows that space allocation followed by hard news were the most preferred attributes.
67
Using Reader Preferences to Optimize News Content
Space Allocation of Hard News, Features, Sports News and Advertisements 60.00 40.00 35.70
20.00
30.08
11.94
0.00 –20.00 –40.00
Downloaded by [University of Miami] at 04:39 17 September 2015
–60.00
–77.72
–80.00 –100.00 (10%,20%,30%,40%) (20%,30%,40%,10%) (30%,40%,10%,20%) (40%,10%,20%,30%)
Attribute Importance 0.40 0.35 0.34
0.30 0.25 0.20
0.24 0.19
0.15 0.10
0.12
0.10
0.05 0.00
Hard News
Features
Sports News
Advertisements
Space Allocation
FIGURE 3 Attribute importance using CBC.
SOLVING
A PRESSING MANAGEMENT DECISION PROBLEM
In order to boost its quality and thereby circulation, our collaborating newspaper was planning to add a six-page special section to the Sunday edition. However, the management was unsure which news topic areas to emphasize and the division of the additional space across news categories. Our CBC analysis provided guidance to resolve management’s dilemma. Specifically, based on the estimated part-worths, we could recommend to management that, on average, a reader’s utility would be maximized if he/she gets state government news (M = 45.27) in the hard news category, arts and entertainment news (M = 15.87) in the features category, pro sports (M = 24.64) in the sports news category, and retail advertisements (M = 11.16) in the advertisements category. Additionally, readers, on an average, preferred the 30-40-10-20 split of hard news, features, sports news, and advertisements
68
V. K. Kanuri et al. Hard News
7.5
Features
7
7.38
6.88 6.5
7
6.58
7.03 6.76
6.5
6.33
6 6.44
6.34 6
5.83
5.5 5 Arts and People Features Health News Entertainment News
5.5 Business News
Education State and Technology Crime and News Local News News Courts News
Sports News
Advertisements
8
6
7 6
6.43
Downloaded by [University of Miami] at 04:39 17 September 2015
5 4
Home and Real Estate News
5
6.92 5.56
5.04
5.7 4.6
4
4.77 4.38
4.03
4.13
3
3
2
2 1
1
0 High School Sports News
College Sports Pro Sports News Participartory Sports News News
0 Retail Advertisements
Real Estate Advertisements
Auto mobile Want Advertisements Advertisements
Jobs Advertisements
Services Advertisements
FIGURE 4 Preferences measurement using direct elicitation technique.
(M = 35.70) over other space allocation levels. Therefore, the firm should ideally allocate additional pages to coverage of the four domain emphases levels (i.e., state government news, arts and entertainment, pro sports, and retail advertisements) in a 30-40-10-20 ratio. That is, the firm should allocate 30% of the added pages to state government news, 40% to arts and entertainment news, 10% to pro sports, and 20% to retail advertisements. IMPLEMENTATION
AND RESULTS
Following our recommendations, the newspaper management decided to create a Sunday special section reporting hard news that has a state government news emphasis—specifically, a roundup of state government news, along with relevant editorial commentary. The newspaper also decided to purchase Bloomberg business news wire, which provided each day far more business news (e.g., personal finance, technology, management, and investment news). The remaining space was allocated to cover feature stories including one big story that readers care about or should care about. When these changes were rolled out in fall 2012, they were accompanied by promotion by both the publisher and the editor of the changes and how these were based on readers’ own needs and wants revealed by the survey. Follow-up evaluations from the newspaper’s management team indicated that circulation and overall revenue were enhanced. Moreover, at the same time the content changes were rolled out, the newspaper significantly
Using Reader Preferences to Optimize News Content
69
Downloaded by [University of Miami] at 04:39 17 September 2015
increased its print and online subscription prices. A consulting firm working with the newspaper had indicated that the newspaper could expect a 12–15% decrease in renewed subscriptions with the price hike. What the newspaper actually experienced, however, was only a 3% decrease. The editor acknowledged that aligning content preferences with content delivery helped make readers less price sensitive, which led to an increase in retention rates. In addition, the editor also reported (and continues to report) a great deal of positive reader feedback. Lastly, the publisher decided to invest half of the revenue gains back into the newsroom. The newspaper is now hiring six additional journalists to further expand the depth and quantity in stories concerning significant community issues.
CONCLUSION Extant media research suggests an increase in newsroom investments as a strategy to achieve superior circulation performance (e.g., Bogart, 2004; Lacy, 1989; Stone et al., 1981) and to cope with the economic crisis facing the newspaper industry (Tang et al., 2011). However, scholars such as St. Cyr, Lacy, and Guzman-Ortega (2005, pg. 58) have cautioned: “editors and reporters must use the increased investment in ways that improve the paper from the perspective of readers if circulation is to increase” (emphasis added). In particular, newsroom investments should follow readers’ preferences for coverage of different topics. The present research demonstrates how the technique of CBC analysis can help to systematically and meaningfully obtain quantitative assessments of readers’ preferences over different news content areas and subsequently be utilized to guide newspaper space allocation and investment decisions.
Implications for Media Scholars and Practitioners The key contribution of this study to the journalism and communications research field is the introduction and demonstration of the benefits of the CBC analysis technique to measure readers’ preferences. Managerially, this study shows how the CBC approach can benefit newspapers interested in making customer-centric “product design” decisions—and guide their related newsroom staffing decisions.5 For example, if a CBC analysis indicates that “sports” news is more preferred than “features,” the firm can redistribute its journalists’ effort accordingly and if need be, the firm can hire additional journalists to specialize in the kinds of stories that readers indicate they most want. Besides validating the findings presented in this article, future research can also apply CBC analysis to elicit readers’ preferences that can help print and hybrid (print plus digital) media firms, for example, magazines as well
70
V. K. Kanuri et al.
as newspapers, solve some important and timely problems, for example, bundling or unbundling of content, or configuring and pricing single- and multi-format (e.g., print plus digital) subscription packages. In closing, as articulated by our two research questions, the purpose of this research was to describe and demonstrate in a case study the value of CBC analysis in improving newspapers’ content mix offerings. We hope our work stimulates more study of the potential of this tool in helping to solve critical problems confronting the newspaper industry today.
Downloaded by [University of Miami] at 04:39 17 September 2015
ACKNOWLEDGMENTS We would like to thank the editors and the review team for their insightful comments and suggestions that improved the quality of the article.
FUNDING We thank the Reynolds Journalism Institute for its financial support and the management of a newspaper firm in southeastern United States, which wishes to remain anonymous, for allowing us access to their print subscribers for this study.
NOTES 1. In this research, we focus on the print newspaper with physical space constraints. Such space constraints are removed in the digital environment. It would be worthwhile for future research to review and address problems pertinent to digital news content. 2. We conducted a pre-test on 25 frequent readers of the newspaper to determine whether the conjoint task was cognitively taxing. Almost all reported that while it took them some time to comprehend the first task, the subsequent tasks were easy to complete. Further, to ensure no systematic bias in responses, we estimated part-worths for multiple randomly selected samples of 100 respondents and used these to predict their choices in holdout analyses (described later in text). We repeated this process 3 times with each sample of 100 respondents. Subsequently, we compared the predicted with actual choices in the holdout tasks. The different samples yielded approximately the same hit rates, indicating there is no systematic bias in the responses. Finally, upon examining response times, we found respondents took the most time to answer the first four conjoint questions but subsequent tasks took much less time indicating quick learning. Hence, cognitive overload was not an issue in the conjoint exercise. 3. To minimize the possibility of any misunderstanding of the various attributes and levels used in the survey, we (a) presented clear definitions when we had respondents’ maximum attention; (b) required subjects to complete a warm up task right after presenting them with the news category descriptions; and (c) provided detailed instructions, right before the conjoint task, of what was expected of the respondents on two pages. 4. Note that the results outlined here are specific to our sponsor firm. Other newspapers are advised to explore these values for their own readers. 5. While we focus on newspapers in this article, the CBC approach can be similarly applied to determining content mix and page allocation preferences of newsmagazine readers as well as content mix and time allocation preferences of that is, television news hour program viewers.
Using Reader Preferences to Optimize News Content
71
Downloaded by [University of Miami] at 04:39 17 September 2015
REFERENCES Berry, C. (1983). Learning from television news: A critique of research. Journal of Broadcasting, 27(4), 359–370. Blankenburg, W. B. (1989). Newspaper scale & newspaper expenditures. Newspaper Research Journal, 10(2), 97–103. Bogart, L. (1989). Press and public: Who reads what, when, where, and why in American newspapers. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Bogart, L. (2004). Reflections on content quality in newspapers. Newspaper Research Journal, 25(1), 54–65. Chen, R., Thorson E., & Lacy, S. (2005). The impact of newsroom investment on newspaper revenues and profits: Small and medium newspapers, 1998–2002. Journalism & Mass Communication Quarterly, 82(3), 516–532. Gladney, G. A. (1996). How editors and readers rank and rate the importance of eighteen traditional standards of newspaper excellence. Journalism & Mass Communication Quarterly, 73(2), 319–331. Green, P. E., Krieger, A. M., & Wind, Y. (2001). Thirty years of conjoint analysis: Reflections and prospects. Interfaces, 31(3), S56–S73. Green, P. E., & Srinivasan, V. (1978). Conjoint analysis in consumer behavior: Issues and outlook. Journal of Consumer Research, 5, 103–123. Hauser, J. R. (1978). Testing and accuracy, usefulness, and significance of probabilistic choice models: An information-theoretic approach. Operations Research, 26(May–June), 406–421. Johnson, R. M. (1974). Trade-off analysis of consumer values. Journal of Marketing Research, 11, 121–217. Kuhfeld, W. F. (1997). Efficient experimental designs using computerized searches. Sawtooth research paper series. Retrieved January 31, 2013, from http://www. sawtoothsoftware.com/download/techpap/effdesgn.pdf Lacy, S. (1989). A model of demand for news: Impact of competition on newspaper content. Journalism Quarterly, 66(spring), 40–48. Lacy, S. (1992). The financial commitment approach to news media competition. Journal of Media Economics, 5(2), 5–21. Lacy, S., & Sohn, A. B. (1990). Correlations of newspaper content with circulation in the suburbs: A case study. Journalism & Mass Communication Quarterly, 67(4), 785–793. Lacy, S., & Fico, F. (1991). The link between newspaper content quality and circulation. Newspaper Research Journal, 12(2), 46–57. Louviere, J. J., & Woodworth, G. (1983). Design and analysis of simulated consumer choice or allocation experiments: An approach based on aggregate data. Journal of Marketing, 20(4), 350–367. Pardue, M. J. (2004). Quality key to highest city zone penetration in U.S. Newspaper Research Journal, 25(4), 13–25. Robinson, M. J. (2007). Two decades of American news preferences. Report from Pew Center for People and the Press. Retrieved January 31, 2013, from http:// pewresearch.org/assets/pdf/NewsInterest1986-2007Part2.pdf Rosenstiel, T., & Mitchell, A. (2004). The impact of investing in newsroom resources. Newspaper Research Journal, 25(1), 84–97.
Downloaded by [University of Miami] at 04:39 17 September 2015
72
V. K. Kanuri et al.
Ryan, M., & Farrar, S. (2000). Using conjoint analysis to elicit preferences for health care. British Medical Journal, 320(7248), 1530–1533. Sawtooth Software Inc. (2009). The CBC/HB system for Hierarchical Bayes estimation. 2009 Sawtooth Software Conference Proceedings, Delray, FL, March 23–27, 2009. Sridhar, S., Mantrala, M. K., Naik, P. A., & Thorson, E. (2011). Dynamic marketing budgeting for platform firms: Theory, evidence, and application. Journal of Marketing Research, 48(6), 929–943. St. Cyr C., Lacy, S., & Guzman-Ortega, S. (2005). Circulation increases follow investments in newsrooms. Newspaper Research Journal, 26(4), 50–60. Stone, G. C., & Boudreau, T. (1995). Comparison of reader content preferences. Newspaper Research Journal, 16(4), 13–28. Stone, G. C., Stone, D., & Trotter, E. P. (1981). Newspaper quality’s relation to circulation. Newspaper Research Journal, 2(3), 16–24. Sundar, S. S. (1999). Exploring receivers’ criteria for perception of print and online news. Journalism & Mass Communication Quarterly, 76(2), 373–386. Tang, Y., Sridhar, S., Thorson, E., Mantrala, M. K. (2011). The bricks that build the clicks: Newsroom investments and newspaper online performance. International Journal on Media Management, 13(2), 107–128.
Using Reader Preferences to Optimize News Content
73
APPENDIX A: SURVEY SCALES Gender: Please indicate your gender. 1. Male 2. Female
Downloaded by [University of Miami] at 04:39 17 September 2015
Education: What is the highest level of education you have had the opportunity to complete? Please select one response. 1. 2. 3. 4. 5. 6. 7. 8.
Grade school Attended high school, but did not graduate Graduated high school (includes G.E.D.) Trade school Some college (includes Associate degree) Completed a four-year degree (Bachelors) Some post-graduate work Completed post-graduate degree (MBA, MA, Ph.D., Masters, etc.)
Race: What is your race or ethnic background? Please select one response. 1. 2. 3. 4. 5. 6. 7. 8.
Hispanic/Latino Asian or Pacific Islander White, Caucasian Black, African American American Indian, Eskimo, or Alaska native Two or more races Another race I am not comfortable answering
Income: Which of the following categories best represents your total household income before taxes in this past year? Please select one response. 1. 2. 3. 4. 5. 6. 7. 8.
Under $25,000 $25,000–$34,999 $35,000–$49,000 $50,000–$74,999 $75,000–$99,999 $100,000–$149,999 $150,000 or more I am not comfortable answering
Direct elicitation questions: Please rate how attractive each of the content items under each news category is to you on a scale of 1 (not at all attractive) to 9 (very attractive):
74
V. K. Kanuri et al.
1
2
3
4
5
6
7
8
9
Hard news Business news Education news State government news Technology news Crime and courts news
Downloaded by [University of Miami] at 04:39 17 September 2015
Features Arts and entertainment news People features Health news Home or real estate news Sports news High school sports news College sports news Pro sports news Participatory sports news Advertisements Retail advertisements Real estate advertisements Automobile advertisements Want advertisements Jobs advertisements Services advertisements
TECHNICAL APPENDIX B: ESTIMATION OF PART-WORTHS USING HB Conjoint techniques that existed prior to the introduction of HB suffered from an inherent disadvantage with respect to individual utility estimates. Typically, every respondent in a CBC experiment answers only a few choice
Using Reader Preferences to Optimize News Content
75
Downloaded by [University of Miami] at 04:39 17 September 2015
tasks. This limits the ability to estimate the preferences of other product combinations from the relatively small amount of information. CBC/HB overcomes this disadvantage by comparing how different an individual’s utilities are from those of the population. The algorithm estimates average utilities for the entire population first, and then uses individual data to determine how each respondent differs from sample averages. Higher variance in the sample will force the algorithm to rely on the individual’s data, whereas lower variance in the sample will let the algorithm rely on group data. An HB model relies on Bayesian updating of probabilities. It comprises of two levels. The higher level consists of an individuals’ part-worths defined by a multivariate normal distribution given as: βi = Normal (α, D),
(1)
where βi is a vector of part-worths for the ith individual, α is the vector of means of the distribution of individuals’ part-worths, and D is a matrix of variances and co-variances of the distribution of part-worths across individuals. Next, at the lower level, the model assumes that the probability of an individual choosing a particular alternative given his/her part-worths is characterized by a multinomial logit model. Mathematically, this can be represented as: exp(xk , βi ) , pk = j exp xj , βi
(2)
where pk is the probability of the respondent choosing kth newspaper configuration in a choice task and xk is a vector of values describing the jth configuration in a choice task. The algorithm estimates β, α, and D in an interative fashion. Each iteration is characterized by three steps: (1) Generate α from a normal distribution using the current estimates of β and D for a respondent i. (2) Generate a new D from the inverse Whishart distribution using the current values of β and α. (3) Using a Metropolis Hastings Algorithm, derive β based on the current estimates of α and D. Using Gibbs sampling CBC/HB re-estimates a set of parameters conditionally, given current values of other two sets.
