Desirability Bias With Indirect Questioning: Is the Cure Worse than the. Disease? Robert J. Fisher, University of Southem California. Gerard J. Teliis, University of ...
Removing Soda! Desirability Bias With Indirect Questioning: Is the Cure Worse than the Disease? Robert J. Fisher, University of Southem California Gerard J. Teliis, University of Southem Califomia ABSTRACT
American consumer (an average r=.299 across various types of respondents). The result suggests that self-reponed attitudes represent only a small proportion of individuals' predictions about the attitudes of others.1 Accordingly, indirect questions may not provide accurate estimates of respondents' attitudes and behaviors. The present study seeks to assess the validity of indirect questions by evaluating the extent to which they contain information about the self. We do so by comparing the performance of direct and indirect measuresof the imponance of social approval in consumption. We focus on this variable because social infiuence is pervasive in consumer behavior and yet consumers tend to understate ite importance on self-reporte (Fisher 1993).
Researchers in marketing and other social sciences frequentiy employ indirect questions to reduce social desirability bias. Social desirability bias is the systematic error in self-report measures that results from the desire of respondente to project a favorable image to the researcher. Recent evidence indicates that indirect questions can reduce social desirability bias for sensitive questions (see Fisher 1993), but in some situations indirect questions may introduce other forms of error. In particular, they may introduce attitude-irrelevant variance as respondente try to make accurate predictions about the third pany specified in the indirect question. Thus indirect questions may be unbiased but invalid. The present research tries to determine to what extent indirect questions reduce social desirability bias, while also appropriately measuring the true scores. We carr>^ out the study in the context of a socially-sensitive variable: the imponance of social approval in consumption.
HYPOTHESES The Capacity of Indirect Questions to Reduce Social Desirability Bias Underiying projective tests such as indirect questioning is the concept of classical projection.^ Classical projection is said to occur when people are unaware of or deny possessing a negative characteristic that is attributed to another (Sherwood 1981). As noted by Holmes (1981), two criteria are necessary for classical projection to occur: (1) the individual must possess the trait that he or she Is projecting, and (2) the individual must be unaware of or deny his or her possession of the trait. Related to the first requirementof classical projection, social approval is an important factor in the purchase or consumption of many kinds of producte (e.g., Childers and Rao 1992; Fisher and Price 1992; Park and Lessig 1977). In particular, the approval of others is important for products that are conspicuous (i.e., visibly purchased or consumed) and exclusive (Bearden and Etzel 1977). These "expressive" products have social implications because they visibly communicate consumers' values and group associations.
Researchers frequentiy use indirect questions to mitigate the effects of social desirability bias when surveying sensitive topics. Indirect questioning is a projective technique that asks subjects or respondente to answer structured questions from the perspective of another person or group (Anderson 1978; Calder and Bumkrant 1977; Haire 1950; Robenson and Joselyn 1974). The typical indirect question asks respondents to make predictions about how a similar other would think or act in a panicular situation (e.g.. Alpert 1971; Bearden and Etzel 1982; Brinberg and Plimpton 1986; Park and Lessig 1977; Steele 1964). Indirect questions are thought to reduce the distonion of ''private" opinions that are revealed to the researcher. The technique reste on the assumption that respondents project their unconscious biases into ambiguous response situations and reveal their true feelingsabout socially-sensitive issues(Campbell 1950; Holmes 1968; Kassarjian 1974; Sherwood 1981). Indirect questioning allows respondente to project their attitudes into the response situation by asking them to report on the "nature of the extemal worid" rather than themselves (Westfall, Boyd, and Campbell 1957, p. 138). Respondents feel that they are giving information about situations based on fact rather than opinion and so they respond behind "a facade of impersonality" (Simon and Simon 1975, p. 586).
In termsof the second requirement, research on cultural values suggests that the oven pursui t of social approval through consumption is undesirable in American society. When asked directiy, Americans consistently assign a low rank to social recognition (the respect and admiration of others) and a high rank to independence as cultural values. Of 18 terminal values, social recognition was ranked 17th in 1971, and 18th in 1974 and 1981, while independence was ranked 3rd in all three studies (Rokeach 1979, p. 133; Rokeach and Ball-Rokeach 1989, p. 778). Similariy, less than 10% ofa sampleofnoninstitutionalized American adults selected "being well-respected" or "a sense of belonging" as their most important value from the List of Values (LOV) (Kahle 1986). It appears that Americans do not value attitudes and behaviors that are explicitiy motivated by social recognition and approval.
Although recent research suggests that indirect questioning can reduce the effects of social desirability bias on self-report measures (see Fisher 1993), the extent to which predictions about others represent information about the self is less clear. For example, aconsumer's prediction about the alcohol consumption of a typical other may be uncontaminated by social desirability bias, but it may contain Iittie information about the self if the individual making the prediction is a non-drinker. It seems quite likely that as tiie degree of similarity between the respondent and the typical otherdecreases,the level ofirrelevantinformationin the respondent's answers increases. In the present example, non-drinking respondente are less likely to project themselves into an indirect response situation in which the typical other is a heavy consumer of alcohol.
Because the infiuence of others on consumers' consumption decisions is important yet culturally undesirable, it should lead respondents to project their attitudes when answering indirect questions. In contrast, the tendency to present oneself in the best possible light should lead respondents to underreport the impor-
Funher, some evidence suggests that predictions about others do not accurately reflect what respondents would have reponed had they been asked directiy. For example. Hoch (1988) found only a weak association between various groups of respondents' own attitudes and their predictions about the attitudes of the typical
^He then compared these responses to the actual responses of typical American consumers (generated in a national sample) to make inferences about the factors contributing to prediction accuracy. ^Holmes (1968) refers to this as similarity projection. '
Advances in Consumer Research Volume 25, © 1998
564 / Removing Social Desirability Bias With Indirect Questioning: Is the Cure Worse than the Disease? tance of social approval when answering direct questions. As a consequence, estimates of the importance of social approval for consumption should be higher when measured with indirect compared to direct questions. Formally, HI: Because social desirability bias affects direct but not indirect questions: (a) the importance of social approval is higher for direct than for indirect questions, and (b) tendency to respond in a socialty desirable manner corretates with direct but not indirect questions. Estimated True Score Components of Direct and Indirect Questions Ctassicat measurement theory decomposes a measure>• into a component defined as the tme score t, systematic error s, and random error e (Cote and Buckley 1988; Lx)rd and Novick 1968). The tme score is the tatent variabte defined by theory, the systematic error is due to trait-irrelevant factors such as method variance, and the error term represents inconsistency over successive, parattet measurements. The random error is uncorrelated with either the tme score or the error terms of other measures. Algebraically the equation is represented as follows:
Both indirect and direct questions have true score and error term components, but only direct questions are assumed to contain the systematic effects of social desirability bias. Further, because both types of questions are designed to reflect the attitudes and behaviors of the respondent, both direct and indirect items contain some portion of the tme score for a construct. In the present context, both direct and indirect measures of the importance of social approval contain a true score component. Moreover, variance that is common to both the direct and indirect items provides an estimate of the underlying true score. If indirect questioning provides a better estimate than direct questioningofthe respondent's true feelingson a socially-sensitive subject, then it should be refiected in the association between the estimated tme score and each type of question. Specifically, the estimated true score should be more strongly associated with indirect compared to direct questioning because indirect measures do not contain sociat desirabitity bias. Further, because the estimated true score contains onty what is common to both measures, it should not contain sociat desirability bias. Formally, H2: (a) indirect questioning is more strongly associated with the estimated true score of the importance of social approvat than direct questioning, and (b) No association exists btetween the estimated true score of ihe importance of sociat approvat and sociat desirability bias. An attemative to using indirect questioning to mitigate social desirabitity bias is to remove the systematic error from direct measures. One approach to doing this is to undertake a regression of the direct measure on the tendency to respond in a sociatty desirabte manner and save the residuat. Interestingty, the residual (i.e., the corrected direct score) is an estimate of the respondent's score on the variable with social desirability bias removed. The residual is therefore an estimate ofthe true score plus random error. Compared to the corrected direct score, indirect questioning should be more highly correlated with the estimated true score to the extent that it is a superior measure of the respondent's true feelings. Formally,
H3: Indirect questioning is more strongly associ îted with the estimated tme score than direct questioning :orrected for social desirability bias.
METHOD A pretest was undertaken to identify expressif e products that would be widety accessibte within the student popi tation. Twetve undergraduate students were asked to rate the importance of sociat approvat in the brand decisions for a tist of 25 proc ucts thought to be retevant to this population. The five products \/ith the highest mean scores on a 0 to 6 scale anchored by "Purch ises Infiuenced Only by Self and "Purchases Influenced Only b / Others" were selected for inclusion in the main study. The five pre duct categories were athletic shoes, spring break vacation destinât on, music, hair styte, and cologne\perfume. After completion cf the pretest, a convenience sample of 75 mate and female undergiaduate students was used to test the main hypotheses. Measurement Importance of Social Approval. Respondent» were asked to provide evaluations of the importance of sociat api^roval for themsetves (direct question) and the typicat student (in lirect question) in the purchase of each of the five products. Tht fottowing two question stems were used: It's very important TOME that others approve ot my purchase of each of the foltowing five products. It's very important TO THE TYPICAL STUDEf TthiH others approve of his or her purchase of each of the following five products. Items were measured on seven-point "Strongt / Disagree" to "Strongty Agree" Likert scales. Items for the five expressive products were summed to form single indicators (i.e., one direct measure and one indirect) ofthe importance of sociat infiuence for expressive products. Questions were alternated within the questionnaire to avoid order bias. Social Desirability Bias. The tendency to respond ina sociatty desirabte manner was measured with Reynotd's (1982) short form of the Marlowe-Crowne social desirability scati:. The scale is comprised of 13 culturally approved behaviors (fi *'e worded positively and eight worded negatively) that have a loiv probability of occurrence. The scale is administered in a true or lalse format, and a high score on the scale indicates the subject's tendency to present bim'\herself in a socially favorable manner. Scab characteristics are consistent with prior studies (e.g., Reynolds 19 Í2). A summary of scale characteristics is presented in Table 1. Corrected Direct Score. The corrected direcl score is the sum of the five directly-worded questions with social desirability bias removed. Wecalculatedthecorrecteddirectscore )y regressing the direct measure of social approval on the tendency to respond in a socially desirable manner. The residual was then used as a measure ofthe importanceofsocial approval corrected fors x:ial desirability bias.
Estimated True Score. A factor analysis w£s run in which a single factor solution was specified using the five direct items and five indirect items. The factor score provides an estimate of each respondent's underlying true score of the impt nance of social approval in the purchase of expressive products. The factor loadings that result from the an; lysis provide an indication of the strength of the association between each item and
Advances in Consumer Research (Volume 25) / 565
TABLE 1 Scale Characteristics
#of items
Actual scale range
Scale mean
Std. dev.
Alpha
SELF
5
Oto 26
11.2
6.3
.78
TS
5
Oto 30
19.0
6.5
.90
4.5
2.8
.69
Measure Importance of social approval
Tendency to respond in a socially-desirable manner SDR
13
0 to 11
n = 75 Key: SELF = predictions about self; TS = predictions about typical student; SDR = the tendency to respond in a socially desirable manner as measured by the Marlowe-Crowne short form.
TABLE 2 Estimated True Score Factor Loadings Factor Loading
Item Direct Athletic shoes
.42
Spring break destination
.41
Hair style
.44
Music CD
.46
Cologne\perfume
.56
Indirect Athletic shoes
.76
Spring break destination
.78
Hair style
.81
Music CD
.79
Co!ogne\perfume
.79
n = 75 NOTES: Eigenvalue = 4.16 with 42% of the variance explained.
the factor. As revealed in Table 2, the indirect items are more
RESULTS
strongly associated with the factor (i.e., the estimated true score) than the direct items. The result suggests that compared to the direct items, a greater ponion of the variance in the indirect items is common with the true score (see Dillon and Goldstein 1984).
Hypothesis Tests. HI stated that social desirability biaswould have a significant effect on directly worded items but not indirectly worded items. This hypothesis received suppon with significant differences in the means for the two types of questions and the
566 / Removing Social Desirability Bias WUh Indirect Questioning: Is the Cure Worse than the Disease?
TABLE 3 Correlation Matrix DQ
CDQ
10
ETS
Direct questions (DQ)
1.0
Corrected direct questions (CDQ)
.93*
1.0
Indirect questions (10)
.33*
.33*
1.0
Estimated true score (ETS)
.61*
.65'
.93*
1.0
Tendency to respond in a sociallydesirable manner (SDR)
-.38'
.00
-.08
.00
SDR
1.0
n = 75
extent to which they were correlated with the Marlowe-Crowne short form. An examination of mean scores for the importance of social approval in the purchase of expressive products reveals that subjects evaluated social approval to be more important for the typical student than for themselves. Based on a dependent t-test, a significant difference existed between direct questions related to the self and indirect questions related to the typical student (M(self)=11.2, M(typicai student)=19.0, i=9.16,/;
