Judgment and Decision Making, Vol. 8, No. 5, September 2013, pp. 561–572
Choice blindness in financial decision making Owen McLaughlin ∗
Jason Somerville †
Abstract Choice Blindness is an experimental paradigm that examines the interplay between individuals’ preferences, decisions, and expectations by manipulating the relationship between intention and choice. This paper expands upon the existing Choice Blindness framework by investigating the presence of the effect in an economically significant decision context, specifically that of pension choice. In addition, it investigates a number of secondary factors hypothesized to modulate Choice Blindness, including reaction time, risk preference, and decision complexity, as well as analysing the verbal reports of non-detecting participants. The experiment was administered to 100 participants of mixed age and educational attainment. The principal finding was that no more than 37.2% of manipulated trials were detected over all conditions, a result consistent with previous Choice Blindness research. Analysis of secondary factors found that reaction time, financial sophistication and decision complexity were significant predictors of Choice Blindness detection, while content analysis of non-detecting participant responses found that 20% implied significant preference changes and 62% adhered to initial preferences. Implications of the Choice Blindness effect in the context of behavioural economics are discussed, and an agenda for further investigation of the paradigm in this context is outlined. Keywords: choice blindness, investment, pensions.
1 Introduction The Choice Blindness paradigm is a novel experimental method which uses misdirection or sleight-of-hand to manipulate the relationship between intention and choice, by presenting decision-makers with outcomes they did not actually choose and eliciting a response (Johansson, Hall and Sikström, 2008). The principle findings in the Choice Blindness literature are that many participants do not notice when their stated preferences are manipulated, even when factors such as attention, task engagement and social effects are taken into account (Johansson, Hall and Chater 2011). First demonstrated in a study examining preferences for female faces (Johansson, Hall, Sikström and Olsson 2005), Choice Blindness experiments have since been extended into various other domains, including naturalistic consumer choices (Hall et al. 2010), attitude formation and moral sentiment (Hall, Johansson and Strandberg 2012), polling and political preferences (Hall et al. 2013), symptomatology and psychiatric self-diagnoses (Merckelbach, Jelicic and Pieters 2010), and to other The authors gratefully acknowledge the support and guidance of Professor Liam Delaney and the Geary Institute, University College Dublin, and thank Róisín White, Clare Delargy and Áine Ní Choisdealbha for their insightful comments and assistance. Copyright: © 2013. The authors license this article under the terms of the Creative Commons Attribution 3.0 License. ∗ Geary Institute, University College Dublin, Ireland. Email:
[email protected] † Department of Economics, Trinity College Dublin, Ireland. Email:
[email protected]
sensory modalities (Steenfeldt-Kristensen and Thornton, 2013). Choice Blindness studies to date have consistently found that only 20-35% of participants detect the manipulation of their preferences (Johansson, Hall and Chater, 2011). The significance of Choice Blindness can be extended beyond the simple question of whether participants detect a manipulation. In their original study, Johansson et al. (2005) demonstrated not only that many participants failed to notice manipulation of their preferences but also that they could be induced to defend these altered preferences when asked to motivate their choice. These “confabulated” explanations were not qualitatively different from explanations of unaltered preferences, and in some cases, participants would confabulate explanations contradicting their original choices—mentioning details entirely absent from the images initially chosen. The purpose of the present study was to devise a Choice Blindness experiment involving a significant and realistically framed economic decision. In addition, we examine the factors that underlie and modulate it. This is an area which the existing literature has had some difficulty addressing (Johansson, Hall, and Sikström, 2008). Therefore the design was also intended to isolate a variety of factors intuited to underlie the Choice Blindness effect. An economically significant class of decision was examined through a design featuring decisions about personal finance, specifically the choice of a pension. Pensions are a very familiar topic in behavioural economics (e.g., Madrian and Shea 2001, Thaler & Benartzi 2004,
561
Judgment and Decision Making, Vol. 8, No. 5, September 2013
Tapia & Yermo 2007) in part because they represent a class of decision with important ramifications for both individual welfare and the future stability of an economy. Moreover it is an area in which traditional economic theorising has failed to adequately anticipate the financial decisions of agents, as evidenced by the abundance of chronically underfunded pension schemes (Thaler & Benartzi, 2004). Pension decisions are also relatively accessible (if not familiar) because most individuals will make at least one such decision in their lifetime. Pensions also represented a topic of particular local relevance both before and during the time the study was conducted, as Ireland was then preparing to implement wide-ranging pension reforms and introduce an autoenrollment program (National Pensions Framework, 2010). Furthermore, a pension paradigm requires participants to construct a portfolio of options, a task of greater complexity than the binary choices presented in most previous Choice Blindness studies. This has the potential to elicit richer and more nuanced responses from participants, and this added granularity is of crucial importance in evaluating confabulatory responses, and so distinguishing instances of significant preference change from responses motivated by inattention to specific choice characteristics or lack of engagement. The design was intended to replicate the conditions, materials and appearance of actual personal finance decisions with maximum accuracy, so as to elicit as great a degree of participant engagement as possible within the confines of a hypothetical scenario. The presence of a Choice Blindness effect consistent with previous findings was thus the study’s primary hypothesis. Additionally, the design built on previous Choice Blindness experiments by including a variety of auxiliary measures, enabling the investigation of a range of factors with the potential to influence the prevalence of Choice Blindness, represented by participant detection rates. Measures were included to evaluate participant reaction time, memory and task recall, risk preferences, and financial sophistication. The design also included cheater-detection priming for some participants. Reaction time was recorded as the time taken (in seconds) for a participant to complete each trial. Reaction time was intended as a proxy measure for the degree of care and attention participants applied to the task. It was hypothesized that participants who spent longer on the task would be more likely to detect manipulations. A brief memory task was administered to participants at the end of the experiment, just prior to debriefing. The task involved answering five simple questions about the task, of increasing sophistication and “difficulty”, and recorded as a score out of five. This instrument was in-
Choice Blindness and Financial Decision-Making
562
tended primarily to address the theory that Choice Blindness represents a memory effect, in which participants appear to reverse their preferences because they do not recall their initial choices clearly (Johansson, Hall and Sikström 2008). A positive relationship between detection rate and recall accuracy would be expected in this case. The experiment also included a brief risk profile (adapted from Holt & Laury, 2002) as well as a selfreport measure of participants’ preferences for risk. We hypothesized that participants with a lower risk preference would be more cautious and reflective, and therefore more likely to detect a manipulation. Additionally, findings from an initial pilot study suggested that familiarity with the subject matter would provide a strong influence on detection rates, in accordance with previous findings on biases in financial decision-making (e.g., Feng and Seasholes, 2005). The design therefore included a brief financial sophistication instrument (Lusardi & Mitchell, 2006), which has proven to be a robust proxy for general knowledge of financial issues such as personal finance. It was hypothesized that greater financial knowledge would increase detection rates. Half of the participants were assigned to a condition in which a subtle cheater detection prime was inserted into the experiment’s introductory literature, and it was hypothesized that this group would be more likely to detect manipulations.
2 Method 2.1 Participants A total of 100 participants (53 female) were recruited for the study, ranging in age from 18 to 60 years (mean = 24.69, SD = 7.33). 41 were undergraduate students, with the remainder being graduate students and professionals ranging from basic degree to Ph.D. level of educational attainment. Participants were recruited through various forms of public advertising on university campuses. The study was advertised as an investigation into the factors affecting decisions about personal finance, and all participants were initially naive as to the experiment’s true purpose.
2.2 Materials The primary materials consisted of two pages outlining the investment options (which numbered between ten and twelve) offered by one of six fictional pension companies. One page depicted the options graphically in a tiered format, from low risk to high risk. The other briefly explained each option in relatively simple language consistent with the vocabulary observed in actual financial
Judgment and Decision Making, Vol. 8, No. 5, September 2013
prospectuses. The overall presentation was modelled closely on the explanatory materials provided by actual pension funds in the Republic of Ireland to new and prospective clients. An example of the experimental stimuli can be found in Appendix A. Additional materials consisted of an information and briefing sheet, a short six-question financial literacy survey (adapted from Lusardi and Mitchell, 2006), a basic demographic questionnaire, and a one-question risk preference instrument (adapted from Holt and Laury, 2002). Participants in the cheater-detection condition were given a slightly different briefing sheet, in which the paragraph regarding auto-enrolment in Ireland in 2014 contained the following additional sentence as a cheater-detecting prime: However, concerns have been raised in some quarters about the security and fairness of an auto-enrolment system for pensions, particularly the possibility that they may afford employers greater opportunity to cheat their employees by changing their preferences or opting them out of the system without their consent. During the experiment proper, participants indicated their choices using a simple response sheet instructing them to choose between three and six options and allocate a percentage of their hypothetical investment to each one. All experimental materials are reproduced in Appendix B. The experimenters themselves made use of a twomonitor computer system, with one display facing the participant. Participant responses were entered by an experimenter into a specialized interface which was designed to facilitate data entry, presentation and manipulation as quickly as possible, so as to minimize both memory degradation effects and participant suspicion.
2.3 Procedure Following the general pattern of previous studies (e.g. Johansson et al. 2005, Hall et al. 2010) the experiment was administered by two experimenters, one who interacted directly with the participant and presented the physical materials and another responsible for notation and computerized data entry. At the beginning of the experiment, participants were shown an information sheet explaining the ostensible purpose of the study (“to investigate some of the different factors which affect people’s choice of a pension plan”) and providing some simple background information on what pensions are, how they work, and how proposed auto-enrolment schemes in the future make them relevant to everyday life. Each participant was also allocated to a cheater-detection (CD) or non-CD condition, and given
Choice Blindness and Financial Decision-Making
563
the explanation with or without the CD priming paragraph as appropriate. The brief demographic questionnaire, risk preference profile, and financial sophistication survey were then administered before the actual experiment began. The experimenters also recorded each participant’s responses on a hand-held recording device, for the purpose of qualitative analysis. The experiment featured five trials, each featuring the materials for one of the six simulated companies. The first, third and fifth of the five trials were standard: The participants chose a portfolio, which was then presented to them on-screen for comment, just as they had chosen it. The second and fourth trials were manipulated— participants were shown a portfolio in which one item differed from the one they actually chosen at the beginning of the trial. The second trial was a manipulation of the similar condition, and involved the second experimenter replacing one of their selections, chosen at random, with another fund on the list of similar type and risk return profile, and differing primarily in investment type and name. The fourth trial represented the dissimilar condition: here the experimenter randomly replaced one of the participant’s choices with another fund from the list of significantly different character and risk profile— that is, an option significantly more (or less) risky than the original choice. The order in which the trials themselves were presented was not varied, as the design assumes (following the findings of previous Choice Blindness experiments, e.g., Johansson et al,. 2005; Hall et al. 2010) that the detection threshold for the similar condition is lower than for the dissimilar condition. Therefore it is implicitly assumed that any participant who detects a manipulation in the similar condition (i.e., the one more difficult to detect) would also detect a dissimilar manipulation, and the experiment was concluded once any detection was observed. Proceeding with this assumption was also deemed more efficient as it maximized the number of trials per participant. On each trial, participants were given the pension fund materials for one of the six fictional companies, chosen at random by the experimenter. Participants were instructed to allocate their hypothetical pension contribution between these fund options, selecting a portfolio of between three and six different funds, and attributing a percentage to each one. Participants were encouraged to take as much time as they needed to make a decision, and indicated their choices by filling in a response sheet. On completion of each portfolio, the participant returned this sheet to the experimenter, and the explanatory materials were also withdrawn. The first experimenter then explained that the selected portfolio would be displayed on the monitor facing the participant, who would then be asked to explain the se-
Judgment and Decision Making, Vol. 8, No. 5, September 2013
lection of each fund in the portfolio, while the second experimenter quickly entered her chosen portfolio into the computer. At this point in the second and fourth trials, the experimenter also made the appropriate similar or dissimilar manipulation. The result, with or without manipulation, was then displayed in a simple graphical format on the monitor facing the participant. The average time between participants handing over a response sheet and their portfolio being displayed back to them was approximately 5 seconds. In order to control for possible primacy or recency effects, all selected portfolios (manipulated or not) were presented to participants in a randomized order. Participants were then invited to comment on their portfolio, and were free to do so at their leisure. The experimenter’s responses were limited to simple verbal prompting of the participant for more detail when required, and participants were not specifically asked if the pension portfolio shown was the same one they chose. During the explanation of fund choices, if a participant pointed out or otherwise expressed notice of a mismatch, the experiment ended and the participant was debriefed at the end of that trial. Following the established paradigm, such a result was recorded as concurrent detection. Debriefing involved explaining the actual purpose of the experiment and the deception/manipulation which it involved. Immediately before debriefing, all participants, detected or not, were asked a series of 5 memory questions designed to evaluate their recollection of and attention to the task. If the participant finished explaining fund choices without any such evidence of detection, the trial was concluded and the next trial began, featuring a different set of stimuli (i.e. a different company’s brochure). Participants who proceeded through all trials without detecting a mismatch were told that the experiment was almost over and would conclude after a few brief questions. Consistent with previous studies, participants who had not detected any mismatch were then asked a series of increasingly specific questions designed to elicit how confident they were of their choices and whether they had noticed anything suspicious or unusual during the experiment. The questions were as follows: 1 Did you notice anything unusual during the experiment? 2 Did you suspect anything was amiss with your final portfolios? 3 Did you suspect any trickery on behalf of the experimenters? 4 Did you notice that we switched your portfolio options? Any affirmative answer to the these question was recorded as retrospective detection. Finally, participants who showed no retrospective detection were explicitly
Choice Blindness and Financial Decision-Making
564
asked whether they believed they would notice any hypothetical manipulation of their choices. Once these questions had been asked, all participants were fully debriefed.
3 Results The experiment produced 100 sets of individual data across all experimental conditions, in addition to demographic and instrumental information. The data for two participants who withdrew before completing the experiment were discarded, leaving a final total of 98. The principle descriptive data in this study were the number of detections (both concurrent and retrospective) registered in both the Similar and Dissimilar manipulated conditions. In the Similar condition, only 24.5% of responses feature detection of portfolio manipulation. In the Dissimilar condition, 33.8% of responses detect the manipulation. However, if we assume that the 24 participants who detected the Similar mismatch would have also detected the Dissimilar mismatch, this figure increases to 50%. We can thus view a 33.8% detection rate as a lower bound estimate in the dissimilar condition and 50% as an upper bound estimate. Combining the Similar and Dissimilar figures, we find an overall detection rate across all conditions ranging from 28.5% to 37.2%, depending on the bound. The difference in detection rates between Similar and Dissimilar manipulations was statistically significant when compared using a simple t-test (p 0.05). The marginal effects, presented in the third column of table 2, provide a more intuitive interpretation of these results. For each additional minute spent constructing portfolios, participants were 4 percentage points more likely to detect a manipulation. Each additional question answered correctly on the financial sophistication questionnaire corresponded to 8 percentage point increase in detection. For portfolio complexity, adding one extra fund reduced the probability of detection by 6 percentage points. Finally there was a large (and statistically significant) discrete effect for the pension dummy variable— those enrolled in a pension were 25 percentage points less likely to detect a manipulation. Possibly they were less interested in the task.
3.2 Qualitative analysis The experiment featured a total of 398 experimental trials over its 98 participants, each with an associated audio recording of the participant’s response. Participants spent an average of 4.7 minutes constructing each portfolio, and described each choice with an average of 11 words. 174 trials involved a preference manipulation. Of these, 43 trials were detected by the participant concurrently, and 10 more in retrospect. A further 6 responses were dropped due to recording errors. The remaining 115 responses represent instances where the original choice was manipulated, but not detected by the participant. Responses in this “choice blind” group contained 13.2 words on average, but these participants spent slightly
Judgment and Decision Making, Vol. 8, No. 5, September 2013
Table 2: Random effects logit model. Dependent variable is Detection (1=yes; 0=no). Coefficient P-value† Age 0.031 Gender 0.307 Pension −1.780∗ Cheater detection −0.308 Reaction time 0.210∗ Financial sophistication 0.421∗ Memory task 0.254 Risk survey −0.061 Education −0.038 No. of funds picked −0.315∗
0.250 0.428 0.010 0.207 0.021 0.029 0.114 0.355 0.378 0.046
Marginal effect‡ 0.006 0.059 −0.249∗ −0.059 0.04∗ 0.08∗ 0.048 −0.012 −0.007 −0.060∗
† p-values correspond to one-tailed hypotheses (with the exception of the control variables—gender, age and pension—which are computed using a two-tailed hypothesis.) * p
