Encouraging Critical Thinking Skills among College ...

The Exchange

Volume 2, Number 2 - December 2013

9

Encouraging Critical Thinking Skills among College Students Ronald E. Goldsmith, Florida State University

Abstract The present paper reflects on the role of "critical thinking" in college level education. The paper first presents definitions of critical thinking and explains why this topic is important. The following sections contain descriptions of some of the intellectual errors that inhibit critical thinking and thus good decision making. Next comes a discussion of why people in general, and especially college students, are so deficient in critical thinking ability. The paper closes with recommendations for improving critical thinking in college education and an example of a diagnostic intervention specific to marketing research. Keywords: critical thinking, reasoning, student skills Introduction A large part of a life successfully lived is the ability to solve problems or make decisions in a successful manner, that is, the outcome solves the problem or the decision is considered the right one. Not all problems, however, are solved successfully. Not all decisions are the correct ones. Everybody “makes mistakes” (Hallinan, 2009). Consequently, a worthy goal is to minimize mistakes and to make "better" decisions (e.g., Dorner, 1996; Kida, 2006). The present paper takes as its topic "critical thinking" and the work that educators can do to help students acquire critical thinking skills. These skills should improve decision-making success in school, in the workplace, and beyond. What Is Critical Thinking? The term "critical thinking" means different things to different people, often depending on the context of the discussion. Scholars and educators offer a variety of definitions, far too many to be considered here. A few examples should, however, outline the general ideal. The Foundation for Critical Thinking (http://www.criticalthinking.org/pages/defining-critical-thinking/766) begins its 412-word explanation (!) with the introductory statement: Critical thinking is the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action. In its exemplary form, it is based on universal intellectual values that transcend subject matter divisions: clarity, accuracy, precision, consistency, relevance, sound evidence, good reasons, depth, breadth, and fairness.

© 2013 – The Academic Forum

ISSN 2325-4181 Online

Electronic copy available at: http://ssrn.com/abstract=2395479

The Exchange


10

The Association of American Colleges and Universities propose a two-page critical thinking “value rubric” intended to guide teaching and evaluation of this aspect of student learning. Part of this rubric is their concise, but perhaps vague definition: “Critical thinking is a habit of mind characterized by the comprehensive exploration of issues, ideas, artifacts, and events before accepting or formulating an opinion or conclusion.” Meltzoff (1998, p. xi) states that his use of the term "ability to think critically" refers to "the skill that one has in thinking about an issue, analyzing it, looking at it from all sides, and weighting whether there is sufficient evidence of good-enough quality to warrant making a reasoned judgment that is as free of personal bias as possible." More concisely, a prominent expert on critical thinking, Dr. Michael Bishop of Florida State University, concludes that this topic is too broad and there are too many such definitions for any consensus to emerge on a precise meaning, and offers simply the idea that "critical thinking is the application of good reasoning strategies to reasoning problems" (Bishop, 2013). An exhaustive review is not necessary to the purpose of the present paper, however, which has as its context, student skills in evaluating arguments and using information in order to draw veridical conclusions from them in order to solve problems or make decisions. For the purpose of this paper, I define the term critical thinking in a way that touches upon many of the definitions experts offer, and I think strikes a middle ground between extensive detail and brevity. My understanding is that five elements characterize critical thinking: 1. 2. 3. 4. 5.

define a decision problem correctly determine the relevant information needed to make the decision evaluate the quality of the available information use the information correctly to solve the problem or make the decision evaluate the process and improve it if necessary

To elaborate on point four above, the core of critical thinking is "good reasoning" (Bishop, 2013) or "rational decision making" (Hastie & Dawes, 2010). By this, I believe that critical thinking follows the rules of logic and probability to use information to make a successful decision. This proposal is the working definition of critical thinking underlying this paper. Why Is Critical Thinking Important? Why should we be concerned about the critical thinking skills of our students? One answer is that critical thinking is one of the most important skills learned in a college education. As evidence of this claim, a 2013 survey of employers conducted for The Association of American Colleges and Universities (Hart, 2013) reported that “Nearly all employers surveyed (95%) agree, ‘a candidate’s demonstrated capacity to think critically, communicate clearly, and solve complex problems is more important than their undergraduate major.’” Thus, "higher education" isn't only about learning more facts or learning specific skills; the college experience has always uniquely involved learning advanced thinking and problem solving skills. In certain disciplines (e.g., accounting, engineering, medicine), in fact, students expect that much the substance of what they learn might outdate in coming years, but the analytic skills they learn will serve them well in the changing informational environment of their professional lives. College students who receive degrees without acquiring critical thinking skills have been shortchanged. Thus, one of the most important jobs of college educators is to teach these skills.



Electronic copy available at: http://ssrn.com/abstract=2395479

The Exchange


11

A second reason critical thinking skills are so important, to everybody and not just college students, is that modern life confronts the individual with an overwhelming array of problems to solve and decisions to make where many agencies present information bearing on that decision (Dorner, 1996; Stein et al., 2010). Media reports, marketing professionals, competing groups, and many others attempt to persuade citizens to opt for a choice, make a decision, buy something, join a movement, or reach a conclusion (Kida, 2006; Seife, 2010). The ability to evaluate the quality of the information these agents present and to use this information correctly is an essential skill citizens need to function effectively in modern society. A lack of critical thinking ability easily leads to poor decisions (Hastie & Dawes, 2010). Poor decision making has negative consequences for decision makers and society in general. In summary, critical thinking is important because its absence belies a poorly educated mind that is unprepared to make successful decisions in the modern world. This absence leads to personal, professional, and societal failure that can be avoided by better decision making. When Does Thinking Fail To Be Critical Thinking? I believe that the errors in critical thinking common to most efforts people make to solve problems and make decisions can be sorted into three categories: logical, quantitative, and reasoning. By logical errors, I mean the traditional logical fallacies identified in any logic text. For example, here are some principal ones (by no means are these lists exhaustive): • • • • • • • • • •

Ad Hominem argument - attacking the character of the person Ad Populum argument - appeals to prejudice, fear, emotion Begging the Question - assuming as true something that is yet to be proven False Analogy - drawing a parallel between dissimilar things False Dilemma - presenting a choice problem in too simple terms Red Herring - something unrelated is used to distract from the real issue Slippery Slope - assumes that an event inevitably leads to other (bad) events Stacking the Deck - presenting only selected evidence (cherry picking the data) Affirming the Consequent - affirming the consequence of a statement to support its premise Denying the Antecedent - denying the premise of a statement to deny the consequence

Quantitative reasoning and probability errors abound. They stem largely from a lack of experience and training in reasoning from numerical data to sound conclusions and to being exposed to the constant misuse of numbers by many in society that propagate these errors (Crossen, 1994; Savage, 2012, Seife, 2010). Students must learn quantitative reasoning with effort, and many people find quantitative reasoning hard to believe, preferring instead, to trust their intuitions, which are probably wrong (Niederman & Boyum, 2003). Scholars have identified many errors of this type. Here are a few examples: • • • • •

Relying on small samples (AKA small numbers) to draw conclusions Ignoring base rates Not understanding the rules of probability (e.g., the Gambler's Fallacy) Ignoring regression to the mean Ignoring the role of chance or coincidence



The Exchange


12

Finally, a third category of failed critical thinking can be found in Kahneman and Tversky's investigations of human judgment and decision making under uncertainty. Kahneman (2011) summarizes a lifetime of his research and that of others detailing the variety of thinking mistakes humans commonly make. These errors are systematic, predictable, largely unconscious, and ingrained in the way the human brain has evolved to be the "machinery of cognition" (Kahneman, 2011, p. 8). In summary, humans think in two ways, in Kahneman's terminology, fast and slow, sometimes referred to as System 1 and System 2. Hastie and Dawes (2010, p. 3) use the terms, automatic and controlled. Fast thinking (System 1) "operates automatically and quickly, with little or no effort and no sense of voluntary control" (Kaheman, p. 20). Fast thinking is innate and shared with other animals and cannot be turned off. We can only become aware of its operation. Fast thinking operates largely using heuristics or rules of thumb and is plagued by biases or systematic errors. Many failures in critical thinking can be ascribed to System 1 in action. Slow thinking (System 2) "allocates attention to the effortful mental activities that demand it, including complex computations. The operations of System 2 are often associated with the subjective experience of agency, choice, and concentration" (Kahneman, p. 21). System 2 is deliberate, requiring self-control, mental work, and attention. For our purposes, some good examples are: "compare two washing machines for overall value, fill out a tax form, check the validity of a complex logical argument" (Kahneman, p. 22). System 1 thinking is likely to jump to conclusions with little evidence, and even more telling: "The amount of evidence and its quality do not count for much, because poor evidence can make a very good story” (Kahneman, 2011, p. 209). A list containing some of the most prominent heuristics leading to failures of critical thinking (see Hollingworth, 2012, p. 101): • • •

Availability – People predict the frequency of an event based on how easily they bring an example to mind. Loss Aversion / Avoidance Bias – People tend to dislike loss more than they like gain. Anchoring – Relying too heavily on the first piece of information to make subsequent judgments

Researchers have identified a wide variety of critical thinking failures, but these three categories are good ones to begin with as educators try to help students overcome these problems. Unless trained to avoid such thinking mistakes, students are likely to make them. Identifying which errors they make is the first step to intervening and improving their skill. Once instructors can identify the major types of errors students make, they can systematically begin to try to improve the students' skills. Instructors can do this by developing effective classroom interventions that form the basis for the successful improvement of critical thinking skills. Why Do People Make So Many Mistakes? Why are people in general (and thus students) so poor at critical thinking. At least three explanations can account for this pervasive aspect of human decision making. As noted above, the first and broadest answer to this question is that, in the judgment of those who study this problem (Gilovich, 1991; Kahneman, 2011), the human mind has not evolved to think critically when evaluating information and making decisions. System 1 thinking is evolutionarily much older than System 2, a later development in human evolution. Moreover, discovering the



The Exchange


13

mistakes themselves is the product of many minds working hard to think about these issues for the past few thousand years. Most authors on this topic reach similar conclusions in this regard: •

• •

Evolution has given us powerful intellectual tools for processing vast amounts of information with accuracy and dispatch, and our questionable beliefs derive primarily from the misapplication or overutilization of generally valid and effective strategies for knowing. Just as we are subject to perceptual illusions in spite of, and largely because of, our extraordinary perceptual capacities, so too are many of our cognitive shortcomings ‘closely related to, or even an unavoidable cost of, [our] greatest strengths’ (Gilovich, 1991, p. 2). . . . we all have natural tendencies to search for and evaluate evidence in a faulty manner (Kida, 2006, p. 15). . . . our minds are susceptible to systematic errors (Kahneman, 2011, p. 10).

Thus, the basic reason people are poor in critical thinking ability is not that they are ruled by their emotions rather than reason (both faculties are needed for good decision making), not to say that often this is the case; nor that they are basically irrational or incapable of reasoning, not to say that some people are better than others are at reasoning; but that human rationality is bounded or limited (Gilovich, 1991). This means that even when trying to make a rational decision, many people fall prey to the inherent limitations of their rational minds (Kahneman, 2011). In the words of Piattelli-Palmarini (1994, p. 142): The systematic failure of many of our 'judgments under uncertainty' is not argument against the canons of rationality, but rather a demonstration that we frequently, without being aware of it, adopt strategies and mental intuitions that vary quite a bit from the formulas prescribed by those rational rules. . . . We have come to see that our minds spontaneously follow a sort of quick and easy shortcut, and that this shortcut does not lead us to the same place to which the highway of rationality would bring us. The second explanation for a lack of critical thinking in general is that many actors or agents in society promote poor thinking. Politicians, marketers, pollsters, journalists, and the media in general should be held accountable for a constant barrage of violations of critical thinking (Blastland & Dilnot, 2009; Crossen, 1994; Seife, 2010). Sometimes they do this out of ignorance, simply repeating the mistakes that many people naturally make; sometimes out of carelessness, not taking the trouble to apply critical thinking skills to a problem; but often deliberately, in an effort to persuade or mislead in pursuit of some goal that benefits them. The media is especially guilty in this regard, preferring to report the outrageous, the astonishing, the facile, and the conventional aspects of the story rather than critically analyze it and judge its soundness (Seife, 2010). The third explanation for a lack of general critical thinking skills, and the one most directly relevant to the intent of this paper, is that they are not taught in school systems (Kida, 2006, p. 15). Most college professors got that way because they are more or less good at reasoning and critical thinking. Most college courses focus on teaching subject matter and lack formal instruction in critical thinking skills. Students are only sporadically shown some of the major mistakes in human reasoning that characterize much of their decision making, and teachers make few efforts to train them systematically in good reasoning skills.



The Exchange


14

Are College Students Good At Critical Thinking? The simple answer to this questions is generally "no." Two, and perhaps three, arguments support this conclusion. The first is that, in general, one major feature of human psychology is that people in general (students are people) often make poor decisions. Many scholars make this fact the starting point for their studies, the most prominent example being Kahneman and Tversky, who transformed the modern understanding of human decision making and problem solving with their seminal work on thinking (see Kahneman, 2011). The second support for this assertion comes from the efforts of scholars to measure and assess empirically the critical thinking ability of college students. The extensive work of the Center for Assessment and Improvement of Learning at Tennessee Tech University (see Stein & Haynes, 2011) has led to the development of the Critical Thinking Test or CAT, a 15-item paper and pencil test that takes about an hour to complete (see also, http://www.CriticalThinkingTest.org). This instrument has proved to be a reliable and valid assessment of critical thinking ability. For the purposes of the test, the following comprise the elements of critical thinking (Stein & Haynes, 2011, p. 45): •

•

•

•

Evaluating Information o Separate factual information from inferences. o Interpret numerical relationships in graphs. o Understand the limitations of correlations data. o Evaluate evidence and identify inappropriate conclusions Creative Thinking o Identify alternative interpretations for data or observations. o Identify new information that might support or contradict an hypothesis. o Explain how new information can change a problem. Learning and Problem Solving o Separate relevant from irrelevant information. o Integrate information to solve problems. o Learn and apply new information. o Use mathematical skills to solve real-world problems. Communication o Communicate ideas effectively.

For example, here is a sample analogous question (Stein & Haynes, 2011, p. 45): A scientist working at a government agency believes that an ingredient commonly used in bread causes criminal behavior. To support his theory the scientist notes the following evidence:  99.9 percent of the people who committed crimes consumed bread prior to committing crimes.  Crime rates are extremely low in areas where bread is not consumed. Do the data presented by the scientist strongly support their theory? Yes --- No --Are there other explanations for the data besides the scientist's theory? If so, describe. What kind of additional information or evidence would support the scientist's theory?



The Exchange


15

Stein and Haynes (2011) report that about 20% to 30% of college students say the information supports the scientist's theory. For the CAT results overall, Stein et al. (2010, p. 4) report that the "average senior score was about 51% of the maximum possible score on the CAT instrument (maximum score = 38 points)". That is, although scores on the CAT improve steadily from a freshman low of about a mean of 14.8, at the end of four years, the senior average is only about 19.5! Empirically many U.S. college students seem to lack critical thinking skills. A third bit of evidence can be offered. Ask any college professor if his or her students are good at critical thinking, and the likely answer will be "no." This "evidence" however, is anecdotal, and thus is the poorest type of evidence and should be avoided if one is a good critical thinker! An important point about the above is that for most college professors, the availability and personal familiarity of specific instances would make this evidence compelling. Intervening and Assessing Critical Thinking in the Classroom Stein and Haynes (2011, p. 46) present a model for encouraging critical thinking skills among college students: Assess Student Performance Improve Student Learning

Increase Faculty Awareness of Student Weaknesses

Increase Faculty Awareness of Effective Practices The model starts with the goal of improving student learning. The first step is to assess student performance. Instructors can complete this step in a variety of ways. The Center for Assessment and Improvement of Learning developed the CAT for this purpose, but instructors can develop "home grown" assessment tools for individual disciplines. The results of assessing student performance should give faculty the information to evaluate how deficient students are in critical thinking and diagnose specific weaknesses in critical reasoning skills. For instance, results of student attempts to use information to solve a problem such as the CAT question example above betrays a lack of understanding of causal reasoning. Faculty can then develop effective interventions to "teach" students this specific critical thinking skill. Repeating the procedure should lead to overall quality improvement in student ability to think critically. In essence, the model proposes a "test or measure, diagnose, intervene, test or measure again" procedure. Bishop (2013) also describes a three-step framework to improve student critical thinking ability. Step one is to articulate clearly a teaching goal. Step two is to articulate target reasoning strategies you wish your students to adopt. Step three is to develop a classroom intervention to reach the goal. Some recommendations for how to do this are to (1) debias, get students to stop thinking a certain way; (2) repurpose, alter the way they think; and (3) add a new and different way to think. Devising the classroom interventions to accomplish these tasks is the difficult part. For example, here is a diagnostic intervention that a marketing professor could use to assess whether students had assimilated the principles needed to assess whether evidence is valid or not for a causal claim:



The Exchange


16

The Story of a Brand Manager Today is April 5, 2013. Five months ago, last December 1, to be exact, your boss appointed you to manage a brand. The previous manager had done nothing new with the brand for four years. Your boss asked you to boost sales of your brand by 3% over the next 3 months. You drew upon your knowledge of marketing strategy to develop a marketing plan that you thought had a good chance of increasing sales. The plan essentially used a game consumers could play that you thought would attract their interest, involve them enough so that they would increase their purchases of the brand, and perhaps attract new buyers as well. The strategy combined the game with increased TV advertising to inform buyers. You also persuaded a restaurant chain to crosspromote the game in their restaurant using table tents their diners would find on tables when they entered the restaurant. The table tents promoted both the restaurant and the game. Your promotion strategy kicked off on January 1, 2013. It is now three months later. You have just received the syndicated sales data from your research supplier detailing brand sales. This information appears in the appendix. You are thrilled that the data show that your promotion led to the hoped for increase in sales. As of January 1, 2013, sales were flat for the final quarter of 2012. Now, sales during the first quarter of 2013 show an increase of 3%, showing your strategy worked. You turn in these numbers to your boss, expecting a pat on the back. The boss, however, disputes your claim. Why does she do this? What evidence would you need to support your claim that the promotion had the desired effect? The intervention focuses on causal reasoning and the type of evidence that is needed to support a causal claim. The instructor would present basic principles of understanding causality and the evidence needed to support causal claims in class and discuss them with students. Then the scenario is presented, and student answers are recorded. The student answers are assessed using a point system (0 to 3) that assigns points for correctly recognizing flaws in the evidence and its interpretation. The three key aspects of sound causal reasoning lacking in the story are: a. Are more than one potential cause mentioned in the story? b. Is there a background rate of change; can they see a base rate? c. Is the data from an experiment or not? What is the control group? 1. Common problems: (1) Students need practice in detecting confounding, or the mixing of potential causal factors together. They focus on the one they are interested in (the game) and neglect others (increased TV advertising and cross promotions). (2) They have difficulty in accepting random fluctuations as real phenomena. They think every change is caused by something they can identify, ignoring the background effects of many other causes that are producing the fluctuations. The data show fluctuations within a narrow range that represents the base rate of change. (3) Finally, there is no control group to which the data can be compared. 2. The assessment plan: The instructor can assess student progress in critical thinking by practicing this type of exercise in class and then including test questions on the exam that



The Exchange


17

present the same type of problem. Success or failure of the assessment can be determined by the number of students who correctly answer the question. Following Stein and Haynes model, faculty can use such interventions to assess how well students understand and can apply sound reasoning, faculty can also diagnose where students are deficient so they can devise and perfect interventions that improve student learning. Conclusion "A good critical thinker properly applies good reasoning strategies to reasoning problems" (Bishop, 2013), but critical thinking does not come “naturally.” Avoiding systematic errors in critical thinking stemming from logical, quantitative, and automatic biases is a first step in improving critical thinking. "Critical-thinking skills are regarded by many faculty as the most important outcome of an undergraduate education" (Stein & Haynes, 2011, p. 49). Unfortunately, many if not most, undergraduates in U.S. colleges and universities appear not to be learning these skills. Never mind the reasoning shortcomings of the vast majority of the population that has not been exposed to these skills at all. Given the pervasive and systematic human shortcomings in critical thinking, fueled and abetted by numerous agents in society, scholars from a variety of disciplines (e.g., philosophy, psychology, engineering) have studied and reflected on critical thinking. Despite the wide variety of formal definitions, good reasoning can be described in an efficient way across many disciplines. A note can be added regarding the success or failure of decisions. When decisions go wrong, failing to lead to the desired outcomes, it might be better not to judge or criticize the decision maker for the failure, but by the process used to make the decision. Critical thinking is promoted as a method of making decisions and solving problems that should lead to more successful outcomes than flipping a coin or following a hunch or gut feeling (unless the gut belongs to a real expert and not a pundit). Critical thinking does not guarantee successful outcomes, but it is in the long run better than the alternatives. Decisions made using the best critical reasoning procedures can still not succeed. The role of chance in an uncertain and unpredictable world cannot be ignored (see Dorner, 1996; Taleb, 2004, 2010). In order for universities and colleges to improve critical thinking, several elements are needed. Perhaps the most important element is faculty engagement and commitment; faculty must embrace the need to teach critical thinking. Wanting to do so and being successful, however, require the ability to measure or diagnose the problems students have in good reasoning. Moreover, faculty need resources and administrative support to help develop successful interventions to give students experience in acquiring critical thinking skills. This means that college and university administrations must fully support and encourage faculty in their efforts to train students. If college administrators do not supported these initiatives, and not just with lip service, then faculty might not see them as important enough to attempt them. Following a “diagnose, intervene, assess progress, and improve” model should lead to continual improvement not just in student critical thinking ability, but also in the process itself. One potentially contentious issue in this regard will arise. Educators disagree on whether it is best to conceptualize and address critical thinking as a discipline specific topic or to think of it as a set of general skills and abilities. Any body of educators is unlikely to reach a consensus opinion on this issue, so perhaps the best way to resolve it is to embrace both ideas as partially correct and work from there. Each unit will most likely need to develop its own domain-specific program, but units can learn best practices from each other by sharing ideas. The core of critical



The Exchange


18

thinking is common to all disciplines as it is to solving problems and making decisions out of the class room setting. After all, the goal is to produce citizens who can make successful decisions in all aspects of their lives. References Bishop, M. (2013). Thinking about critical thinking. Presentation, May 22, 2012, Florida State University. Blastland, M., & Dilnot, A. (2009). The numbers game: The commonsense guide to understanding numbers in the news, in politics, and in life. New York: Gotham Books. Crossen, C. (1994). Tainted truth: The manipulation of fact in America. New York: Simon & Schuster. Dorner, Dietrich (1996). The logic of failure. Cambridge, MA: Perseus Books. Gilovich, T. (1991). How we know what isn't so. New York: The Free Press. Hallinan, J. T. (2009). Why we make mistakes. New York: Broadway Books. Hart Research Associates (2013). It takes more than a major: Employer priorities for college learning and student success. Association of American Colleges and Universities. Retrieved from http://www.aacu.org/leap/documents/2013_EmployerSurvey.pdf Hastie, R., & Dawes, R. M. (2010). Rational choice in an uncertain world (2nd ed.). Los Angeles, CA: SAGE. Hollingworth, C. (2012). Behavioral economics: A blueprint for new aha moments, in R. Kadens, G. Linda, & M. Price (Eds), Leading Edge Marketing Research: 21st-century Tools and Practices, Thousand Oaks, CA: SAGE, pp. 89-119. Kahneman, D. (2011). Thinking fast and slow. New York: Farrar, Straus and Giroux. Kida, T. (2006). Don't believe everything you think: The 6 basic mistakes we make in thinking. Amherst, NY: Prometheus Books. Meltzoff, J. (1998). Critical thinking about research. Washington, DC: American Psychological Association. Niederman, D., & Boyum, D. (2003). What the numbers say: A field guide to mastering our numerical world. New York: Broadway Books. Piattelli-Palmarini, M. (1994). Inevitable illusions. New York: John Wiley. Savage, S. L. (2012). The flaw of averages: Why we underestimate risk in the face of uncertainty. Hoboken, NJ: John Wiley. Seife, C. (2010). Proofiness: The dark arts of mathematical deception. New York: Viking. Stein, B., & Haynes, A. (2011). Engaging faculty in the assessment and improvement of students' critical thinking using the CAT. Change: The Magazine of Higher Learning, 43 (2), 44-49.



The Exchange


19

Stein, B., Haynes, A., Redding, M., Harris, K., Tylka, M., & Lisic, E. (2010). Faculty driven assessment of critical thinking: National dissemination of the CAT instrument. Proceedings of the 2009 International Joint Conferences on Computing, Information, and Systems Sciences and Engineering, 2010. Taleb, N. N. (2004). Fooled by randomness. New York: Random House. Taleb, N. N. (2010). The black swan. New York: Random House.

Author Information Ronald E. Goldsmith, PhD. Marketing Department College of Business 821 Academic Way Florida State University Tallahassee, FL 32306 [email protected] Ronald E. Goldsmith, Ph.D., is the Richard M. Baker Professor of marketing at Florida State University, where he teaches consumer psychology and marketing research. He has published four book and over 150 journal articles. His research interests are measurement in consumer research and personality influences on buyer behavior. His interest in this topic was stimulated by a summer 2013 grant to participate in a university wide initiative to encourage critical thinking among FSU undergraduates.

