Proceedings of the 38th Hawaii International Conference on System Sciences - 2005
Stopping Rule Use During Web-Based Search Glenn J. Browne Rawls College of Business Administration Texas Tech University
[email protected]
Mitzi G. Pitts Fogelman College of Business and Economics The University of Memphis
[email protected]
Abstract The world wide web has become a ubiquitous tool for information search. The focal point of web navigation has changed over the past few years from destination sites to search engines, and search activity thus determines where most people spend most of their time online. However, little is known about how and why people stop their search efforts. Such an understanding holds much promise for both website design and marketing efforts. Building on an established theoretical and empirical foundation, we investigate the heuristics, or stopping rules, that people use to end search behavior. In the present study, subjects engaged in an online shopping task and then completed a questionnaire concerning why they stopped their searches. Results showed that some stopping rules were used more than others, and the proportions differed from those used in some prior contexts. Implications for information search theory and website design are discussed.
1. Introduction Search behavior is now a principal driver of consumer activity on the world wide web. Search engines such as Google and Yahoo, each of which now has nearly 60 million unique visitors per month, are rapidly becoming the hubs of people’s online activities [5]. In January 2004, 76% of (a total of 151 million) American internet users used a search engine at least once during the month, and each user spent an average of 40 minutes searching [20]. In total, users spent nearly 19 million hours searching using Google in 2003, and more than 49 million hours using the top 5 search engines [12]. Although no figures are available, the amount of time users spent searching for information on particular destination sites such as retailers, online news agencies, etc., is undoubtedly huge, since search engines are typically used
James C. Wetherbe Rawls College of Business Administration Texas Tech University
[email protected]
only to direct a person to a destination site. However, despite the ubiquitous presence of online search, very little is known about why people stop their search behavior. This paper builds on theory and prior empirical findings to provide initial evidence for why people stop their web-based searches. Understanding why people stop information search is important to both theory and practice. From a theoretical viewpoint, the examination of stopping behavior during information search is adding to our understanding of the full human decision-making process [3]. Information search directly affects the number and quality of alternatives considered, and thus also has an important impact on choice [23]. From a practical standpoint, stopping behavior in online information search has critical implications for website design and marketing efforts. For example, if vendors know why people stop searching for information about a product or service, they can design features into websites that help people reach a stopping point. Thus, an understanding of stopping behavior holds much promise for more effective and efficient electronic commerce. The paper is organized as follows. We next present theoretical background on web-based search behavior and stopping rule use. We then present the research methodology used in the present study, followed by the results of the study. We conclude with a discussion of the findings and implications for theory and practice.
2. Background 2.1 Information search on the world wide web The world wide web has become the search tool of choice for hundreds of millions of people worldwide. The access to almost unlimited sources of information and the nearly instantaneous results of website queries have changed information search in ways that would have been inconceivable only 15 years ago. Furthermore, access to
0-7695-2268-8/05/$20.00 (C) 2005 IEEE
1
Proceedings of the 38th Hawaii International Conference on System Sciences - 2005
knowledge has been democratized, changing from the privileged few to the many. The ageless problem of access to knowledge has been replaced by the problems of filtering and organizing knowledge. Because the knowledge available on the web is essentially unlimited, knowing when to stop searching for information has taken on increased importance (since one could conceivably search indefinitely). We next discuss various rules that people may use to stop their search behavior.
Table 1. Stopping rules Rule
Description
Example
Mental List
Person has a mental list of items that must be satisfied before he will stop collecting information.
In searching for a new car to buy, person will keep searching until he fills items on his mental list (e.g., price, color, safety features, etc.).
Magnitude Threshold
Person has a cumulative amount of information that he needs before he will stop searching.
In skimming a book, a reader may skim until he has enough information to develop a “sense” of what the book is about.
Difference Threshold
Person sets an a priori difference level to gauge when he is not learning anything new. When he stops learning new information, he stops his information search.
To determine the cause of an accident, an investigator interviews witnesses until he determines that he is not learning anything new from the additional witnesses.
2.2. Stopping rules in information search Information search is a process during which people engage in divergent thinking to seek possibilities [8]. At some point during search, a person makes a judgment that he or she has sufficient information to proceed to the next step in the decision-making process. The decision maker utilizes a heuristic, or stopping rule, to make that judgment of sufficiency [16]. The stopping rule serves to end the information search process and move the decision maker to consider alternatives and/or make a choice. Stopping rules have been investigated in much prior research (e.g., [1], [4], [7], [11], [17], [18], [25]). For example, stopping rules have been identified that rely on judgments of the economic value of information [24], expected loss from stopping [4], and the expected value of additional information [13]. Recently, a distinction has been made between the use of stopping rules in information search and design and stopping rules in choice [3]. In information search and design, stopping rules cause the decision maker to stop acquiring new information and to assess the evidence and determine his alternative set. Stopping rules are thus utilized in this stage to assess the sufficiency of information gathered. In the choice stage, stopping rules are used by the decision maker to cease his evaluations of alternatives and make a choice. The current research continues prior work concerning stopping rule use in the information search and design stage of decision making. The stopping rules considered are based on an initial study by Nickles, Curley, and Benson [15]. These authors proposed four stopping rules that decision makers might use. The stopping rules were termed the Mental List Rule, the Magnitude Threshold Rule, the Difference Threshold Rule, and the Representational Stability Rule. The use of these stopping rules in information search and design tasks was later verified for systems analysts in an information requirements determination setting [16]. Additionally, a fifth rule, which we term the Single Criterion Rule, was suggested in prior empirical data [3]. A list and descriptions of the five rules appear in Table 1.
0-7695-2268-8/05/$20.00 (C) 2005 IEEE
2
Proceedings of the 38th Hawaii International Conference on System Sciences - 2005
Representational Stability
Person searches for information until his mental model, or representation, stops shifting and stabilizes.
To understand how a machine works, a person gathers information by asking experts and by reading materials. When his mental model of how the machine works stabilizes, he stops gathering information.
Single Criterion
Person decides to search for information related to a single criterion (typically the most important one) and stops when he has enough information about that criterion.
In searching for a mortgage for a new house, the buyer searches for lenders based only on the annual percentage rate of the mortgage offered.
The context for the current study is an information search task on the world wide web. In particular, as discussed below, participants searched the web for information about digital cameras. We used theory and prior empirical findings to anticipate the use of stopping rules in this task. The first factor to consider in hypothesizing the use of stopping rules is how experienced and familiar people are with the task. Shopping for consumer products is a familiar task for nearly everyone, and thus everyone has considerable experience. In familiar contexts, use of the mental list and magnitude threshold stopping rules should be anticipated [16]. The mental list rule can be applied in familiar tasks because people have enough experience to create a mental list of items that are necessary to create an alternative set and make an ultimate choice. The magnitude threshold rule may be used because people’s familiarity with the task allows them to set a volume of information that will be sufficient for generating alternatives and making a choice a priori. In tasks in which people do not have much experience, creating a mental list of factors or setting a volume of information to collect a priori is difficult or impossible. In addition to these two rules, we anticipate that the single
criterion rule will be used in tasks with which people have considerable familiarity. In such tasks, people have enough experience to determine which criterion is most important and to focus solely on that criterion. However, it is also possible that in unfamiliar tasks people may use a single criterion as a means for simplifying the search. This type of strategy has been seen in heuristics in choice tasks (see [10]). Thus, the use of this rule in familiar and unfamiliar task types is an empirical question. However, our a priori belief is that it will be used frequently in familiar search domains. In contrast, use of the difference threshold rule and representational stability rules seems less likely in familiar search domains in which people have considerable experience. Both rules are arguably simpler rules that seem more likely to be applied in unfamiliar search contexts. The difference threshold requires only that the decision maker ascertain when he is not learning anything new from the information he is acquiring. The representational stability rule requires only that the decision maker determine when his mental model is not being significantly altered by new information. Neither of these rules requires any a priori mental activity on the part of the decision maker, and thus these rules are more likely to be used when the decision maker is unfamiliar with the context. A second, related, factor to consider, in addition to the familiarity of the task, is the “structuredness” of the task to the participants. A task is well-structured if it has recognizable inputs and outputs, and if the task’s goals and the operations that need to be performed on the inputs to create the outputs are easily understood [23]. The current task, searching for information about a particular product on the web, should be well-structured for most people. The goals of the task are easy to understand, and the inputs, operations, and outputs of the process are easily recognizable to people. In well-structured tasks, it is likely that people will employ more precise rules for stopping information search. The mental list, single criterion, and difference threshold rules seem to reflect more precision than the magnitude threshold rule and representational stability rule. The mental list and single criterion rules require that people focus on particular factors or attributes they are seeking. The difference threshold rule requires that a person set a precise difference to gauge when he is not learning anything new. In contrast, the magnitude threshold rule does not require focus on individual factors, relying instead on just the volume of information collected. Similarly, the representational stability rule requires only that the decision maker assess all of the information acquired and judge when his overall mental model is not changing. Thus, in well-structured tasks, we anticipate that the mental list, single criterion, and difference threshold stopping rules will be used for information search. In sum, the task for this study is both familiar to
0-7695-2268-8/05/$20.00 (C) 2005 IEEE
3
Proceedings of the 38th Hawaii International Conference on System Sciences - 2005
participants and well structured. The mental list and single criterion stopping rules are the only two rules whose use is anticipated in both familiar and well-structured tasks. Thus, we expect them to be used most commonly in the present task. Given this reasoning and the theoretical background, we formulated the following hypotheses (stated in the alternative form):
guidelines established by Landis and Koch [14]. A third coder coded the ten instances in which the first two coders had disagreed. The third coder agreed with one of the two original coders on nine of the ten codes. The nine codes agreed upon by two of the coders were put into the agreedupon categories. The single code upon which there was no agreement was added to the “Other” category.
H1a: More participants will use the mental list stopping rule than the difference threshold rule; H1b: More participants will use the mental list stopping rule than the representational stability rule; H1c: More participants will use the mental list stopping rule than the magnitude threshold rule; H2a: More participants will use the single criterion stopping rule than the difference threshold rule; H2b: More participants will use the single criterion stopping rule than the representational stability rule; H2c: More participants will use the single criterion stopping rule than the magnitude threshold rule.
4. Results
3. Methodology and Coding Subjects were 90 undergraduate and M.B.A. students in the college of business at a large university. Members of this age group (18-34) are the heaviest users of the world wide web [6], and thus this population is ideal for investigating online search behavior. Subjects were asked to search for a 5-megapixel camera on an electronics retailing website, and to shop to the point at which they had enough information to choose a camera to purchase at a later date. The task was performed by subjects individually in a laboratory setting. Following the task, subjects were given a questionnaire that asked two intentionally redundant questions designed to assess their reasons for stopping their search: 1. Why did you stop searching for information about the camera when you did? 2. How did you decide to stop searching for information? The written answers to these two questions were used to determine stopping rule usage. Two research assistants unfamiliar with the purposes of the study coded subjects’ answers into one of six categories using descriptions of the categories. The categories included the five stopping rule categories discussed above and an “Other” category for answers that did not fit into the stated stopping rule categories. Analysis of the coding revealed that the coders assigned 80 of the 90 subjects’ answers into the same category, for an interrater agreement of .89. We calculated Cohen’s kappa to assess the amount of agreement not attributable to chance. The kappa coefficient was .827, which is considered “almost perfect” agreement under the
The coding revealed that the following stopping rules were used by subjects in the online search task: Mental List = 46; Single Criterion = 25; Magnitude Threshold = 10; Representational Stability = 5; Difference Threshold = 2; Other = 2. To test the hypotheses, we conducted 32 tests for the various comparisons of interest. 32 tests were used because we were testing numerical counts of stopping rules utilized. As per the hypotheses, in each case the expected value of each rule in the 32 test was .50 (the null hypothesis). The results of the tests appear in Table 2. As can be seen, all of the alternative hypotheses were supported. Participants in this study used the mental list and single criterion stopping rules more than the other three rules. Table 2. Results of hypothesis tests Comparisons of stopping rules used
32
d.f.
p-value
H1a: ML > DT
40.33
1
RS
32.96
1
MT
23.14
1
DT
19.59
1
RS
13.32
1
.0003
H2c: SC > MT
6.43
1
.011
Hypothesis
Key: ML = Mental List; DT = Difference Threshold; RS = Representational Stability; MT = Magnitude Threshold; SC = Single Criterion.
5. Discussion Our results showed that most of our participants used the mental list and single criterion stopping rules when gathering information for a later purchase of a digital camera in an online search task. This finding has important implications for information search theory and practice. From a theoretical standpoint, the current research adds to our growing body of knowledge concerning stopping rule use
0-7695-2268-8/05/$20.00 (C) 2005 IEEE
4
Proceedings of the 38th Hawaii International Conference on System Sciences - 2005
in information search tasks. Previous research [3] has identified stopping rules used in several types of tasks. The present research extends this prior research by identifying the use of the mental list and single criterion rules in online consumer product search tasks. A major difficulty in decision making research generally has been the connection between people’s behavior in the information search and design phases of the decision-making process and the choices they ultimately make. Research has shown that poor choices are often the result of several types of mistakes that occur in information search and design, including not accessing relevant information [22] and stopping information search too soon [2] [19]. By documenting the stopping rules people use in information search, we can improve our understanding of why people often stop search too soon and why mistakes are made in choice. Numerous reasons for mistakes are possible given the heuristic stopping rules people use to terminate information search. For example, people’s mental lists may omit important attributes, which can lead to useful alternatives being left out of the final choice set. Application of the single criterion rule may focus on a suboptimal criterion, and in any case greatly narrows the person’s scope of search. A person using the magnitude threshold rule may set his magnitude for information too low and thus miss important information. Similar problems with the other stopping rules are also possible. Understanding how the use of various stopping rules contributes to subsequent problems in choice is a fertile area for future research. Our findings are also relevant to understanding the “stickiness” of websites. The ultimate goal of marketers is to have websites that are sticky, in the sense that once a person lands on a website, he “sticks” there [9]. Stopping rules are one reason a person might stick to a website. If website designers can take advantage of the stopping rules people use in information search, website stickiness may be enhanced. For example, if a website provides answers to every item on a person’s mental list, or addresses the criterion he thinks is most important, he is likely to be satisfied and to stick to that site. To understand people’s mental lists of important factors, market research is clearly necessary. But successful companies collect information about their customers all the time, so collecting information relevant to stopping rules for information search is certainly feasible. An interesting question for future research is whether different segments of consumers use different stopping rules in information search. Although such “individual differences” are of less importance in establishing the existence of a general phenomenon such as stopping behavior, segment-specific differences become critical in target marketing [21]. In addition to being of interest
theoretically, knowing how and why people in certain customer segments stop searching for information can help in the design of websites and value propositions that can attract or retain these customers. For example, it is likely that different customer segments (e.g., Generation Y and elderly people) have different mental lists of criteria and different primary criteria that they use in searching for information. Satisfying people’s needs for these criteria can make them stick to websites and keep them as customers. In addition, it is important to continue to investigate the impacts of task familiarity and structuredness on the use of stopping rules. For this study, the participants had high task familiarity and the task was well structured. Our hypotheses based on these conditions were supported in the current context. It is theoretically important to continue to test the hypothesized use of stopping rules under differing levels of task familiarity and task structuredness to understand stopping rule use in search behavior more fully. Another issue for future research is whether the rules identified in the present research will still be used if the ability to filter information in online search improves. It is arguable that the search mechanisms available on the web are at this point still fairly primitive. For example, well-crafted searches in Google regularly yield more than 10,000 “hits,” which is hardly useful. Despite this overabundance of information, people still need to stop searching. The stopping rules they use might be different, or might be used in various combinations, if the filtering mechanisms become better. This points out a difference between web-based search and search in everyday tasks in which the memory and mental processing aids are more elementary. The web holds the potential to change people’s stopping rules because of the help it can provide to cognition. People may shift to different stopping rules as the filtering capabilities on the web improve. Search is now at the center of consumer behavior on the world wide web. Understanding why people stop searching is thus critical to comprehending online behavior. The present research provides another step in building this understanding. References 1. K.M Aschenbrenner, D. Albert, and F. Schmalhofer. “Stochastic Choice Heuristics.” Acta Psychologica, 56, 1984, pp. 153-166. 2. J. Baron, J. Beattie, and J.C. Hershey. “Heuristics and Biases in Diagnostic Reasoning: Congruence, Information, and Certainty.” Organizational Behavior and Human Decision Processes, 42, 1988, pp. 88-110. 3. G.J. Browne and M.G. Pitts. “Stopping Rule Use During Information Search in Design Problems.” Organizational Behavior and Human Decision Processes, in press. 4. J.R. Busemeyer and A. Rapoport. “Psychological Models of Deferred Decision Making.” Journal of Mathematical Psychology,
0-7695-2268-8/05/$20.00 (C) 2005 IEEE
5
Proceedings of the 38th Hawaii International Conference on System Sciences - 2005
32, 1988, pp. 91-143. 5. comScore Media Metrix. “Top 50 U.S. Internet Property Rankings for March 2004.” Available at http://www.comscore.com/press/release.asp?press=447, 2004. 6. comScore Media Metrix. “Marketers Take Note: The Elusive 18-34 Year-Old is Habitually Online.” Available at http://www.comscore.com/press/release.asp?press=445, 2004. 7. T. Connolly and B.K. Thorn. “Predecisional Information Acquisition: Effects of Task Variables on Suboptimal Search Strategies.” Organizational Behavior and Human Decision Processes, 39, 1987, pp. 397-416. 8. Couger, J. D. Creativity and Innovation in Information Systems Organizations. Danvers, MA: Boyd and Fraser Publishing Company, 1996. 9. T.H. Davenport. “Sticky Business.” CIO, 13, 2000, pp. 58-60. 10. G. Gigerenzer and D. Goldstein. “Betting on One Good Reason: The Take the Best Heuristic.” In G. Gigerenzer et al. (Eds.), Simple Heuristics That Make Us Smart. New York: Oxford University Press, 1999, pp. 75-95. 11. Gigerenzer, G., P.M. Todd, and ABC Research Group (Eds.). Simple Heuristics That Make Us Smart. New York: Oxford University Press, 1999. 12. Kent, P. Search Engine Optimization for Dummies. New York: Wiley, 2004. 13. C.A. Kogut. “Consumer Search Behavior and Sunk Costs.” Journal of Economic Behavior and Organization, 14, 1990, pp. 381-392. 14. J.R. Landis and G.C. Koch. “The Measurement of Observer Agreement for Categorical Data. Biometrics, 33, 1977, pp. 159174. 15. K.R. Nickles, S.P. Curley, and P.G. Benson. “Judgment-Based and Reasoning-Based Stopping Rules in Decision Making Under Uncertainty.” Working Paper, Wake Forest University, October 1995. 16. M.G. Pitts and G.J. Browne. “Stopping Behavior of Systems Analysts During Information Requirements Elicitation.” Journal of Management Information Systems, 21, 2004, pp. 213-236. 17. G.F. Pitz, H. Reinhold, and E.S. Geller. “Strategies of Information Seeking in Deferred Decision Making.” Organizational Behavior and Human Performance, 4, 1969, pp. 1-19. 18. G. Saad and J.E. Russo. “Stopping Criteria in Sequential Choice.” Organizational Behavior and Human Decision Processes, 67, 1996, pp. 258-270. 19. D.A. Seale and A. Rapoport. “Sequential Decision Making with Relative Ranks: An Experimental Investigation of the ‘Secretary Problem’.” Organizational Behavior and Human Decision Processes, 69, 1997, pp. 221-236. 20. searchenginewatch.com. “Nielsen Netratings Search Engine R a t i n g s . ” A v a i l a b l e a t http://searchenginewatch.com/reports/article.php/2156451, 2004. 21. Selden, L. and G. Colvin. Angel Customers and Demon Customers. New York: Portfolio, 2003. 22. E. Shafir and A. Tversky. “Thinking Through Uncertainty: Nonconsequential Reasoning and Choice.” Cognitive Psychology, 24, 1992, pp. 449-474. 23. Simon, H.A. The Sciences of the Artificial. Cambridge, MA: MIT Press, 1981. 24. C.S. Spetzler and C.-A. Staël von Holstein. “Probability
Encoding in Decision Analysis.” Management Science, 22, 1975, pp. 340-358. 25. R.G. Swensson and R.E. Thomas. “Fixed and Optional Stopping Models for Two-Choice Discrimination Times.” Journal of Mathematical Psychology, 11, 1974, pp. 213-236.
0-7695-2268-8/05/$20.00 (C) 2005 IEEE
6
