Information searching on the Web: the cognitive difficulties experienced by older users in modifying unsuccessful information searches Aline Chevalier1, Aurélie Dommes2 and Jean-Claude Marquié1 1
Laboratoire CLLE-LTC (UMR 5263, CNRS, Université de Toulouse, EPHE), Maison De la Recherche, 5 Allées Machado, 31058 Toulouse Cedex 9, France ² IFSTTAR, French institute of science and technology for transport, development and networks. 25 allée des Marronniers, 78 000 Versailles-Cedex, France {aline.chevalier, marquie}@univ-tlse2.fr;
[email protected]
Abstract. The present study addressed age-related differences in performances and strategies developed by web users while searching for information. Ten older and 10 younger adults had to search for information with Google and to answer 9 questions varying in complexity: from simple ones (participants needed to use keywords provided in the questions) to impossible ones (no answer existed). The results showed that older participants had lower performances than younger ones; age-related differences were more particularly marked as the question complexity increased. Regression analyses showed that processing speed and cognitive flexibility accounted for a large part of the variance in performances. The younger and older participants also differed in the strategies they developed while searching for information. The older participants tended to focus on the evaluation of the results provided by Google. In contrast, the younger participants tended to plan and regulate their activity, this last strategy provided better performances. Keywords: Information searching; aging; question complexity; cognitive abilities; strategies.
1
Introduction
The information searching (IS) activity can be considered as problem-solving and decision-making activities whereby the problem-solver’s knowledge and other mental representations are manipulated in order to achieve a goal. More precisely, according to [11], the information problem-solving process is divided into the three following sub-processes: The representation of the problem to be solved. The problem statement is internalized in order to build up a mental representation of the information elements to be searched.
2
Aline Chevalier1, Aurélie Dommes2 and Jean-Claude Marquié1
The planning. A method for coming up with a solution is elaborated. It often requires dividing the problem into sub-goals. The execution. The operations that were elaborated during the planning process are carried out. These three processes are iterative as planning may generate further insights into the problem and thus promotes modified problem representations. IS using a search engine requires more particularly the individuals to generate keywords relevant to their request, to evaluate the relevance of the results provided by the search engine, and then to select one or more web pages to be visited. If the search engine does not provide the expected result, the IS activity becomes more complex: the individuals have to reformulate their first request by adding and/or suppressing keywords. Reformulating unsuccessful requests may be a high-demanding task and involves several cognitive abilities and processes such as processing speed, flexibility and working memory. Processing speed and working memory may allow individuals to remember information previously seen and to compare it to the new elements displayed on the current webpage. Cognitive flexibility may be involved to modify search strategy and requests, and vocabulary abilities to generate new keywords. Cognitive flexibility — defined as the ability to switch between cognitive strategies in order to adapt to unexpected conditions in the environment [1, 6] — working memory and processing speed are usually reported to decline with aging. In contrast, vocabulary is commonly reported to be higher in older adults than in their younger counterparts. In a recent study comparing younger and older web users, we showed that cognitive flexibility was strongly involved in the reformulation of unsuccessful requests and that age-related decline in flexibility ability affected searching performances [4]. Based on this previous work, we carried out a new experiment with younger and older web users. The aim was to replicate earlier findings and further identify the cognitive processes involved in IS and that may explained age-related differences in performances and strategies. Based on the models proposed in the literature [11, 14], the IS activity of the younger and older web users has been analyzed according to four activity components: Planning: individuals elaborate a plan to reach IS goal. The plan allows them to formalize the first request. Evaluating the relevance of the information displayed by the search engine or in the websites by comparison with the information question. Regulating: if the first request does not provide relevant result(s), individuals have to modify their strategy to find the answer. Monitoring: individuals continuously recall the information problem at hand and examine whether the actions made allow them to achieve the search goal.
Information searching on the Web: the cognitive difficulties experienced by older users in modifying unsuccessful information searches 3
2
Method
2.1
Participants
10 younger adults (age range: 21-27 years; M=24.6, SD=1.78) and 10 older adults (age range: 60-68 years; M=62.6, SD=2.12) volunteered to participate to the study. All participants were French native speakers, in good health, and had normal or corrected-to-normal vision. Education (Myounger=15.1 years, SDyounger=1.52; Molder=15.2 years, SDolder=1.87) and familiarity with Internet were controlled (Internet was used for at least five years for information searching, mailing, and chatting). 2.2
Experimental procedure
The participants first took a battery of cognitive tests: The letter comparison test [10] was used to measure processing speed. The participants had 30 sec. to examine as many pairs of letters (e.g., X O) as possible and decide whether the two letters of the pair were similar or different. The Corsi block test (initially developed by [9]) was used to assess working memory [12]. A 4×4-cells grid was shown to participants on a computer screen. Visual patterns were formed by the successive blackening of a variable number of cells at each trial. Immediately after each presentation, the participants had to reproduce the sequence in the correct order by moving the finger on this interface. The Trail Making Test reflected cognitive flexibility abilities (part B) [13]. Twenty five circles were distributed over a sheet of paper; the circles included both numbers (1 – 13) and letters (A – L). Participants were asked to draw lines as quickly as possible to connect the circles in an ascending pattern, and alternating between the numbers and letters (i.e., 1-A-2-B-3-C, etc.). The French version of the Raven Mill Hill Vocabulary scale was used to assess vocabulary skills [3]. Part 1 asked the participant to define 44 words. The part 2 consisted of a series of 44 words. For each word the participant had to choose a synonym from a list of 6 words. Then, by using Google, the participants had to answer 9 information questions varying in complexity: 3 simple questions: the keywords required to obtain the right answer were included in the statement of the question. 3 difficult questions: the relevant keywords needed to obtain the correct answer were not included in the search statement. The participants had to generate the accurate keywords by themselves. 3 impossible questions: these questions contained elements that led participants to believe that an answer could be found. Actually no answer existed on the Web. While performing the IS activity, participants were asked to think aloud [5] in order to identify the strategies they carried out.
4
Aline Chevalier1, Aurélie Dommes2 and Jean-Claude Marquié1
2.3
Variables and data analyses
The effect of age on cognitive abilities was examined using t tests. The following performances were considered: (1) Processing speed was reflected by the number of correct ratings (―Same‖ or ―Different‖ response) on the letter comparison test. (2) The working memory span corresponded to the length of the highest correct sequence that the participant was able to reproduce in the Corsi block test. (3) Cognitive flexibility ability was reflected by the time taken by the participant to complete the Trail Making Test. (4) A vocabulary score was computed (maximum score = 88 points) from responses of each participant to the Mill Hill Test. Three dependent variables were selected with regard to the information search task: (1) Search times (in sec.). For each experimental search question, the search time was calculated from the moment the participants started the search with Google (i.e. after keywords were entered in the text box) until they highlighted the answer by clicking the mouse, or until they declared abandoning the attempt to find the answer. The keywords typing time was separately computed and removed from the search time. Therefore, this measure corresponded to the total time taken by the participant to answer a search question, including all formulated requests and excluding typing speed of all keywords entered in the text box. (2) Number of correct answers. For simple and difficult search questions, a correct answer was scored 1. An incorrect answer or abandoning the attempt to find the answer was scored 0. For impossible questions, both abandoning and the participants’ indication that no answer did exist were scored 1. (3) Number of reformulations of the first request. For each search question, we computed the number of request reformulations that the participant formulated and entered in the search engine text box after a first unsuccessful request. These three dependent measures from the IS task were input into analyses of variance (ANOVAs) with age (young, old) as a between-group factor, and search question complexity (simple, difficult, impossible questions) as an intra-group factor. Qualitative analyzes were also conducted on the basis of the participants’ recorded verbal protocols, especially the planning, evaluating, regulating and monitoring activities while older and younger participants searched for information on Internet.
3
Results
3.1
Cognitive abilities
Processing speed Young adults (M=30; SD=3.13) showed higher processing speed scores than the older ones (M=21.5; SD=3,41) (t(18)=5.812, p
