Computers in Human Behavior Computers in Human Behavior 21 (2005) 195–209 www.elsevier.com/locate/comphumbeh
The effects and interaction of spatial visualization and domain expertise on information seeking Ricard E. Downing a
a,*
, Joi L. Moore b, Steven W. Brown
a
Helzberg School of Management, Rockhurst University, 1100 Rockhurst Road, Kansas City, MO 64110, USA b University of Missouri, Columbia, MO, USA
Abstract Information seeking skills are becoming increasingly important as rapid and widespread developments in technology have made information available in more formats and from more sources than ever before. Research in human computer interaction (HCI) has demonstrated that primary cognitive abilities represent a powerful predictor of information-seeking success in electronic information systems. Specifically, spatial visualization ability (SVA) seems to be particularly related to hierarchical menus systems navigation within databases, online learning environments, information archival systems, and virtually all internet web sites. Research indicates that individuals with low SVA take longer to complete tasks and experience more errors on first attempts to find information in hierarchical databases compared to those with high SVA. Understanding the influences of SVA as well as its interaction with other aspects of individual differences, such as domain expertise, is critical to the design of systems intended to accommodate individual differences in users. Thirty-five college students (23 males and 12 females) were selected from the general student body of two universities and assigned to groups based upon their self-reported membership in one of two specific disciplines: business ðn ¼ 26Þ or biology ðn ¼ 9Þ. Participants were then assigned to groups based upon scores on tests of SVA using a median-split. Each participant conducted five searches: one neutral search, two searches for business related information, and two searches for biology related information using the FirstSearch archival search tool. A 2 2 factorial Analysis of Variance with one between-groups variable (high vs. low SVA) and one within-group variable (high vs. low domain expertise) indicated a significant main effect of SVA as well as a significant main effect of Domain Expertise on the time required to
*
Corresponding author. Tel.: +1-816-501-3592. E-mail address:
[email protected] (R.E. Downing).
0747-5632/$ - see front matter 2004 Elsevier Ltd. All rights reserved. doi:10.1016/j.chb.2004.03.040
196
R.E. Downing et al. / Computers in Human Behavior 21 (2005) 195–209
find their first relevant article on the search topic. The analysis also revealed that there was no main effect for SVA on the total number of relevant articles found during the search period but there was a significant main effect of Domain Expertise on the total total-number-of-relevantarticles found. There was no interaction between SVA and Domain Expertise on either time to first article or total number of articles found. Results of the study extend existing knowledge regarding the effects of SVA and domain expertise on information seeking by demonstrating a strong effect of SVA and domain expertise on information seeking skills. The results of this study also provide evidence in support of interface designs that are friendlier to information seekers who have low SVA. Related findings and suggestions for further research are discussed. 2004 Elsevier Ltd. All rights reserved. Keywords: Spatial ability; Experience level; Information seeking; Computer searching; Problem solving; Internet; Information systems
1. Introduction Rapid and widespread developments in technology have made information available in more formats and from more sources than ever before. Given the tremendous growth in the volume and types of information available, information seeking skills are quickly becoming essential in everyday life (Large, Tedd, & Hartley, 1999; Lazonder, Harm, & Woperies, 2000; Marchionini, 1995; Pirolli & Card, 1999). The number of scholarly electronic conferences has grown from just over 200 in 1991 to over 4000 in 1998. Electronic journals have increased from 27 in 1991 to 2459 in 1997. 1 From 1984 until 2003, the Internet has grown to over 170 million web sites. 2 Use of computer technology in schools can only be categorized as explosive. In 1984, there were just over 631,000 computers for student instruction reported in elementary and secondary schools in the United States; by 2000, the number of computers in use was over 12,000,000. 3 Although researchers continue to study critical aspects of information seeking in electronic environments, relatively little research has been conducted regarding cognitive abilities and their impact on information seeking skills or the interaction of certain cognitive abilities and recognized components affecting information seeking (Chen & Rada, 1996; Holscher & Strube, 2000; Marchionini, 1995; Marchionini, Dwiggins, Katz, & Lin, 1993; Palmquist & Kim, 2000). Marchionini (1995) asserts that each of us has our own set of mental structures and other skills for conducting the information seeking process. He describes this collection of skills, experiences, and resources as a ‘‘personal information infrastructure’’ (p. 11). His conceptualization of the personal information infrastructure
1
Association of Research Libraries (http://arl.cni.org/scomm/edir/index.html). Information available from the Internet Software Consortium (http://www.isc.org/ds/WWW-200301/ index.html). 3 Statistical Abstracts of the United States 1994, 2001. 2
R.E. Downing et al. / Computers in Human Behavior 21 (2005) 195–209
197
consists of four components: general cognitive abilities, domain expertise, system expertise, and searching expertise. System and searching expertise have been addressed in numerous studies examining the information seeking behavior of novices versus experts (Dillon & Song, 1997; Holscher & Strube, 2000; Kuhlthau, 1999; Lazonder et al., 2000). Additionally, research consistently demonstrates that domain expertise enhances search performance. That is, domain experts take less time and retrieve a greater number of on-target resources than those with less domain knowledge (Hirsch, 1997; Kuhlthau, 1999; Marchionini, 1995; McDonald & Stevenson, 1998; Patel, Drury, & Shalin, 1998). Information seekers who possess substantial knowledge of a subject (domain experts) approach the information seeking task differently than novices. Research has demonstrated that experts think differently about the topic. They tend to organize their knowledge around large meaningful patterns (Chase & Simon, 1973), spend qualitatively more time trying to understand a problem, (Glaser, 1987; Marchionini et al., 1993; Patel, Arocha, & Kaufman, 1994) and rapidly make relevancy judgments about the results of their information searches (Marchionini et al., 1993). Thus, domain expertise appears to have a powerful impact on the efficiency with which individuals are able to access electronic information. In addition to domain expertise, a number of cognitive factors may potentially influence the efficiency of information seeking. Researchers studying human computer interaction (HCI) have concluded that, in general, cognitive abilities (e.g., spatial visualization, vocabulary, orientation, spatial memory, spatial scanning, and speed of processing) are significant in predicting performance in HCI (Egan & Gomez, 1985; Gomez, Egan, & Bowers, 1986; Stanney & Salvendy, 1995; Vicente, Hayes, & Williges, 1987). These studies focused on information search tasks, which share characteristics with many electronic information environments that rely heavily on navigation (e.g., hierarchical databases, the World Wide Web, OVID Graphic Interface, FirstSearch, etc.). For example, Vicente et al. (1987) tested the effects of 21 individual differences on finding data within a hierarchical database. The researchers held constant three of the four components of the personal information infrastructure described by Marchionini (1995): system expertise, searching expertise, and domain expertise. All of the 21 predictors (individual differences) correlated with time as a dependent variable, that is, the time it took to find targeted data within the database. Two predictors accounted for 45% of the variance in the data: vocabulary and spatial visualization. Spatial visualization was the only predictor that was statistically significant. For the spatial visualization ability (SVA) measurements, which reflect a person’s ability to manipulate images or spatial patterns into other arrangements, Vicente et al. found that individuals who score low on SVA or spatial memory tests generally have longer mean execution times and more first try errors than individuals with high SVA. These studies also suggest the difficulties experienced by individuals with low SVA were specifically related to system navigation issues. That is, low SVA users often report being ‘‘lost’’ within hierarchical menu systems (Sellen & Nicol, 1990). Could it be that spatial visualization is a more powerful predictor of success in information seeking than any of Marchionini’s other three components? For
198
R.E. Downing et al. / Computers in Human Behavior 21 (2005) 195–209
example, if one varied the level of domain expertise as well as SVA would there be evidence of a significant difference in information seeking success? Would similar results be obtained in a different context using a different type of information system (e.g., FirstSearch, Ovid, and the WWW)? It seems clear that more research is needed in a variety of contexts in order to further test the predictive power of SVA in other contexts and in conjunction with other of Marchionini’s predictors.
2. Purpose of the Study While there has been ample research examining individually each of the four components of Marchionini’s (1995) personal information infrastructure within a set of search tasks, there has been little research examining the relationship between SVA and domain expertise. The purpose of this study is to examine the effects of SVA and domain expertise in an electronic information searching system called FirstSearch – one of a number of information searching tools in widespread use. Specific research questions to be addressed are: • Does SVA influence the time to first article and the total number of articles found for information seekers using the FirstSearch searching tool? • Does domain expertise influence the time to first article and the total number of articles found for information seekers using the FirstSearch searching tool? • Is there an interaction between SVA and domain expertise on either the time to first article or the total number of articles found? It was hypothesized that there would be a main effect for SVA consistent with the findings of (Egan & Gomez, 1985; Gomez et al., 1986; Stanney & Salvendy, 1995; Vicente et al., 1987), such that those with greater SVA will find the first article faster and find more total articles. It was further hypothesized that there would be a main effect for domain expertise consistent with the findings of (Hirsch, 1997; Kuhlthau, 1999; Marchionini, 1995; McDonald & Stevenson, 1998; Patel et al., 1998), such that those with high domain expertise will find the first article faster and find more articles than those with less domain expertise. In addition, it was anticipated that the research would provide a better understanding of the interaction (if any) between the two. For whom is SVA a more powerful predictor, those with high or low domain expertise? For example, we would like to understand if the effects of SVA diminish as domain expertise increases.
3. Methodology The experimental design was a 2 2 factorial design with one between-groups variable (high vs. low SVA) and one within-groups variable (high vs. low domain expertise). Using a 2 2 analysis of variance (ANOVA), the two subject variables (SVA and domain expertise) were tested for their main effects and interaction with two different dependent variables: time required to find the first article and the total number of relevant articles found within the search period.
R.E. Downing et al. / Computers in Human Behavior 21 (2005) 195–209
199
3.1. Participants Thirty five participants (23 males and 12 females) who were college seniors ðn ¼ 19Þ including graduate students ðn ¼ 16Þ, were selected from the student body of two universities: a small, Midwestern, four-year, private, liberal arts University, and a large, Midwestern state University. Recruitment of participants consisted of distribution of flyers across the campus, in-person, in-class recruitment with permission from instructors, and an email message distributed via several listservs to which members of the specific disciplines were known to subscribe. Participants were selected for their participation in the study according to the following: (1) hold at least senior status or greater (including graduate students) who have a declared major in business ðn ¼ 26Þ or biology ðn ¼ 9Þ, and (2) have experience using a library searching software tool. 3.2. Materials FirstSearch software is one of a number of archival search tools available for use with electronic journals and library holdings. The software offers access to over 4000 electronic scholarly journals, more than 10,000,000 full-text articles, and access to over 70 databases fully on-line. FirstSearch software provides a front-end interface from which users may initiate a search of up to 21 keyword fields as well as a multitude of other search strategies such as searching by journal or subject. The FirstSearch tool was chosen for this study because it is similar in a number of ways to many of the common archival search tools and it is available to the participants of the study based upon their student status at the university. 3.2.1. Countdown timer This freeware software package was downloaded and used for timing the searches. The software is free and available at http://www.gonebowlin.com/freeware.html. 3.3. Instruments 3.3.1. Search Phase Instructions and Search Questions Each participant was required to search for information related to a series of 5 questions using FirstSearch software (see Fig. 1). There was one neutral search for which domain expertise played no role, two business oriented questions, and 2 biology oriented questions. The questions for each discipline were developed with assistance from doctoral qualified members of the faculty from the School of Business or the Department of Biology. They were asked the following question in order to create a common focus for the search tasks: What knowledge will all of the seniors in your discipline have as general knowledge that seniors in the other discipline will not have? The questions were formed in a way that avoided keywords and provided the participant who knew the answer to the question a way to find information regarding the answer quicker than those who did not know the answer. The biology majors
200
R.E. Downing et al. / Computers in Human Behavior 21 (2005) 195–209 Neutral Question Drivers often display certain behaviors that place them, as well as others, in greater danger than would normally be expected while driving. These behaviors have become known by a certain well-known name. What is the name of this behavior? Your task is to use the FirstSearch search tool to find as many articles as fast as you can that identify (or discuss) this behavior. Business Questions 1. In addition to increasing shareholder value, businesses are accountable to certain external entities and institutions. What is the process used to determine these accountabilities? Your task is to use the FirstSearch search tool to find as many articles as fast as you can that identify (or discuss) this process. 2. Excess cash (or cash equivalents) is considered a primary protection against temporary downturns in a firm’s operations. Therefore, a high ratio of a certain indicator would compromise a firm’s ability to overcome a downturn that lasted more than a few months. What is the indicator? Your task is to use the FirstSearch search tool to find as many articles as fast as you can that identify (or discuss) ways to reduce this ratio. Biology Questions 1. The process which usually begins with glucose and ends with oxygen serving as the final electron and proton receptor is an important intracellular process. What is the overall name for this process? Your task is to use the FirstSearch search tool to find as many articles as fast as you can that identify (or discuss) this important process. 2. Sixty to ninety-five percent (60 – 95%) of every cell is composed of this substance. Therefore, the number one threat to every terrestrial organism is what? Your task is to use the FirstSearch search tool to find as many articles as fast as you can that identify (or discuss) this threat.
Fig. 1. Search phase questions.
were considered to have high domain expertise when searching for information related to the two biology questions and low domain expertise when searching for information related to the two business questions. Similarly, the business majors were considered to have high domain expertise when searching for information related to the two business questions and low domain expertise when searching for information related to the two biology questions. Kit of Factor-Referenced Cognitive Tests (Visualization Tests 1 and 2) (Eckstrom, French, Harman, & Derman, 1976) VZ-1 and VZ-2, respectively, assesses the ability to manipulate or transform a spatial image into other arrangements. Spatial visualization requires either the mental restructuring of a figure into components for manipulation or the mental rotation of a spatial configuration in short term memory, and it requires performance of serial operations, perhaps involving an analytic strategy (Eckstrom et al., 1976). VZ-1, the Form Board Test, is a timed test consisting of 48 problems in 2-parts. It tests the ability to determine which pieces of a diagram in a series of pieces can be assembled (or put together) to match the stimulus figure. Participants may use blank paper on which to draw shapes for assistance on the test. Only Part 1 of VZ-1 was used for this study. Answers were corrected for guessing following the equation described in the Manual for the Kit of Factor-Referenced Cognitive Tests. VZ-2, the Paper Folding Test, is a timed test consisting of 20 problems in two parts. It tests the ability to mentally fold and unfold a paper image and imagine a pencil hole punched through the folded paper. Each item in this test shows successive drawings of two or three folds made in a square sheet of paper. The final drawing shows a hole being punched in the folded paper. The participant is to select one of five drawings to show how the punched sheet would appear when fully opened. Only Part 1 of VZ-2 was used for this study.
R.E. Downing et al. / Computers in Human Behavior 21 (2005) 195–209
201
4. Procedure 4.1. Phase 1 (cognitive testing) All Phase 1 testing took place over the course of 3 weeks during sessions consisting of 1–5 participants. Eleven participants, 7 males and 4 females, completed Phase 1 testing at the large state University and 44 participants, 24 males and 20 females, completed Phase 1 testing at the small private University. During Phase 1 testing, all participants completed and signed the Informed Consent document, completed the Demographic Information form and completed cognitive testing using Spatial Visualization Ability (Eckstrom et al., 1976) tests 1 and 2 (VZ-1 and VZ-2, respectively). 4.2. Phase 2 (search) Phase 2 testing was conducted over the course of 3 weeks on the campus of each of the two Universities in sessions consisting of 1–7 participants based upon the availability of participants. Eleven participants, 7 males and 4 females, completed Phase 2 testing at the large state University and 26 participants, 18 males and 8 females, completed Phase 2 testing at the small private University. Each participant conducted five searches: one neutral search, two searches for business related information, and two searches for biology related information. The neutral search provided critical information in two areas: a measure of the participant’s familiarity with the FirstSearch system, and a measure of the participant’s searching expertise. Controlling for system and search expertise was critical to the success of the study. Prior to the actual search phase, the only confirmation of the participant’s familiarity with FirstSearch was their own report. The neutral search also provided a method for controlling within-group variability in search speed. The speed-to-first-article for the neutral search was used as a covariate of speed-to-firstarticle in the main searching exercises. Before beginning the searches, the computers used by participants and the computer used by the researcher to time the searches were prepared. The system clock was synchronized with the clock at the United States Naval Observatory (http:// www.time.gov/timezone.cgi?Central/d/-6/java). Manual collection of user behavior in a computer environment is tedious and error prone. Participants were required to use Microsoft Internet Explorer version 5.5 to conduct searches using the FirstSearch library archival tool. Using the ‘‘Page Setup’’ feature within IE, the header script was set to print the participant’s last name on each page they printed. The footer script was set to print the exact time the participant sent an article to the printer. Based upon these settings, the researcher was able to determine the author of each print-out and the time-to-first-article accurately. The countdown timer software was loaded on the computer used by the researcher for tracking the time for each search conducted by the participants. Finally, the FirstSearch library search tool was loaded using IE and set to ‘‘advanced search’’ (see Fig. 2). The FirstSearch archive indexes thousands of journals and other articles. In order to narrow the search
202
R.E. Downing et al. / Computers in Human Behavior 21 (2005) 195–209
Fig. 2. FirstSearch access screen used by participants.
criteria and speed searching during these timed searches, ‘‘ArticleFirst’’ was selected for the participants. Each participant’s familiarity with printing from within FirstSearch was confirmed by observation. That is, each participant conducted a simple keyword search using the word ‘‘anger’’ as the key word. Each participant was then given instructions to select the first article listed by the search and click on ‘‘print’’. After preparation of the computers, participants were given instructions regarding their task in this phase of the study and allotted 5 min to conduct each search. Part of the instructions was to print an article if they believed it was relevant to the search topic. In all cases, participants made their determination regarding relevance by selecting only those articles that contained keywords that they viewed as relevant in the title. When the 5-min time expired for a given search, participants were instructed to stop work. Participants were given a brief pause between each search to minimize fatigue. In order to control bias that may be caused by practice effects or fatigue, the order of search questions was counter-balanced across subjects. 4.3. Determining the relevance of articles Relevance of participant responses was determined in consultation with experts in the Department of Biology and the School of Business. After 2 meetings during which criteria were established to determine relevance, doctoral qualified faculty recommended that we determine relevance independently and consult with them if clarification was required. Judgments of relevance were based on the identification of particular keywords in the title of the articles.
R.E. Downing et al. / Computers in Human Behavior 21 (2005) 195–209
203
4.4. Group assignment for SVA Given the relatively small number of participants for the study, assignment to high or low SVA groups was determined by adding participant scores for VZ-1 and VZ-2 (after correction for guessing) and conducting a median split. Eighteen participants above the median were placed in the high SVA group and 17 participants below the median were placed in the low SVA group. The number of participants for each group approximates the actual population of the participant pool. For example, the number of business students meeting criteria for inclusion (i.e., 13) in the study is approximately 3 times the number of biology students (i.e., 9).
5. Results The dependent variables of time-to-first-article and the total number-of-relevantarticles were analyzed using a 2 2 mixed factorial Analysis of Covariance (ANCOVA) with one between-groups variable (high vs. low SVA), one within-group variable (high vs. low domain expertise), and using measures from the neutral search as a covariate. An a level of 0.05 was used to establish the significance of all statistical tests. 5.1. Time-to-first-article The first analysis conducted was an ANCOVA on time-to-first-article using timeto-first-article on the neutral search as a covariate. The 2 2 ANCOVA revealed a significant main effect for SVA (Fð1;32Þ ¼ 5:36, p < 0:027) with high-SVA having an average time-to-first-article of 120.72 s and low-SVA at 179.89 s.
210
Time-to-First-Article (in seconds)
190
Low-SVA NonExpert 199.65
170 150 130
Low-SVA Expert 160.12 High-SVA NonExpert 132.5
110 90
High-SVA Low-SVA
High-SVA Expert 108.94
70 50
Expert
Non-Expert
Fig. 3. Relationship between SVA and Domain Expertise based on time-to-first-article.
204
R.E. Downing et al. / Computers in Human Behavior 21 (2005) 195–209
There was also a significant main effect of domain expertise (Fð1;32Þ ¼ 7:44, p < 0:01) with experts exhibiting a shorter time-to-first-article than non experts (133.80 s vs. 165.11 s). However, there was no significant interaction between Domain Expertise and SVA (Fð1;32Þ ¼ 7:52, p ¼ 0:392) High-SVA domain experts exhibited an average of 108.94 s, while high-SVA non-experts produced an average time of 132.50 (see Fig. 3). Low-SVA domain experts required an average time of 160.12 s compared to low-SVA non-experts demonstrating an average time-to-firstarticle of 199.65 s. 5.2. Total number-of-relevant-articles found The second analysis was performed on the total-number-of-relevant-articles using a 2 2 mixed factorial ANCOVA with one between-groups variable (high vs. low SVA) and one within-group variable (high vs. low domain expertise) and using the total-number-of-relevant-articles on the neutral search as the covariate. The analysis revealed that there was no main effect for SVA on total-number-of-relevant-articles (Fð1;32Þ ¼ 1:82, p ¼ 0:19) with high-SVA participants having an average total-number-of-relevant-articles of 3.9 and low-SVA at 2.24 articles. However, there was a significant main effect of Domain Expertise on the totalnumber-of-relevant-articles found (Fð1;32Þ ¼ 8:01, p ¼ 0:008) with experts exhibiting a greater total-number-of-relevant-articles found than non-experts (4.86 articles vs. 1.83 articles). In addition, there was no significant interaction between SVA and Domain Expertise (Fð1;32Þ ¼ :1:34, p ¼ 0:26) on total-number-of-relevant-articles (see Fig. 4). 5.3. Relationship between time-to-first-article and total-number-of-relevant-articles As a final analysis, a Pearson correlation was used to examine the relationship between time-to-first-article and total-number-of-relevant-articles. As expected, there was strong negative correlation ðr ¼ :622Þ between these two variables in-
Number of Articles found
7.00 6.00
High-SVA, 5.83
5.00 4.00 3.00
High-SVA Low-SVA, 3.82
2.00
Low-SVA High-SVA, 2.00
Low-SVA, 1.65
1.00 0.00
Expert
Non-Expert
Fig. 4. Relationship between SVA and Domain Expertise based on total-number-of-relevant-articles.
R.E. Downing et al. / Computers in Human Behavior 21 (2005) 195–209
205
dicating that individuals who found the first article in less time tended to find a greater number of articles. As the time-to-first-article decreases, the total-number-ofrelevant-articles found increases.
6. Discussion The purpose of this study was to examine the effects of SVA and domain expertise on time-to-first-article and the total-number-of-relevant-articles found in an electronic information searching system called FirstSearch. Specific research questions to be addressed were: Does a main effect for SVA exist for information seekers using the FirstSearch searching tool? It was hypothesized that there would be a main effect for SVA on time-to-firstarticle and the total-number-of-relevant-articles found. Results indicated high-SVA individuals take significantly less time to find the first relevant article than those with low-SVA. However, there was no significant main effect for SVA on the totalnumber-of-relevant-articles found. Although the difference in time-to-first-article between high and low SVA individuals is not as strong, the results of this study are consistent with those reported by Vicente et al. (1987) who demonstrated that high SVA individuals found a target in a hierarchical database over twice as fast as a low SVA individual, as illustrated in Fig. 5. 6.1. Does a main effect for domain expertise exist for information seekers using the FirstSearch searching tool?
Time to Target (in seconds)
It was hypothesized that there would be a main effect for domain expertise on time-to-first-article and total-number-of-relevant-articles found. Results of the study confirmed the hypothesis. There was a significant main effect of domain expertise on time-to-first-article with experts exhibiting a shorter time-to-first-article than nonexperts (133.80 s vs. 165.11 s). There was also a significant main effect of domain expertise on the total-number-of-relevant-articles found with experts exhibiting a
200 175 150 125 100 75 50 25 0
Non-domain expert Domain expert Vicente study High SVA
Low SVA
Spatial Visualization Ability Fig. 5. Results of the present study contrasted with the Vicente et al. (1987) study.
206
R.E. Downing et al. / Computers in Human Behavior 21 (2005) 195–209
greater total-number-of-relevant-articles found than non-experts (4.86 articles vs. 1.83 articles). In addition, the results of this study regarding the total-number-of-relevant-articles found are consistent with those reported by Hirsch (1997), McDonald and Stevenson (1998), and Patel et al. (1998) who demonstrated that domain experts retrieve a greater number of on-target resources than those with less domain knowledge. In this study, there is a negative correlation between time-to-first article and the total-number-of-relevant-articles. For example, those individuals who found the first article quickly were likely to find more articles than an individual who took longer to find the first article. This is indicative of one of the main differences between individuals with high domain knowledge versus those with low (or less) domain knowledge. Finding information quicker and finding more information within a given time span are characteristics of domain experts (Chase & Simon, 1973; Chi, Glaser, & Farr, 1988; Patel et al., 1998) – unless they are constrained by low spatial visualization ability. 6.2. Is there an interaction between SVA and domain expertise? Trends in the data were in the predicted direction of an interaction between SVA and Domain Expertise despite the fact that the analysis revealed no statistically reliable interaction between two variables. Had there been more biology participants ðn ¼ 9Þ, there might well have been a reliable interaction between SVA and domain expertise. Nevertheless, in all cases, SVA had a stronger effect for individuals with no domain expertise compared to individuals with domain expertise. The best performance was exhibited by domain experts with high-SVA. The worst performance was recorded by individuals who are not domain experts and who have low-SVA. Therefore, when comparing two domain experts, one with low SVA and one with high SVA, the domain expert with low SVA will take longer to find the target. In addition, given the negative correlation between time-to-target and total-number-ofrelevant-articles, the domain expert with low-SVA will likely find fewer articles as well.
7. Conclusions Low SVA should not be viewed as an unmalleable, biological deficit. There is evidence that SVA can be improved through training and practice (Vicente & Williges, 1988). Although training is certainly a good idea for helping those with low SVA, it is not likely to help in the short term and may not be practical for most adults. To address low SVA in a more immediate way, focus should be placed on changing the interface instead of the user. For example, Stanney and Salvendy (1995) used two different user interfaces designed to assist low SVA individuals. They demonstrated that compensatory user interfaces could eliminate the differences between high and low spatial groups. However, we must caution that changing the interface to accommodate low SVA users could potentially introduce difficulties for
R.E. Downing et al. / Computers in Human Behavior 21 (2005) 195–209
207
high SVA users. Borgman (1984) determined that individuals with high spatial abilities perform better with graphic or spatially oriented interfaces. More research regarding the effects of changes in user interfaces is warranted. Specifically, research comparing the speed and accuracy of users with both high and low SVA after changes have been made to the interface to accommodate either low or high SVA users. The notion of changing the interface to meet the varying needs of individual users is a typical viewpoint of system designers of information seeking software. In order to facilitate more advanced interface capabilities, we must focus on the nature of the task first and the interface second. Information seekers are trying to solve a problem for which they need information. Thus, the user is focused on the problem or, more specifically, the solution to a problem and not the interface. This is not to say that the user interface is not important. Knowledge-based electronic information systems already contain much information and considerable resources must be devoted to the user interface. For example, analyses of expert systems with strong inference logic have demonstrated that from 40–60% of the code for the program was related to the user interface whereas only 8–10% was dedicated to the inference engine (Smith, 1988). Clearly we have not devoted enough resources to the development of the inference aspect of information seeking software. Perhaps, we need to begin to conceptualize information seeking as a problem solving process rather than a search for key terms. In order to address the problem solving characteristics of information seeking, the user interface in the electronic information seeking environment should provide a method for conducting a natural language conversation between the user and the system. Information problems should be addressed incrementally with the system providing feedback that would help the user narrow or broaden the scope of their search or additional domain knowledge regarding the topic of their search. In this way, the user can incrementally enhance their understanding of both the problem and potential solutions. Finally, there should be a mechanism for the system to ‘‘learn’’ as each information problem is solved. In this way, the system can adapt to an individual user’s information seeking methods and strategies. In summary, the results of this study as well as others over a considerable period demonstrate the need for a system design that is more accommodating to users with low SVA. We suggest that the first step in developing such a system requires focusing on the task (e.g., problem solving) instead of the user interface. An interactive ‘‘learning’’ system that provides a natural language interface and incremental feedback is an attractive starting place. In the end, Human Computer Interaction should evolve into an environment in which the computer becomes invisible and the user has a sense of interacting directly with the problem domain (Fischer & Reeves, 1995).
References Borgman, C. L. (1984). Psychological research in human computer interaction. Annual Review of Information Science and Technology (ARIST), 19, 33–64.
208
R.E. Downing et al. / Computers in Human Behavior 21 (2005) 195–209
Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive psychology, 4, 55–81. Chen, C., & Rada, R. (1996). Interacting with hypertext: A meta-analysis of experimental studies. Human– Computer Interaction, 11(2), 125–156. Chi, M., Glaser, R., & Farr, M. (Eds.). (1988). The nature of expertise. Hillsdale, NJ: Erlbaum. Dillon, A., & Song, M. (1997). An empirical comparison of the usability for novice and expert searchers of a textual and a graphic interface to an art-resource database. Journal of Digital Information, 1(1), 33– 42. Eckstrom, R. B., French, J. W., Harman, H. H., & Derman, D. (1976). Kit of factor-referenced cognitive tests. Princton, NJ: Educational Testing Service (ETS). Egan, D. E., & Gomez, L. M. (1985). Assaying, isolating, and accommodating individual differences in learning a complex skill. In R. F. Dillon (Ed.), Individual differences in cognition (vol. 2, pp. 173–217). Orlando, FL: Academic Press. Fischer, G., & Reeves, B. (1995). Beyond intelligent interfaces: Exploring, analyzing, and creating success models of cooperative problem solving. In R. M. Baecker, J. Grudin, W. Buxton, & S. Greenberg (Eds.), Readings in Human Computer Interaction Toward the year 2000 (second ed.). San Francisco, CA: Morgan Kaufmann Publishers Inc.. Glaser, R. (1987). Thoughts on expertise. In C. Schooler & W. Schaie (Eds.), Cognitive functioning and social structure over the life course. Norwood, NJ: Ablex. Gomez, L. M., Egan, D. E., & Bowers, C. (1986). Learning to use a text editor: Some learner characteristics that predict success. Human–Computer Interaction, 2, 1–23. Hirsch, S. G. (1997). How do children find information on different types of tasks?: Children’s use of the science library catalog. Library Trends, 45(4), 725–745. Holscher, C. & Strube, G. (2000). Web search behavior of internet experts and newbies. In Paper presented at the Proceedings of the 9th International WWW conference (pp. 337–346). Kuhlthau, C. (1999). The role of experience in the information search process of an early career information worker: Perceptions of uncertainty, complexity, construction, and sources. Journal of the American Society for Information Science., 50(5), 399–412. Large, A., Tedd, L. A., & Hartley, R. J. (1999). Information seeking in the online age: Principles and practice. East Grinstead, West Sussex: Reed Business Information, Ltd.. Lazonder, A. W., Harm, J. A., & Woperies, I. J. (2000). Differences between novice and expert users in searching information on the world wide web. Journal of the American Society for Information Science, 51(6), 576–581. Marchionini, G. (1995). Information seeking in electronic environments. Cambridge, UK: Cambridge University Press. Marchionini, G., Dwiggins, S., Katz, A., & Lin, X. (1993). Information seeking in full-text end-user oriented search systems: The roles of domain and search expertise. Library and Information Science Research, 15(1), 35–69. McDonald, S., & Stevenson, R. J. (1998). Navigation in hyperspace: An evaluation of the effects of navigation tools and subject matter expertise on browsing and information retrieval in hypertext. Interacting with Computers, 10, 129–142. Palmquist, R. A., & Kim, K. S. (2000). Cognitive style and on-line database search experience as predictors of web search experience. Journal of the American Society for Information Science, 51(6), 558–566. Patel, S. C., Drury, C. G., & Shalin, V. L. (1998). Effectiveness of expert semantic knowledge as a navigational aid within hypertext. Behavior and Information Technology, 17(6), 313–324. Patel, V., Arocha, J., & Kaufman, D. R. (1994). Diagnostic reasoning and medical expertise. In D. L. Medin (Ed.), The psychology of learning and motivation (31, pp. 187–252). San Diego, CA: Academic Press. Pirolli, P., & Card, S. K. (1999). Information foraging. Psychological Review, 106, 643–675. Sellen, A., & Nicol, A. (1990). Building user-centered on-line help. In B. Laurel (Ed.), The art of human– computer interface design (pp. 143–153). Reading, MA: Addison-Wesley. Smith, R. G. (1988). On the development of commercial expert systems. AI Magazine, 5, 61–75.
R.E. Downing et al. / Computers in Human Behavior 21 (2005) 195–209
209
Stanney, K. M., & Salvendy, G. (1995). Information visualization: Assisting low spatial individuals with information access tasks through the use of visual mediators. Ergonomics, 38(6), 1184–1198. Vicente, K., Hayes, B., & Williges, R. (1987). Assaying and isolating individual differences in searching a hierarchical file system. Human Factors, 29(3), 349–359. Vicente, K., & Williges, R. (1988). Accommodating individual differences in searching a hierarchical file system. International Journal of Man–Machine Studies, 29, 647–668. Ricard E. Downing is Assistant Professor of Information Systems in the Helzberg School of Management of Rockhurst University with an interest in human-computer interaction, enterprise knowledge management systems, and online educational environments. Joi L. Moore is Assistant Professor of Instructional Technology in the School of Information Science and Learning Technologies of the University of Missouri – Columbia with an interest in interactive learning environments, electronic performance support systems, and user-centered system design. Steven Brown is Associate Professor of Psychology in the College of Arts and Sciences of Rockhurst University with an interest in the cognitive aspects of instructional technology, mental models, and human learning.
