Explicitness of local navigational links: comprehension, perceptions of ...

Journal of Information Science http://jis.sagepub.com

Explicitness of local navigational links: comprehension, perceptions of use, and browsing behavior Kathryn A. Mobrand and Jan H. Spyridakis Journal of Information Science 2007; 33; 41 DOI: 10.1177/0165551506068144 The online version of this article can be found at: http://jis.sagepub.com/cgi/content/abstract/33/1/41

Published by: http://www.sagepublications.com

On behalf of:

Chartered Institute of Library and Information Professionals

Additional services and information for Journal of Information Science can be found at: Email Alerts: http://jis.sagepub.com/cgi/alerts Subscriptions: http://jis.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations (this article cites 35 articles hosted on the SAGE Journals Online and HighWire Press platforms): http://jis.sagepub.com/cgi/content/refs/33/1/41

Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on February 7, 2008 © 2007 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

Explicitness of local navigational links: comprehension, perceptions of use, and browsing behavior

Kathryn A. Mobrand and Jan H. Spyridakis 14 Loew Hall, Box 352195, University of Washington, Seattle, WA 98195-2195, USA

Received 23 September 2005 Revised 24 February 2006 Abstract. This study investigated the effect of explicitness of navigational links on comprehension, perceptions of use, and browsing behaviour in an informational web site. The purpose was to determine whether link explicitness would assist users in overcoming cognitive overload and disorientation. Subjects took a pre-knowledge survey, browsed web pages in one of four link explicitness conditions, and took a post-browsing survey on comprehension and perceptions of use. Link explicitness differentially affected the outcome measures. Organizationally explicit navigational links resulted in lower scores on the post-comprehension survey. A combined condition of semantically and organizationally explicit links resulted in subjects reporting that they followed more embedded links. Traditional links and semantically/organizationally explicit links resulted in subjects exploring more of the study web site. These results, together with subjects’ comments and webserver log files, indicate that navigational link labels clearly affect user performance – ambiguous link labels degrade comprehension and constrain browsing; traditional navigational links and links that provide dual signaling encourage broader sampling of a web site.

Keywords: comprehension; browsing behaviour; navigation; link labels; internet-based experiment

1.

Introduction

The goal of this study was to test the effect of navigational link phrasing on users’ comprehension, perceptions of use, and browsing behaviour when reading hypertext on the web. Specifically, we wanted to test whether more explicitly worded links would provide coherence that would help users overcome the cognitive overload and disorientation so often associated with reading hypertext. Explicitness was Correspondence to: Kathryn A. Mobrand, 14 Loew Hall, Box 352195, University of Washington, Seattle, WA 98195-2195, USA. E-mail: [email protected] Journal of Information Science, 33 (1) 2007, pp. 41–61 © CILIP, DOI: 10.1177/0165551506068144 Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on February 7, 2008 © 2007 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

41

Mobrand & Spyridakis: Explicitness of local navigational links

examined with respect to the content of the target node (semantic explicitness), the location of the target node within the overall hypertext structure (organizational explicitness), and a combination of these two approaches. Research on print documents has shown that signals – headings, previews, and logical connectives – are effective in providing coherence to readers, thus enhancing their performance [1–5]. This study sought to determine whether certain hypertext elements could be used in a similar way, acting as signals to enhance user performance on the web. In order to understand the effect of signaling on user performance in the web environment, one must look not only at research on signaling (both in print and on the web), but also at research on text structure, user task, and prior knowledge because these text and user characteristics interact with signaling. Some empirical studies have looked at signaling with hypertext elements, including link labels; however, most studies have been conducted with closed hypertext systems [6–10]. There is a paucity of empirical work on how signaling interacts with text structure, task, and prior knowledge, and how such interactions inform web authoring guidelines, and what research there is often provides contradictory or unclear results [11–13]. It has been noted that further empirical work is needed that controls for reading strategies, investigates more complex navigational aids, uses more controlled experiments with random assignment, employs different tasks and goals, uses naturally occurring web sites as test beds, and looks at more complex hypertext structures to highlight relationships between overviews and learning [13–16]. Because of the lack of extensive research in this area, much of the literature on authoring hypertext for the web is based on expert opinion, not empirical evidence. This study sought to derive empirically-grounded guidelines for web authors regarding the phrasing of navigational links in support of user performance. In looking specifically at one element (i.e. local navigational links), we also intended to lay groundwork for a broader examination of signaling effects on user performance with Web-based hypertext.

2.

Background

In the last decade, centuries of reading habits have been irrevocably altered by the emergence of the World Wide Web, as it has become a major vehicle for information dissemination supporting the growing complexity of everyday life. Agencies and businesses are increasingly turning to the web as their primary communication medium, and face-to-face encounters are becoming rarer, as evidenced by automated telephone messages that encourage consumers to visit a web site. As large bodies of expository information migrate to the web, users find themselves not just searching the web but also browsing and reading web texts for meaning [17]. This expository information assumes a variety of text structures that – depending on the degree of linearity and amount of reader control offered by the hypertext – bring new responsibilities and require new strategies for both readers and writers [18–21]. At the more linear, author-controlled, end of the spectrum are expository hypertext documents which contain only forward and backward navigational links (e.g. the next and previous links often found in the upper and lower corners of the text area of a web page). Many of these documents are similar to print documents in their linear nature. These documents lack the visual signals typically used in the world of print (e.g. page numbers, chapters, discrete pages), however; text simply appears and disappears into an abyss as the reader selects ‘next’ or ‘previous.’ The reader has no sense of progression through, or extent of, the text as a whole, and this lack of structural context taxes cognitive resources [22, 23]. Additionally, hypertext readers must divert some of their cognitive resources from comprehension to activities such as scrolling, clicking on navigational buttons, setting page size, or even adjusting resolution or brightness of the screen [24–26]. One strategy that authors of linear hypertext can employ to help minimize the reader’s cognitive burden is the use of traditional signaling [1]. In contrast to the linear end of the spectrum is the nonlinear, more reader-controlled, end of the hypertext spectrum where one encounters modularized, interconnected hypertext documents that Journal of Information Science, 33 (1) 2007, pp. 41–61 © CILIP, DOI: 10.1177/0165551506068144 Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on February 7, 2008 © 2007 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

42


require truly different reading strategies than those needed for the more linear hypertexts. No longer following the author’s prescribed path, readers control the journey in a number of ways: by selecting hyperlinks that are embedded in the text and transport them to new nodes (or web pages); by using the linear next or previous links; by clicking on links in navigation columns; by using the browser’s back or forward buttons; or by using search boxes [13, 19–21]. Traditional print-based signals do not translate easily to such hypertexts, and the reader finds no comforting boundaries or sense of closure in the malleable web environment. The reader must share his or her limited pool of cognitive resources available for reading between comprehending the text, manipulating the reading tool, and making decisions about the path to take through the text [24, 26–27]. To further challenge the web reader, much expository information on the web has been adapted from print text intended to be read linearly by audiences whose reading strategies were developed long before the web. To meet the ever-increasing demand for online information, authors rush print-based documents to the web, often with little knowledge of writing techniques that would assist web readers with their new shared authorship role and that would be sensitive to a variety of reader tasks (e.g. searching vs browsing, reading to do vs reading to learn) and a range of web/internet experiences and skills. These authors are in need of research-based guidelines for creating expository hypertext documents that facilitate users’ browsing and reading for meaning in a web environment through the incorporation of web-based signals that provide coherence to modular hypertext articles. The study presented here seeks to provide a basis for the development of such guidelines by investigating the effect of the explicitness of local navigational links on user performance; specifically, comprehension, perceptions of use, and browsing behavior.

3.

Literature review

The current study builds upon existing research that examines the effect of signaling on user performance and the interaction of signalling with text structure, user task, and prior knowledge. Before we discuss studies that investigate signaling effects in print and in hypertext, we first examine studies of performance with different text structures (print vs hypertext) as well as studies that compare varying levels of hypertext linearity. We close by describing studies of the effects of user task and prior knowledge, including interactions between text structure, signaling, user tasks, and prior knowledge. 3.1.

Text structure and user performance

While the current study focuses on web-based hypertext, it draws upon literature that examines text linearity from a wider perspective. Hypertext refers to any text with links that can transport readers instantaneously from one place in the text to another. Hypertext may be as linear as print text if the only navigational options consist of links such as ‘next’ and ‘previous’ that perform a page-turning function. Hypertext is frequently nonlinear, however with navigational options consisting of links that let readers move through the text in a nonsequential fashion. Hypertext visionaries have grappled with the challenges of electronically publishing exponentially growing amounts of information in an increasingly complex world, specifically seeking to facilitate access and retrieval of information by accommodating, mimicking, or even augmenting the associative and hierarchical workings of the human mind [28, 29]. Conklin [24] noted that although hypertext technology provided new opportunities for using the computer to communicate, it also presented many challenges – most notably, cognitive overload and disorientation. Many scholars maintain that hypertext organizes information in ways that can accommodate human cognition and allow for information retrieval in a meaningful, nonlinear fashion [30–32]. In response to the growing popularity of disseminating expository information on the web as well as the growing number of textbooks in hypertext format, some researchers are shifting their focus from an examination of search and retrieval tasks with hypertext to learning and comprehension [12, 17]. Journal of Information Science, 33 (1) 2007, pp. 41–61 © CILIP, DOI: 10.1177/0165551506068144 Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on February 7, 2008 © 2007 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

43


Many advantages have been noted for learning with hypertext. Hypertext allows readers to approach nodes of information from different angles, creating multiple links between existing and new information, which leads to deeper processing and greater learning [6]. It allows readers to tailor the pace and sequence of navigation to individual learning styles or prior knowledge [6, 25, 30, 33]. It sparks motivation by offering choices and interactivity, thus empowering readers [34] and providing pleasure through serendipitous browsing [35]. Empirical studies of learning with hypertext vs linear text have found advantages for several measures: comprehension [12, 25], problem solving [27], and perceptions of use (e.g. self-determination and motivation, attitude toward learning with hypertext, perceptions of disorientation, and perceptions of navigational freedom/control) [27, 30, 31, 36]. Studies have also shown that different hypertext structures may differentially encourage extent of coverage of a hypertext space for subjects with different learning styles [19]. Many disadvantages have also been identified for learning with hypertext. Hypertext can lead to cognitive overload as users share their limited pool of cognitive resources between comprehending text and attending to navigational decisions [24, 25, 37]. The balance of cognitive resources required for comprehension and retention of hypertext differs from that required for traditional linear text – while processing demands are not greater overall for hypertext, processing relational information in hypertext is more demanding than in linear text [18]. Hypertext readers often become disoriented, not knowing where they are, where they came from, or how to get to where they would like to go [24]. Following embedded associative links in hypertext may interrupt the narrative flow, leaving the reader to provide context and form inferences [37–39], a step that can use up cognitive resources and degrade comprehension [26]. Hypertext may actually fragment thinking when its web-like, modularized structure does not fit the way in which humans process information [40–41]. Further, hypertext readers with little prior knowledge or motivation may be unable to make effective navigational decisions [6, 24]. Empirical studies of learning with hypertext vs linear text have found disadvantages for several measures: comprehension and various measures thereof [18, 23, 27, 36], perceptions of use [22, 42], navigational performance [22], and coverage of materials [23, 27, 36]. Collectively, many studies emphasize the need to mitigate the problems of cognitive overload and disorientation that can occur with hypertext if readers are to realize the benefits that hypertext, with its learner control and cognitive flexibility, can offer for comprehension. 3.2.

Signals and user performance

Many researchers have looked to the addition of signaling and other navigational aids to mitigate the costs of hypertext and increase its benefits. Signals – words, phrases, or clauses that announce content or reveal content relationships – act as structural cues that alert readers to upcoming text and help them create hierarchical frameworks to incorporate new information, making it more readily understood and remembered [1–5]. Empirical studies with print documents have shown that signals (headings, previews, and logical connectives) help readers remember superordinate information and make inferences, particularly with long, difficult passages, and over time [3]. In one study, signals were shown to enhance memory for topics and their organization but decrease the recall of details [5]. Another study found that headings increased the amount of information recalled for more topics in general and for unfamiliar topics and the number of unfamiliar topics accessed [4]. While signals have been used effectively in print documents, they are often not relevant or easily adapted to the nonlinear and modular nature of hypertext. Being unable to predict the reader’s path, hypertext authors can find it difficult to provide effective signals that facilitate understanding [7]. Hypertext authors may, however, address coherence between nodes in several ways: by explicitly indicating the semantic relationship between linked nodes, juxtaposing the content of the preceding and current nodes in order to connect existing (or ‘given’) information to new information, or providing orientation cues that help readers identify their current position and obtain an overview of the hypertext structure [37, 39, 43]. Authors of hypertext documents may also provide Journal of Information Science, 33 (1) 2007, pp. 41–61 © CILIP, DOI: 10.1177/0165551506068144 Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on February 7, 2008 © 2007 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

44


other types of navigational aids (e.g. embedded hyperlinks, indexes, local navigational links) [6, 8], and the hypertext system or web browser interface itself may provide other navigational aids (e.g. search functions, back and forward buttons) [44]. Placement of hyperlinks within the text provides context for those links, while placement of hyperlinks in lists outside the text may enhance search efficiency, particularly if the information associated with the hyperlinks is already known [11]. While these aids can assist readers in moving through the hypertext space, they may or may not function effectively as signals to aid comprehension. Many have theorized about the advantages of explicit link labels in facilitating coherence between nodes and providing the context that readers need to summon relevant prior knowledge when they move to a new web page [45, 46] by providing information about the relationship between the source and target nodes or by indicating a category to which the link belongs [20, 47, 48]. Explicit link labels (e.g. links labeled with wording such as ‘is an example’ or ‘is a contrast’) vs less explicit links labels (e.g. links labeled with wording such as ‘related content’) or links without labels [6] can help readers make better decisions about whether to follow links, thus minimizing surprises and missed opportunities [49]. Link titles (short descriptions that pop up when one mouses over hyperlinks) can also provide coherence and facilitate comprehension by helping readers form expectations about upcoming text [35, 50]. Many studies have empirically investigated the topic of link labels. Link labels that reveal the semantic relationship between the source and target text have been shown to lead to higher comprehension scores and more positive perceptions of use than link labels that simply say ‘related content’ [6]. Link indicators with labels that contain explicit information about the type of content at the target node have been shown to promote more focused reading even though unlabeled link indicators have been shown to encourage wider sampling of a hypertext system [8]. In one study, the addition of labeled content links that expressed the relationship of the source to target text enhanced search performance [7]. Further, links labeled ‘next’ and ‘previous’ confused users because the link labels conflicted with the browser’s ‘back’ and ‘forward’ buttons [44]. This finding is of particular interest because one motivation for the current study was to show that replacing the widespread ‘next’ and ‘previous’ with more explicitly worded navigational links would provide coherence to interconnected expository hypertext articles and thus enhance user performance. Empirical studies of signaling with hypertext elements have also shown that clearer organization (e.g. structured vs unstructured overviews, graphical vs text overviews) facilitate comprehension [15, 51], enhance user perceptions, increase time spent engaging with the text [16], decrease perceptions of disorientation [14], and focus browsing [15]. Studies have also shown that placement of links can positively affect search speed and perceptions of search efficiency [11]. The empirical study of signaling with hypertext in the web environment is an emerging field. Studies are needed that examine different ways of making link labels explicit – particularly in environments in which readers follow embedded links across hypertext articles, a scenario that is becoming extremely common with the abundance of linear documents that are being converted to hypertext on the web. 3.3.

Task

The effect that text structure and signaling have on user performance is greatly influenced by the user’s task. Real world needs lead users into a variety of tasks on the web, from searching for specific information to browsing for specific or nonspecific reasons. The boundaries of these categories often blur, and addressing multiple user behaviors presents a challenge for web authors [52]. The nature of a given task relates to the definition of success for that task. Search success may be easily measured by accuracy and efficiency of information finding, and by perceptions of efficiency [7, 11, 27, 32]. However, browsing (reading and consuming information) success, which is a bit trickier to assess, may be measured by comprehension (how much of the content is understood and retained) [6, 7, 12–14, 16, 18, 51], exploration of the hypertext (extent of the hypertext covered) [6, 12, 26, 27], perceptions of use [6, 7, 12, 14, 16, 53], and engagement with the hypertext (e.g. length of time on each page) [6, 7, 16, 26]. Journal of Information Science, 33 (1) 2007, pp. 41–61 © CILIP, DOI: 10.1177/0165551506068144 Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on February 7, 2008 © 2007 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

45


User tasks have clearly been shown to affect performance. Subjects with specific learning objectives have been shown to recall more factual concepts and relations than subjects without objectives [54, 55]. User tasks have also been shown to interact with the effects of text structure and text signaling. In one study, subjects with problem-solving goals had higher comprehension scores and perceptions of control with nonlinear vs linear hypertext, while subjects reading to learn experienced no effect from text structure on performance or perceptions of use [27]. Subjects with search goals have been shown to use a more linear reading strategy, viewing fewer nodes and pages of text and reading for shorter amounts of time [26]. Subject task has also been shown to reliably predict behavior patterns in log file data, with problem-solvers displaying more constrained navigational patterns than browsers [55]. Finally, subjects with search tasks have expressed enhanced perceptions of use through the addition of explicit link labels [7] and received more benefit from structural contextual aids than subjects with browsing tasks [53]. In summary, many studies demonstrate that users with specific vs nonspecific tasks perform better in general, tend to be more positively affected by nonlinear vs linear texts, benefit more from signaling, and display more constrained navigational patterns. The ease of measuring search success may explain why so many studies have examined such tasks. What is needed now are studies that employ a hybrid task – a focused browsing task that allows readers of expository text to explore a hypertext document with a specific purpose. 3.4.

Prior knowledge

The effect that text structure and signaling have on user performance is also influenced by users’ prior knowledge. Prior knowledge helps readers select information to place in memory, link new information to information that is already in memory, and provide placeholders for the instantiation of information [40]. When investigating readers of hypertext prose on the web, researchers must consider the prior knowledge of the hypertext structure and navigational options (web experience and structural schemata for hypertext on the web), and of the subject matter itself (content familiarity). When the text structure and content presented match the information stored in the reader’s memory, interpretation and recall are facilitated; when there is a mismatch, the stored structure and information base vie for power with new information [40]. Thus, learning with hypertext is affected by many interacting factors: the learner, the task, the system features, and the educational setting [32]. For example, the more unfamiliar the screen elements are, the more difficult it will be for meaningful learning to occur [56]. Clearly, prior knowledge will affect performance and thus the benefits to be derived from signaling with hyperlinks [4, 32]. Empirical studies have shown that prior knowledge affects performance. Knowledgeable subjects performed better and had more positive perceptions of use than unknowledgeable subjects [9, 14, 16, 57]. Prior knowledge has also been shown to interact with the effects of text structure and signaling. Subjects with high prior content knowledge performed better with linear text than with hypertext, while low prior knowledge subjects were more successful at problem solving with hypertext than with linear text [36]. High prior knowledge subjects also had higher reading comprehension scores with linear text than with hypertext, with low prior knowledge subjects showing higher reading comprehension scores with mixed hypertext vs linear text [12]. One study found that low prior knowledge subjects learned more by following a highly coherent reading order, while high prior knowledge subjects learned more following a low coherence order [13]. Another study found that low prior knowledge subjects had higher global comprehension with hierarchical maps than with a more complicated network map or an alphabetical list [51]. In contrast, the presence of graphical overviews resulted in less disorientation and higher comprehension scores, particularly for subjects with high prior knowledge [14]. Yet subjects with little or no prior knowledge struggled more with the freedom and lack of context offered by nonlinear text [26]. Similar to findings with print text [1, 3, 4], hypertext studies have shown that subjects who possess a high level of content knowledge benefited less from navigational aids than those with little or no prior content knowledge [9]. Prior knowledge of

Journal of Information Science, 33 (1) 2007, pp. 41–61 © CILIP, DOI: 10.1177/0165551506068144 Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on February 7, 2008 © 2007 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

46


navigational traditions has also been shown to affect perceptions of use: novice users reported being confused by the navigational links ‘previous’ and ‘next’; interestingly, experienced users reported being confused by the concept of ‘previous’ and ‘next’ vs the traditional browser functionality of ‘back’ and ‘forward’ [44]. In summary, these studies show that users with high prior knowledge vs those with low prior knowledge – in terms of content, information structure, or web experience – perform better in general and typically benefit less from signals. There are mixed results for the interaction of prior knowledge and text structure, but it appears that less knowledgeable users may struggle more with the use of hypertext. 3.5

The current study – hypotheses

Given the existing literature on the influence of text structure and signaling on user performance, we designed the current study to assess four versions of link explicitness of navigational links, ranging from least to most explicit: generic, semantically explicit, organizationally explicit, and a combination of semantic and organizational explicitness. Generic links were defined as naturally occurring links (e.g. next and previous); semantically explicit links revealed the content of the target node; organizationally explicit links revealed the location of a target node in the overall article structure; and the semantically and organizationally explicit combined links revealed both the content of the target node and the location of a target node in the overall article structure. Subjects logged into the study remotely, took a pre-knowledge test, read a scenario, browsed web pages in one of the link explicitness conditions, and took a post-browsing survey on demographics, computer experience, comprehension, and perceptions of use. Subjects’ navigational activity was captured in web server log files. The following hypotheses were formulated regarding the effect of explicitness of local navigational links. We anticipated that the replacement of generic navigational links with explicit links would lead to: • Higher scores on measures of comprehension; the more explicit the links, the higher the comprehension scores. • More positive ratings of perceptions of use; the more explicit the links, the more positive the perceptions of use. • Greater exploration (broader coverage) of the site; the more explicit the links, the broader the coverage.

4. 4.1.

Methods Design

This study employed a one-way design. The between-subjects variable of link explicitness had four levels: (1) generic, (2) semantic, (3) organizational, and (4) semantic/organizational. The dependent measures were comprehension, perceptions of use, and site exploration. 4.1.1. Subjects Eighty-four individuals participated in this study; their mean age was 48.5 (SD = 14.2), and 70% were female. Participants were recruited through a variety of venues that targeted adults who in the course of their daily lives might seek out medical information. Study advertisements were posted on web sites of many organizations (e.g. the University of Washington Orthopedics and Sports Medicine, the State of Washington Arthritis Foundation) and flyers were placed at senior centers, retirement facilities, churches, etc. One of the goals of this study was to approach subjects in their own natural environment and subjects who would be likely to seek medical information online, in that the test web materials were drawn from an ehealth web site.


47


4.1.2. Materials Study materials included experimental web pages, survey instruments, and instructional pages. The experimental web pages were adapted from the Arthritis Source (AS) web site (developed by Dr Frederick Matsen of the University of Washington, in conjunction with the State of Washington Arthritis Foundation). The AS was selected because it was a live, naturally occurring web site with an established history of activity and its content would be of interest to readers and would support a plausible browsing scenario. The selected web pages consisted of four hypertext articles on general topics concerning arthritis and spanned 28 pages (Table 1). The pages were relatively short (on average, 534 words per page) with easy enough content for subjects to cover much of the material in a short timeframe while being substantive enough to support meaningful comprehension. Also the texts had originally been adapted from linear print documents, a primary focus of this study being an examination of print documents adapted for the web. In their native state, all texts contained both generic local navigational links (‘next’ and ‘previous’) that supported sequential exploration within an article and embedded associative hyperlinks that supported nonsequential navigation within and between articles. A template for the experimental web site was created from these pages through three steps: (1) stripping extraneous material, (2) removing links that led outside the site, and (3) replacing the traditional left-hand navigation column with two buttons (‘Home’ and ‘Survey’) to force subjects to rely on the local navigational and embedded links for movement within and between web articles. The content of the articles was preserved verbatim, and the Arthritis Source logos were preserved to maintain credibility. Figure 1 shows the finished template for one of the web pages (note the generic navigational links in the corners of the content area). After replicating the template three times, we replaced the generic wording of the local navigational links (i.e. next and previous) in the corners of the web pages in three of the four versions so as to create the four link explicitness conditions, using the wording shown in Table 2, where the levels of the independent variable (link explicitness) are defined. As shown in the table, the generic links (derived from the Arthritis Source web site) represent the common standard of ‘next’ and ‘previous.’ To create more semantically explicit links, we used the title of the target node to replace the word ‘next’ or ‘previous’ as appropriate; for example, on one page ‘next’ was replaced with ‘Causes.’ To create organizationally explicit links, we used the location of the target node within the overall structural space, using, the word ‘section’ to denote that location because the Arthritis Source web site refers to the individual pages of each article as ‘sections.’ Two surveys were used to assess comprehension and perceptions of use, as well as to collect information on subject and text perception variables. Survey 1, administered before subjects browsed the test web site, collected two types of information: (1) personal identifiers to facilitate matching participants’ responses to the two surveys (while preserving anonymity) and (2) domain knowledge (including an arthritis knowledge pre-test). Survey 2, administered after browsing, collected six types of information: (1) personal identifiers, (2) web experience, (3) demographics, (4) comprehension measured through an arthritis knowledge post-test identical to the Survey 1 domain knowledge test, (5) perceptions of the text and the ease of use of the test web site, and (6) additional comments about the web pages. The comprehension test consisted of 18 multiple-choice items with three answer choices and was created following the guidelines of Haladyna [59] and Oosterhof [60]. The comprehension test was Table 1 Page and word counts for the four web articles used in the experimental web site Title

No. of pages

Total words

Average words/page

Frequently asked questions (FAQ) Pain management (Pain) Fatigue Diet Summation: all articles

9 8 4 7 28

4,910 4,426 2,847 2,758 14,941

547 553 712 394 534


48


Fig. 1.

Template for the experimental pages – a page from the generic condition web site.

Table 2 Operationalization of link explicitness Condition

Definition

Example link

Generic

Naturally occurring links: ‘next’ and ‘previous’

Next

Semantic

Links explicit about the content of the target node

Causes

Organizational

Links explicit about the organization of the articles, specifically about the location of the target node within the overall structure of the hypertext article

Section 3 of 9

Semantic/ organizational

A combination of links used for the semantic and the organizational conditions, i.e. links explicit about target content and target location within the article

Section 3 of 9: Causes

reviewed multiple times by Dr Matsen and tested for study mechanics and passage dependency by University of Washington (UW) graduate students. The goal of passage dependency testing was to ensure that the questions and answers did not predispose any specific answer. The rationale for the matching pre- and post-comprehension tests was to obtain prior knowledge scores for participants so as to use those scores as a covariate in the analysis of post-comprehension scores. The instructional pages contained introductory information and a browsing scenario. The introductory information overviewed the procedure and informed subjects that they could withdraw at any time, they would remain anonymous, and they could enter a drawing for an Amazon.com gift certificate. The browsing scenario sought to encourage exploration of the entire hypertext space as well as reading for meaning rather than just searching for discrete facts – it asked subjects to envision that they or a close friend or relative had arthritis and to focus their browsing on medical topics related to the four articles. This page also asked subjects to browse the experimental web pages for at least 15 minutes. Finally, a confirmation page asked subjects to confirm their decision to quit browsing and move on to Survey 2; it also instructed them not to use the browser ‘Back’ button for the remainder of the study. Journal of Information Science, 33 (1) 2007, pp. 41–61 © CILIP, DOI: 10.1177/0165551506068144 Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on February 7, 2008 © 2007 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

49


4.1.3. Technical administration of the study Components of this online study resided on two servers at the University of Washington. A cgi script on the browsing instruction page randomly (and transparently) assigned subjects to one of the four experimental conditions. While many of the studies we reviewed utilized client-side tracking software on subjects’ local machines in a laboratory or other controlled environment to track subject behavior, studies conducted remotely over the Internet must depend upon information stored in web server logs or gained from server-based tracking programs [46, 61]. Many researchers warn that log file statistics should be examined carefully and interpreted cautiously because the data have limitations with respect to completeness and accuracy [62–65]. In our study, we used a program option that disabled caching on the web server that hosted the experimental web pages. This disabling is important because caching is the main limiting factor cited for the usefulness of web server log files: caching prevents many page requests from reaching the server and being recorded in the server log, resulting in greatly underestimated page request figures [62–65]. 4.1.4. Procedures Subjects accessed the study site by typing the study URL into their browser or by clicking on the URL in an announcement on a web page. Subjects read the study overview, took the first survey, and read the browsing instructions and scenario. They were then randomly assigned to one of the conditions where they browse one of four test web sites, take the second survey, and enter the drawing.

5.

Results

Results are grouped by demographics, prior knowledge, web use patterns, comprehension, perceptions of use, subject comments, and log files. Results are considered to be significant at an alpha level ≤ 0.05. 5.1.

Demographics

Subjects in this study were well educated, with 39.3% (n = 33) of the subjects having taken some undergraduate coursework or earned a bachelor’s degree; another 51.2% (n = 43) of the subjects had taken some graduate level coursework or earned a graduate or professional degree. Ninety three percent (n = 78) of the subjects reported English as their native language. 5.2.

Prior knowledge

Prior knowledge measures were grouped into domain knowledge and text perceptions. Subjects in this study were well connected to arthritis, with 71.4% (n = 60) reporting that either they or a relative or close friend had arthritis. Subjects were primarily newcomers to arthritis web sites, with only 12% (n = 10) reporting that they had previously visited an arthritis web site; however, almost twothirds of the subjects (n = 56) reported occasionally visiting medical web sites. Subjects’ pre-comprehension scores revealed that they were moderately knowledgeable about arthritis, scoring an average of 9.57 (SD = 2.89) out of a possible score of 18. Given that the questions had three answer choices, a score of 6 would have represented a chance score. To assess text perceptions, we asked subjects to rate their familiarity with, interest in, and views of difficulty of the web site text on five-point scales. Subjects in this study were moderately familiar with the text (M = 2.89, SD = 1.05; 1 = very unfamiliar, 5 = very familiar); were rather interested in the text (M = 3.8, SD = 1.03; 1 = very uninterested, 5 = very interested); and found the material rather easy (M = 3.87, SD = 0.85; 1 = very difficult, 5 = very easy). 5.3.

Web use patterns

Subjects in this study were active users of the internet/web, with 75% (n = 63) reporting daily use. Subjects were also fairly eager web readers, with an average rating of enjoyment of reading on the Journal of Information Science, 33 (1) 2007, pp. 41–61 © CILIP, DOI: 10.1177/0165551506068144 Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on February 7, 2008 © 2007 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

50


web of 3.52 (SD = 1.02; 1 = very unenjoyable, 5 = very enjoyable). Subjects were moderately web literate, having participated in an average of 4.21 (SD = 1.36) of the eight activities listed in the survey (e.g. emailing, shopping, chatting/instant messaging, playing interactive games). Subjects also appeared to be moderately skilled users of web browsers, having performed an average of 2.37 (SD = 1.43) of the five skilled actions listed in the survey (e.g. bookmarking a page, changing a browser setting, creating a web page). 5.4.

Link explicitness and comprehension

To assess the effect of link explicitness on post-comprehension, an ANCOVA was performed using pre-comprehension as a covariate. This procedure was selected to best focus on the between-subjects effects [66]. There was a significant effect for explicitness of local navigational links on postcomprehension, F (3, 81) = 3.305, p = 0.024, partial η2 = 0.112, α = 0.734. Given the relatively small effect size yet sufficient power, we used a least squares difference post-hoc test to identify significant group differences, being willing to accept a small risk of Type II error in order to further analyze group differences. The LSD revealed that subjects in the organizational condition scored significantly lower on the post-comprehension test than subjects in both the semantic and the semantic/organizational conditions (p = 0.023 and p = 0.003, respectively), and somewhat lower than subjects in the generic condition (trend toward significance, p = 0.061) (Table 3). 5.5.

Link explicitness and perceptions of use

To examine the effect of link explicitness on perceptions of use, we performed separate ANOVAs on the variables concerning perceptions of navigation, orientation, and site organization. Readers should note that some of the variables have positive connotations at the high end of their five-point scales; others have negative connotations. A trend toward significance [F (3, 82) = 2.472, p = 0.068] occurred for the effect of link explicitness on subjects’ ratings of their frequency of following embedded links. A least squares difference post-hoc test revealed that there were potentially significant differences between the semantic/organizational condition and the generic (p = 0.026), semantic (p = 0.051), and organizational conditions (p = 0.021), with subjects in the semantic/organizational condition reporting following more embedded links than subjects in any other condition (Table 4).

Table 3 Post-comprehension means Link explicitness

n

Adjusted meana

Standard error

Generic Semantic Organizational Semantic/organizational

22 21 20 21

11.07 11.35 9.87 11.79

0.44 0.44 0.46 0.44

a

Means adjusted for inclusion of covariate.

Table 4 Ratings of frequency of following embedded links (1 = never, 5 = very often) Link explicitness

n

Mean

Standard deviation

Generic Semantic Organizational Semantic/organizational

22 21 20 21

2.95 3.05 2.90 3.76

1.43 1.28 0.97 0.89


51


No significant effects were found for link explicitness on the other perception of use measures. Subjects were not very often surprised by link targets or confused about their own location in the experimental webpages. They could frequently find the information they were interested in, they found the website relatively well organized, and they felt it was enjoyable to obtain information using the site (Table 5). 5.6.

Subject comments

At the end of Survey 2, subjects provided additional comments concerning their perceptions of the arthritis web pages (~50% of the subjects [39 of 84] submitted responses). Blind raters (five graduate students in Technical Communication at the University of Washington) used a plus/minus method to evaluate responses on two measures: (1) perceptions of the arthritis web pages and (2) ease of use of the web pages. Because of the high level of consistency between the raters (96%), individual ratings were tallied and averaged by experimental condition (Table 6). Of the rated comments, only the comments pertaining to ease of use (e.g. navigation, disorientation, link following) are relevant to this study and are discussed here. The generic condition received many negative comments concerning ease of use of the website (about 88% of the comments had ‘minus’ ratings). Subjects expressed dissatisfaction with ease of finding information and following the informational flow and tended to indicate a reliance on the linear ‘next’ and ‘previous’ navigation or a reluctance to follow embedded links. Similarly, the semantic condition received many negative comments (93% of the comments had ‘minus’ ratings). Subjects expressed frustration with embedded links and the multiple pathways they offer, and with disorientation. The organizational condition and the semantic/organizational condition received more positive ratings for ease of use of the web site than the generic and semantic conditions; however, ratings were still somewhat negative, with 65.0% and 56.5% of the comments having ‘minus’ ratings, respectively. Subjects in the organizational condition were divided on the usefulness of embedded vs local navigational links, and they expressed some feelings of disorientation, and subjects in the semantic/organizational condition expressed mixed sentiments about embedded links and frustration with computers in general. Other comments are further examined in the Discussion section where relevant. Table 5 Ratings of perceptual variables Perceptual variable

n

Mean

Standard deviation

84 84 84 83

3.17 2.04 2.08 3.65

1.20 0.90 1.04 0.77

Ratings of site organization (1 = very poorly organized, 5 = very well organized)

84

3.80

0.86

Ratings of enjoyability of information (1 = very unenjoyable, 5 = very enjoyable)

83

3.57

0.95

Reported frequency of navigation behavior (1 = never, 5 = very often): Followed embedded links Were surprised by a link target Were confused about their location Could find desired information

Table 6 Positive and negative ratings (expressed as a percentage of all ratings) concerning ease of use Comments

Generic

Semantic

Organizational

Positive (plus) Negative (minus)

12% 88%

7% 93%

35% 65%

Semantic/organizational 44% 56%


52


5.7.

Log files

Before a discussion of the log file results, readers will find it helpful to understand how we analyzed the log file data that captured browsing activity (i.e. page requests). Individual sessions were determined using a 20-minute time period to denote one session – although log file literature indicated 30 minutes for a typical session [63], the technical staff associated with the test web site that we were using for our study had used 20 minutes in the past when examining the log files from their site, so we opted to go with their protocol. A visit refers to requests received from a unique IP address. When there has been no activity from that IP address for specified period of time (in our case 20 minutes), then that visit (or session) is considered to be over. Within each experimental condition, the total page requests were tallied for each of the four web articles and for the web site as a whole. By subtracting the total repeat page requests within a session from the total page requests, we were able to determine the total number of unique page requests (again, within each condition, for each web article and for the entire web site). We then calculated the extent of coverage by dividing the total number of unique page requests by the total number of possible unique page requests (number of pages multiplied by number of subjects). The extent of coverage, expressed as a percentage for each article and the entire site, is shown in Table 7; the extent of coverage for each article is also depicted graphically in Figure 2. Table 7 and Figure 2 show that all subjects in the study, regardless of condition, saw a great majority of the pages in the first article they encountered (the FAQ article); subjects in the semantic/organizational condition saw all of the pages in the FAQ, subjects in the organizational condition saw 91.0%, subjects in the semantic condition saw 88.9%, and subjects in the generic condition saw 87.9%. Table 7 also shows that subjects in the semantic/organizational and generic conditions saw the largest percentage of the entire web site (46.1 and 45.3%, respectively). Subjects in the semantic and the organizational conditions saw fewer pages (40.1 and 36.1%, respectively). Another result from the log file data of relevance here concerns the percentage of total unique page requests that subjects made for articles outside the initial FAQ article, as shown in Table 8. The higher numbers in the table indicate greater willingness to move beyond the original article, which required the use of embedded links. Conversely, lower numbers in the table reveal browsing that was more constrained to the FAQ article. Subjects in the generic condition ventured outside the FAQ with the most regularity, making 37.6% of their unique requests from other articles. Subjects in the semantic/organizational and semantic conditions moved outside the FAQ less often, making about 29% of their unique requests from other articles. Finally, subjects in the organizational condition moved outside the FAQ the least often, making only 18.8% of their unique requests from other articles. Figure 3 shows the percentage of total unique requests in the web site attributed to each of the four articles by link explicitness condition.

6.

Discussion

Before a discussion of the results in terms of the projected hypotheses, it is informative to briefly discuss the nature of the subjects in this study and how their backgrounds positioned them in terms of potential benefits from signaling provided by explicit link labels. Table 7 Extent of coverage (expressed as a percentage) for each web article and for the entire web site Generic

Semantic

Organizational

Semantic/organization

FAQ Pain Fatigue Diet

87.9% 33.0% 25.0% 16.2%

88.9% 17.9% 9.6% 20.4%

91.0% 6.3% 13.8% 12.1%

100.0% 28.0% 14.3% 14.2%

Total web site

45.3%

40.1%

36.1%

46.1%


53


Fig 2.

Extent of coverage (expressed as a percentage) for each web article.

Table 8 Unique page requests (expressed as a percentage of total unique page requests) made for articles outside the FAQ Requests for pages outside the FAQ Generic Semantic Organizational Semantic/organizational

Fig 3.

37.6% 28.8% 18.8% 29.5%

Total unique page requests (expressed as a percentage) for each web article.


54


This study attempted to utilize subjects in their own natural environment, seeking subjects who might resemble the target audience for the Arthritis Source. Studies have shown that more women than men seek health care information online and most are between 30 and 64 years of age [58]. This study also sought to examine subjects who are old enough to have developed reading skills in a preweb era. Demographic results reveal that the subjects in this study were reasonable representatives of the target population: mostly middle-aged, female, almost exclusively native English speakers, with a higher level of education than the average internet user. A majority of subjects reported that they or a relative or close friend had arthritis and that they used medical web sites occasionally. Not surprisingly, subjects reported being moderately familiar with and rather interested in the information in the experimental web pages and found it quite easy. The pre-comprehension scores reinforced these reports, showing that subjects were moderately knowledgeable about arthritis. Subjects in this study were also fairly facile, confident, eager web users (75% reporting daily use of the web), with moderate skill level with web browser functionality. We wanted to test natural users of the experimental web site in order to increase our external validity and the likelihood that uncompensated subjects would volunteer a sufficient amount of time to complete the entire study even though we knew, based on the literature, that we were running the risk of testing subjects who would least benefit from signals in that they were knowledgeable about arthritis and facile web users [3, 4, 6, 9, 32, 36, 40, 44]. 6.1.

Link conditions and comprehension

The first hypothesis for this study was not supported: comprehension scores were not higher when more explicit wording replaced the generic wording of local navigational links. Unexpectedly, comprehension scores in one of the explicit conditions – the organizational condition – were significantly lower than scores in the semantic and the semantic/organizational conditions, with a trend towards being significantly lower than the generic condition. After we discuss the possible reasons for the poor performance in the organizational condition, we will also address the surprisingly good performance of the subjects in the generic condition and the lack of significant differences between the other conditions. The poor comprehension scores in the organizational condition correlate with the fact that subjects in this condition saw fewer pages from the web site than subjects in other conditions (as shown in Table 7) – covering only 36.1% of the total site. This finding may be attributed to the fact that subjects found the organizational link labels to be confusing and disorienting. Subjects’ comments revealed that they were confused about whether these labels referred to the link’s target or to their current location in the text, they misunderstood the function of the links (e.g. mistaking them for referential links to supplemental information), and they were uncertain about whether they had covered all of the material in the test web site. It was unexpected that these labels would confuse subjects in that previous research had shown that providing structural context to hypertext users helps reduce navigational problems by previewing the hypertext space [53]. Clearly, the lower comprehension scores and negative comments in the organizational condition support other existing research findings that suggest that confusing or misleading signals increase cognitive burden and degrade performance [10, 24, 25, 37, 44]. While we still believe that providing organizational signals can improve performance for hypertext users, it might be helpful to instantiate the concept of ‘current’ and ‘target’ locations more clearly by adding a label between the corner navigational links. This centered label would provide information about the location of the current node within the overall hypertext structure (e.g. ‘Page 4 of 10’) and allow the corner navigational links to simply designate the target nodes (e.g. ‘< Page 3’ and ‘Page 5 >’). It is interesting that subjects in the generic condition performed almost as well on comprehension as those in the semantic and the semantic/organizational conditions. Although some believe that users perform better with conventional features regardless of whether they are the most effective features [61, 50], we hypothesized that using more explicit wording than ‘next’ and ‘previous’ would lead to higher comprehension scores because it would mitigate the cognitive overload and Journal of Information Science, 33 (1) 2007, pp. 41–61 © CILIP, DOI: 10.1177/0165551506068144 Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on February 7, 2008 © 2007 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

55


disorientation when hypertext readers jump to new articles with a different linear sequence. We believe that the unexpectedly high comprehension scores in the generic condition may be attributed to that fact that replacing the generic labels violated a familiar convention and that any potential benefits from explicit signaling were offset by the removal of the familiar wording. Subjects in the generic condition may have been more comfortable with the familiar navigational links and thus more secure about their ability to navigate the site – log files confirm that subjects in the generic condition made a larger percentage of unique requests from articles outside of the FAQ than subjects in the other three conditions. The lack of significant differences in comprehension scores between the three conditions (excluding the organizational condition) is also of interest and may be attributed in part to the subject characteristics (as discussed earlier), the comprehension test itself, and the short browsing timeframe. The comprehension test sought to assess the content of the Arthritis Source, which was rather encyclopaedic, and it was revised numerous times in order to achieve passage dependency, resulting in a comprehension test that assessed more subordinate than superordinate information. The recall of superordinate information has been shown to benefit most from signaling [5] even though Einstein et al. [67] maintain there is a relationship between recall of superordinate and subordinate information. Another cause for the lack of differences between the three conditions could be the short experimental timeframe. Subjects may not have had enough time to browse for information related to the scenario and demonstrate that knowledge on a comprehension test, a notion supported by comments that there was too much information to absorb in the time allotted. While our instructions asked subjects to browse the study web pages for at least 15 minutes, subjects may have interpreted this as an indication that a 15-minute timeframe was adequate for completing the task. Our suggested browsing time was based on the idea that a total of 30 minutes was the maximum amount of time that we believed subjects would stay with the study (including reading instructions, browsing, and answering questions) since the only incentive was the opportunity to enter a drawing for a $75 gift certificate at Amazon.com. 6.2.

Link explicitness and perceptions of use

The second hypothesis for this study was not supported: perceptions of use were not more positive when explicit wording replaced the generic wording of local navigational links. There was a trend toward significance, however, for the effect of link explicitness on subjects’ beliefs concerning following more embedded links, with subjects in the semantic/organizational condition reporting following more links than subjects in the other three conditions. The reporting of following more embedded links in the semantic/organizational condition may be attributed to the fact that subjects who received two types of signaling were more secure about their place within the hypertext and thus more willing to explore the hypertext, knowing that they could find their way back to their current location. Subjects in the semantic/organizational condition made the largest percentage of positive ease of use comments. One possible explanation for the lack of differences between conditions for the other perceptual variables is that subjects in this study were fairly web literate, which would certainly make them more likely to adapt quickly to, and be less affected by, unusual labels for local navigational links. Another possible explanation is that subjects in general did not explore much of the site beyond the FAQ as shown by the log file data (62–81% of unique page requests were made within the FAQ). 6.3.

Exploration of the study web site

The third hypothesis for this study was not supported: exploration (extent of coverage) of the site was not broader when more explicit wording replaced the generic wording of local navigational links. The results for exploration of the site are based primarily on the log file data. It is important to note that log files reveal subjects’ actual performance rather than their perceptions of their performance, an important distinction because we know that what they do can be quite different from what they say they do. Journal of Information Science, 33 (1) 2007, pp. 41–61 © CILIP, DOI: 10.1177/0165551506068144 Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on February 7, 2008 © 2007 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

56


The log file data indicate that subjects in the semantic/organizational condition and the generic condition covered the largest percentage of the study web site, while those in the organizational condition covered the smallest percentage of the study web site. These findings may, as mentioned previously, be attributed to the fact that subjects in the semantic/organizational condition and the generic condition felt confident in their ability to navigate the site and were, therefore, willing to explore more of the web site. Subjects in the semantic/organizational condition may have benefited from dual signaling and subjects in the generic condition may have benefited from using the familiar (traditional) links.

7.

Conclusions and future research

Reading and writing have come a long way – there is a whole new concept of text with responsibility shared between readers and writers. But, as the literature and the results of this study indicate, more work is needed before the tools and technologies of web-based hypertext are mastered. Although the literature documents many benefits for hypertext readers, these benefits have associated costs in the form of cognitive overload and disorientation. Web authors must mitigate these costs by finding ways to increase text coherence and thus facilitate comprehension. The study reported here of the effect of explicitness of local navigational links sought to clarify some of the needs of online readers and to provide a basis for usable guidelines for designers of web sites. This study focused on expository text that was adapted from materials originally intended to be read linearly and that were placed on the Web as a group of interconnected, somewhat linear hypertext articles. Although this study did not find the expected benefits for explicit local navigational links, it did have findings that have immediate implications for web site designers as well as for researchers. First, confusing or ambiguous hyperlinks degrade comprehension and discourage exploration of the hypertext. Web authors should realize that the phrasing of hyperlink labels will impact the comprehensibility of their document or message and will affect whether users are willing to move out of a linear sequence of pages to other hyperlinked articles. Second, hyperlinks providing more than one type of signal promote broader exploration of the hypertext space and cause users to believe they follow more embedded links and to more positively rate the ease of use of a web site. Web authors may find it beneficial to double up on signals wherever appropriate. Third, the widespread standard of ‘next’ and ‘previous’ link labels encourages broad exploration of a hypertext space, both within and between articles, most likely because such labels are familiar to users. Web authors should respect the established standard of ‘next’ and ‘previous,’ replacing them or combining them only with clear, unambiguous labels. Fourth, regarding benefits to researchers, the use of survey results and open-ended comments gathered remotely can provide valuable data about user demographics, comprehension, and perceptions that can be critical in understanding performance data. Further, we had a response rate to open-ended questions of about 50%, which shows that subjects are willing to volunteer information, even if they have to type fully phrased responses. Fifth, webserver log files provide insight into performance results and further insight into subjectreported data. Although much has been written about the problems associated with the use of web server log files, if caching is blocked on the web server, many objections can be overcome. While web server logs cannot provide the level of detail provided by client-side software, they do allow for field study in a naturally occurring environment and can capture subjects’ navigational activity in a very unobtrusive way. The emerging field of Internet-based research is finding many advantages for the conduct of remote Internet-based studies: use of subjects in their own environment, automated data entry, greatly reduced costs, access to large sample sizes and target populations, and use of naturally occurring web sites [61]. Future researchers would find it useful to replicate this study using expository hypertext prose that is longer and more difficult, hypertext that has a clear superordinate structure and would lend Journal of Information Science, 33 (1) 2007, pp. 41–61 © CILIP, DOI: 10.1177/0165551506068144 Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on February 7, 2008 © 2007 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

57


itself more to browsing tasks and comprehension testing of superordinate information. It would be informative to obtain a cross section of the adult population, including people who are not eager to access their medical, financial, or travel information, for example, on the web and who have little experience with the web. It would be ideal to have subjects interacting with web pages that were interesting enough to hold their attention but not so personally motivating that they could succeed with poorly signaled texts. Subjects might be more motivated to participate in such studies if researchers could offer significant participation incentives. Researchers may find it interesting to replicate the current study by adding explicit labels to the generic labels of ‘next’ and ‘previous’ instead of replacing them, as we did. It would also be interesting to conduct this type of study by manipulating other factors such as link labels, placement or density, and other navigational elements (e.g. navigation columns or bars, embedded links, graphical browsers, or headings in the text). Hypertext and the web have truly revolutionized both reading and writing, and there is no turning back. As more agencies and businesses use the web as their portal to the world, the old avenues of information access are disappearing. Thus, people are increasingly forced to access information on the web, and web authors must recognize their responsibility to make online information comprehensible and easy to use. As remote internet-based research progresses in the arena of assessing web site design, more research-based guidelines will evolve to facilitate authors and thus benefit readers.

Acknowledgements The authors would like to thank the Arthritis Source team for their gracious assistance and cooperation and Dr Fredrick Matsen at the University of Washington Medical Center for donating his time and expertise in the inception and creation of the materials for this research. We would also like to thank the editor of this journal and referees for their thoughtful review of this article – their input was invaluable.

References [1] J.H. Spyridakis, An emerging model of signaling effects, Technical Communication 38(4) (1991) 600–603. [2] J.H. Spyridakis, Signaling effects: a review of the research – part I, Journal of Technical Writing and Communication 19(3) (1989) 227–240. [3] J.H. Spyridakis, Signaling effects: increased content retention and new answers – part II, Journal of Technical Writing and Communication 19(4) (1989) 395–415. [4] R.F. Lorch Jr. and E.P. Lorch, Effects of headings on text recall and summarization, Contemporary Educational Psychology 21 (1996) 261–278. [5] R.F. Lorch Jr, E.P. Lorch and W.E. Inman, Effects of signaling topic structure on text recall, Journal of Educational Psychology 85(2) (1993) 281–290. [6] Z. Zhao, T. O’Shea and P. Fung, The effects of visible link-types on learning in the hypertext environment: an empirical study, Computers in the Schools 10(3/4) (1994) 353–370. [7] L. Baron, J. Tague-Sutcliffe, M.T. Kinnucan and T. Carey, Labeled, typed links as cues when reading hypertext documents, Journal of the American Society for Information Science 47(12) (1996) 896–908. [8] T.M. Welsh, K.P. Murphy, T.M. Duffy, and D.A. Goodrum, Accessing elaborations on core information in a hypermedia environment, Educational Technology Research and Development 41(2) (1993) 19–34. [9] S. McDonald and R.J. Stevenson, Navigation in hyperspace: an evaluation of the effects of navigational tools and subject matter expertise on browsing and information retrieval in hypertext, Interacting With Computers 10 (1998) 129–142. [10] K. Khan and C. Locatis, Searching through cyberspace: the effects of link display and link density on information retrieval from hypertext on the World Wide Web, Journal of the American Society for Information Science 49(2) (1998) 176–182. [11] M.L. Bernard, S. Hull and B.S. Chaparro, Examining the performance and preference of embedded and framed/non-framed hyperlinks, International Journal of Industrial Ergonomics 35(2) (2005) 139–147. Journal of Information Science, 33 (1) 2007, pp. 41–61 © CILIP, DOI: 10.1177/0165551506068144 Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on February 7, 2008 © 2007 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

58


[12] F. Calisir and Z. Gurel, Influence of text structure and prior knowledge of the learner on reading comprehension, browsing and perceived control, Computers in Human Behavior 19 (2003) 135–145. [13] L. Salmeron, J.J. Canas, W. Kintsch, and I. Fjardo, Reading Strategies and hypertext comprehension, Discourse Processes 40(3) (2005) 171–191. [14] T. Muller-Kalthoff and J. Moller, The effects of graphical overviews, prior knowledge, and self-concept on hypertext disorientation and learning achievement, Journal of Educational Multimedia and Hypermedia 12(2) (2003) 117–134. [15] S. Puntambekar, A. Stylianou and R. Hubscher, Improving navigation and learning in hypertext environments with navigable concept maps, Human–Computer Interaction 18(4) (2003) 395–428. [16] J.D. Brinkerhoff, J.D. Klein and C.M. Koroghlanian, Effects of overviews and computer experience on learning from hypertext, Journal of Educational Computing Research 25(4) (2001) 427–440. [17] P.R. Vora, Evaluating usability of interface styles and multiple access paths in hypertext (Unpublished doctoral dissertation, State University of New York, Buffalo, NY, 1994). [18] M.J. Wenger and D.G. Payne, Comprehension and retention of nonlinear text: considerations of working memory and material-appropriate processing, American Journal of Psychology 109(1) (1996) 93–130. [19] M. Graff, Individual differences in hypertext browsing strategies, Behaviour & Information Technology 24(2) (2005) 93–99a. [20] D.K. Farkas and J.B. Farkas, Guidelines for designing web navigation, Technical Communication 47(3) (2000) 341–358. [21] D.K. Farkas and J.B. Farkas, Principles of Web Design (The Allyn and Bacon Series in Technical Communication, Longman, New York, NY, 2002). [22] S. McDonald and R.J. Stevenson, Disorientation in hypertext: the effects of three text structures on navigation performance, Applied Ergonomics 2(1) (1996) 61–68. [23] D.S. Niederhauser, R.E. Reynolds, D.J. Salmen and P. Skolmoski, The influence of cognitive load on learning from hypertext, Journal of Educational Computing Research 23(3) (2000) 237–255. [24] J. Conklin, Hypertext: an introduction and survey, Computer (1987) 17–41. [25] O.J. Retterer, Learning from a hypertext: the effect of reading interactive text containing non-sequential, associative linkages on comprehension (Unpublished doctoral dissertation, The University of Toledo, Toledo, OH, 1991). [26] P.W. Foltz, Readers’ Comprehension and Strategies in Linear Text and Hypertext (Unpublished doctoral dissertation, University of Colorado, CO, 1992). [27] S.A. Barab, M.F. Young and J. Wang, The effects of navigational and generative activities in hypertext learning on problem solving and comprehension, International Journal of Instructional Media 26(3) (1999) 283–309. [28] V. Bush, As we may think, Atlantic Monthly 76 (1945) 101–108. [29] D.C. Engelbart, A conceptual framework for the augmentation of man’s intellect. In: P. Howerton and D.C. Weeks (eds.), Vista in Information Handling: Volume I: The Augmentation of Man’s Intellect by Machine (Information for Industry, Inc., Washington, D.C., 1963) 1–29. [30] D. Becker and M.M. Dwyer, Using hypermedia to provide learner control, Journal of Educational Multimedia and Hypermedia 3(2) (1994) 155–172. [31] C.H. Lin and G.V. Davidson-Shivers, Effects of linking structure and cognitive style on students’ performance and attitude in a computer-based hypertext environment, Journal of Educational Computing Research 15(4) (1996) 317–329. [32] J.E. Gall and M.J. Hannafin, A framework for the study of hypertext. Instructional Science 22 (1994) 207–232. [33] G. Marchionini, Hypermedia and learning: freedom and chaos. Educational Technology 28(11) (1988) 8–12. [34] L. Hunter, Text nouveau: visible structure in text presentation. Computer Assisted Language Learning 11(4) (1998) 363–379. [35] S.W. Haas and E.S. Grams, Readers, authors, and page structure: a discussion of four questions arising from a content analysis of web pages. Journal of the American Society for Information Science 51(2) (2000) 181–192. [36] H.Y. Choi, The effects of hypertext and prior knowledge on learning geographic information in social studies (Unpublished doctoral dissertation, University of Illinois, Urbana, IL, 1997). [37] M. Thuring, J. Hannemann and J.M. Haake, Hypermedia and cognition: designing for comprehension, Communications of the ACM 38(8) (1995) 57–66. [38] N.C. Burbules, Rhetorics of the Web: hyperreading and critical literacy. In: I. Snyder (ed.), Page to Screen: Taking Literacy into the Electronic Era (Routledge, New York, NY, 1998, pp, 102–122). Journal of Information Science, 33 (1) 2007, pp. 41–61 © CILIP, DOI: 10.1177/0165551506068144 Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on February 7, 2008 © 2007 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

59


[39] N.P. Dalal, Z. Quible and K. Wyatt, Cognitive design of home pages: an experimental study of comprehension on the World Wide Web, Information Processing & Management 36(4) (2000) 607–621. [40] J.H. Spyridakis and C.S. Isakson, Hypertext: a new tool and its effect on audience comprehension. In: Proceedings of IEEE International Professional Communication Conference (1991) 37–44. [41] P.J. Lynch and S. Horton, Imprudent linking weaves a tangled web, Computer July (1997) 115–117. [42] D.S. Miall and T. Dobson, Reading hypertext and the experience of literature, Journal of Digital Information 2(1) (2001) Article No. 46, 2001–08–13. [43] P.R. Vora, M.G. Helander and V. Shalin, Evaluating the influence of interface styles and multiple access paths in hypertext. In: Human Factors in Computing Systems, CHI94 (Association for Computing Machinery, Boston, MA, 1994) 323–329. [44] D. Bachiochi, M. Berstene, E. Chouinard, N. Conlan, M. Danchak, T. Furey, C. Neligon and D. Way, Usability studies and designing navigational aids for the World Wide Web, Computer Networks and ISDN Systems 29 (1997) 1489–1496. [45] J.H. Spyridakis, Guidelines for comprehensible Web pages, Technical Communication 47(3) (2000) 359–382. [46] C.Y. Wei, M.B. Evans, M. Eliot, J. Barrick, B. Maust and J.H. Spyridakis. Influencing web browsing behaviour with intriguing and informative hyperlink wording, Journal of Information Science 31(5) (2005) 433–445. [47] M. Bieber, Enhancing information comprehension through hypertext. In: C. Nicholas and J. Mayfield (eds.), Intelligent Hypertext: Advanced Techniques for the World Wide Web (Springer-Verlag, Germany, 1997) 1–11. [48] S.W. Haas and E.S. Grams, Page and link classifications: connecting diverse resources. In: Proceedings of Digital Libraries ‘98–Third ACM Conference on Digital Libraries, (1998) 99–107. [49] S.W. Haas and E.S. Grams. A link taxonomy for Web pages. In: Proceedings of the 61st Annual Meeting of the American Society for Information Science, (1998) 485–495. [50] J. Nielsen, Using link titles to help users predict where they are going. (Jakob Nielsen’s Alertbox for Jan 11, 1998). Available at: http://www.useit.com/alertbox/980111.html (accessed February 22, 2006). [51] H. Potelle and J.F. Rouet, Effects of content representation and readers’ prior knowledge on the comprehension of hypertext, International Journal of Human–Computer Studies 58(3) (2003) 327–345. [52] L.D. Catledge and J.E. Pitkow, Characterizing browsing strategies in the World-Wide Web, Computer Networks and ISDN Systems 27 (1995) 1065–1073. [53] J. Park and J. Kim, Contextual navigation aids for two World Wide Web systems, International Journal of Human-Computer Interaction 12(2) (2000) 193–217. [54] H. Nishikura, The impact of content organizers and instructional objectives on learner performance in a Web-based environment (Unpublished doctoral dissertation, Arizona State University, AZ, 2000). [55] S.A. Barab, B.E. Bowdish, M.F. Young and S.V. Owen, Understanding kiosk navigation: using log files to capture hypermedia searches, Instructional Science 24 (1996) 377-395. [56] D.H. Jonassen, Effects of semantically structured hypertext knowledge bases on users’ knowledge structures. In: C. McKnight, A. Dillon, and J. Richardson (eds), Hypertext: A Psychological Perspective (Ellis Horwood Limited, West Sussex, UK, 1993) 153–168. [57] A. Balcytiene, Exploring individual processes of knowledge construction with hypertext, Instructional Science 27(3–4) (1999) 303–328. [58] J. Powell and A. Clarke, The WWW of the World Wide Web: who, what, and why? Journal of Medical Internet Research 4(1) (2002). [59] T.M. Haladyna, Writing Test Items to Evaluate Higher Order Thinking (Allyn and Bacon, Needham Heights, MA, 1997). [60] A. Oosterhof, Oosterhof Developing and Using Classroom Assessments (Prentice-Hall, Upper Saddle River, NJ, 1999). [61] J.H. Spyridakis, C. Wei, J. Barrick, E. Cuddihy and B. Maust, Internet-based research: providing a foundation for Web design guidelines, IEEE Transactions on Professional Communication 48(3) (2005) 242–260. [62] M.C. Burton and J.B. Walther, The value of Web log data in use-based design and testing, Journal of Computer Mediated Communication 6(3) (2001). [63] D. Linder, Interpreting WWW statistics (The National Archives and Records Administration, 1994). Available at: http://www.ario.ch/etc/webstats.html (accessed February 22, 2006). [64] S. Haigh and J. Megarity, Measuring Web Site Usage: Log File Analysis, (Information Technology Services, National Library of Canada, Report No. ISSN 1201–43381998). Available at: http://www.collectionscanada.ca/9/1/p1–256-e.html (accessed February 22, 2006). Journal of Information Science, 33 (1) 2007, pp. 41–61 © CILIP, DOI: 10.1177/0165551506068144 Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on February 7, 2008 © 2007 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

60


[65] J. Goldberg, Why Web Statistics are (Worse than) Meaningless, (Cranfield University, Cranfield Computer Center, UK, 1995). Available at: http://www.goldmark.org/netrants/webstats/ (accessed February 22, 2006). [66] A.J. Klockars, Evaluating pretest-posttest control group experiments, Perceptual and Motor Skills 75 (1992) 96–98. [67] G.O. Einstein, M.A. McDaniel, P.D. Owen and N.C. Cote, Encoding and recall of texts: the importance of material appropriate processing, Journal of Memory and Language 29 (1990) 566–581.


61

Explicitness of local navigational links: comprehension, perceptions of ...

Explicitness of local navigational links: comprehension, perceptions of ...

Suggest Documents

The Effect of Structural Cues on User Comprehension, Navigational ...

The Effect of Structural Cues on User Comprehension, Navigational ...

CONCLUSION EXPLICITNESS IN ADVERTISING

CONCLUSION EXPLICITNESS IN ADVERTISING

Local Perceptions of Livestock Husbandry and Rangeland ...

Local perceptions and importance of endogenous

Individual Local Farmers' Perceptions of Environmental ... - MDPI

Local Perceptions of Changes in Traditional ...

Local Perceptions of Federal Block Grants

Local perceptions of degradation in rangelands

Local health workers perceptions of substandard ... - BioMedSearch

Perceptions of Malaysian Local Government Managers on ...

Local perceptions of jaguar conservation and

Impact of the Feeling of Knowledge Explicitness in the Learners ...

The Navigational System of Birds

Monitoring Local Comprehension Monitoring in Sentence ... - CiteSeerX

Monitoring Local Comprehension Monitoring in Sentence Reading

Explicitness and Conservation: Social Class Differences

Perceptions of success of a local UK public health ...

Fear of the unknown: local knowledge and perceptions of ... - Scandlynx

Perceptions of Local Communities on the Economic Impacts of ...

TEK, local perceptions of risk, and diversity of management practices ...

The Perceptions of Local Communities towards the Conservation of ...

Perceptions of Governance - The Experience of Local ... - Swisspeace