SADIe: Structural Semantics for Accessibility and Device ... - CiteSeerX

SADIe: Structural Semantics for Accessibility and Device Independence SIMON HARPER and SEAN BECHHOFER University of Manchester

Visually impaired users are hindered in their efforts to access the largest repository of electronic information in the world, namely, the World Wide Web (web). A visually impaired user’s information and presentation requirements are different from a sighted user’s. These requirements can become problems in that the web is visually centric with regard to presentation and information order/layout. Finding semantic information already encoded directly into documents can help to alleviate these problems. Our approach can be loosely described as follows. For a particular cascading stylesheet (CSS), we provide an extension to an upper-level ontology which represents the interface between web documents and the programmatic transformation mechanism. This extension gives the particular characteristics of the elements appearing in that specific CSS. We can consider this extension to be an annotation of the CSS elements implicitly encoded into the web document. This means that one ontology can be used to accuratly transform every web document that references the CSS used to generate that ontology. Simply one ontology accuratly transforms an entire site using a generalized programmatic machinery able to cope with all sites using CSS. Here we describe our method, implementation, and technical evaluation. Categories and Subject Descriptors: H.5.4 [Information Interfaces and Presentation]: Hypertext/Hypermedia—Theory; I.7.2 [Document and Text Processing]: Document Preparation— Hypertext/hypermedia; H.1.2 [Models and Principles]: User/Machine Systems—Human factors General Terms: Measurement, Theory, Design, Human Factors Additional Key Words and Phrases: Web accessibility, semantic web, visual impairment, transcoding ACM Reference Format: Harper, S. and Bechhofer, S. 2007. SADIe: Structural Semantics for Accessibility and Device Independence. ACM Trans. Comput.-Hum. Interact. 14, 2, Article 10 (August 2007), 27 pages. DOI = 10.1145/1275511.1275516 http://doi.acm.org/10.1145/1275511.1275516

Author’s Address: Information Management Group, School of Computer Science, University of Manchester, Kilburn Building, Oxford Road, Manchester M13 9P, UK; email: simon.harper@ manchester.ac.uk. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or direct commercial advantage and that copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works requires prior specific permission and/or a fee. Permissions may be requested from Publications Dept., ACM, Inc., 2 Penn Plaza, Suite 701, New York, NY 10121-0701 USA, fax +1 (212) 869-0481, or [email protected].  C 2007 ACM 1073-0616/2007/08-ART10 $5.00 DOI 10.1145/1275511.1275516 http://doi.acm.org/ 10.1145/1275511.1275516 ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

10

Article 10 / 2

•

S. Harper and S. Bechhofer

1. INTRODUCTION Access to and movement around complex hypermedia environments, of which the web is the most popular example, has long been considered an important and major issue in the web design and usability field [Chen 1997; Furuta 1997]. The commonly used slang phrase “surfing the web” implies rapid and free access, pointing to its importance among designers and users alike. It has also been long established [Brambring 1984; Asakawa and Lewis 1998] that this potentially complex and difficult access is further complicated, and becomes neither rapid nor free, if the user is visually impaired.1 Annotation of web documents provides a mechanism to enhance visually impaired peoples’ access to information on the web through an encoding of the meaning of that information [Harper et al. 2005]. Annotations can then be consumed by tools that restructure or reorganize documents in order to pull out salient information (e.g., via some triage2 process) [Yesilada et al. 2004]. However, when working in the real world, there are issues we must face. Empirical evidence suggests that authors and designers will not create separate semantic mark-up to sit with standard extensible hypertext mark-up language (XHTML3 ) because they see it as an unnecessary overhead [Regan 2004]. In addition, designers will not compromise their desire to produce “beautiful and effective” websites. In our conversations with designers [Harper et al. 2004] the resounding message we receive is “If there is any kind of overhead above the normal concept creation then we are less likely to implement it. If our design is compromised in any way we will not implement. We create beautiful and effective sites, we’re not information architects.” Many web designers move from print media to web design and this pregained experience in creating static designed artifacts forces them to see design as fixed and immovable, once created. A designer creates and controls the development of what is in effect a piece of art that therefore once created should not be changed or violated. It can be difficult to convey that users often require web documents to adapt to their needs, and the fact that this sometimes goes beyond art. We therefore approach our research from an application, as opposed to an infrastructure, standpoint [Harper and Bechhofer 2004] and use the implicit structural meaning of the documents, visual presentation to solve the problems of transcoding accuracy balanced with automatic annotation [Gupta and Kaiser 2005]. In effect, we try to bridge the gap between designers who wish to create visually pleasing sites, and visually impaired users who wish to interact with such sites. However, our technology can also be used to assist users of other devices which limit cognition of information. For 1 Here

used as a general term encompassing the World Health Organization’s definition of both profoundly blind and partially sighted individuals [RNIB 1996]. 2 Sorting and allocating information on the basis of need for or likely benefit. 3 XHTML is the lingua franca for publishing hypertext on the World Wide web. It is a nonproprietary format based upon SGML. More information can be found at http://www.w3.org/MarkUp/. ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Structural-Semantics for Accessibility and Device Independence

•

Article 10 / 3

instance, with the move to small screen size, low bandwidth, and different operating modalities, users of mobile web devices [Seeman 2004] face similar sensory4 and cognitive limitations as found in our initial target user group. Paradoxically, it is the designers who help to solve this problem because, as we shall see, they implicitly include labels as part of their design. These labels are normally encoded into the class or id attributes of XHTML elements (see Figure 11) and form the “glue” between the cascading stylesheet (CSS,5 see Figure 12) and the XHTML document structure. By current convention, when an XHTML file is accessed on the web browser, there is a limited visual presentation available. However, before the document is displayed the CSS file is downloaded and applied. Then, by linking these class and id labels along with selected XHTML elements, an independent visual rendering of the XHTML document which does not alter the document structure is generated. In SADIe we use these labels as a semantic annotation [Harper and Bechhofer 2005] and relate them to structural semantics such as footer, menu, content, heading, etc. We take these to be implicit and accurate annotations and use them to deliver better content to users with sensory or cognitive impairments (in this case profound blindness). To be clear, we do not substitute different CSSs, as this only has the effect of altering the visual presentation (not used by accessibility technology), not the deeper XHTML document structure (which is used by accessibility technology). Indeed, our solution uses ontologies, semantic infrastructure, and programmatic document object model (DOM) parsing to accomplish DOM transcoding. The overarching vision of the web [Berners-Lee 1999] is of a web in which the underlying meaning and structure of resources are made explicit using representations that are amenable to machine processing. The consideration of the problem outlined previously leads us to the question: “Can implicit structural semantic information be harvested from the Syntactic Web such that the document is as accessible to visually impaired users as it is to sighted users, without compromising the document’s design vision?” In this article we explain and evaluate our technical solution in light of a new concept called “emergent semantics.” Our proposed approach known asSADIe 6 can be summarized as follows. We provide an ontology that describes the meaning of elements found within CSS metatags and associate this with the data found in documents through the use of “class” and “id” attributes common to most XHTML elements. In this way, CSS presentation will be unaffected but semantics will be an explicit part of the data. We can then provide tools that 4 Mobile

phone users suffer from a keyhole view of a webpage similar a user with tunnel vision: loss of peripheral vision with retention of central vision, resulting in a constricted circular tunnel-like field of vision, and, by extension, a very narrow point of view. Also called tubular vision. 5 Cascading stylesheets are a simple mechanism for adding style (e.g., fonts, colors, spacing) to web documents. More information at http://www.w3.org/Style/CSS/. 6 Structural Semantics for Accessibility and Device Independence. SADIe was developed from our original project called LLIS. ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Article 10 / 4

•

S. Harper and S. Bechhofer

Fig. 1. The Mac Observer (see http://www.macobserver.com/ for the original).

consume this information, manipulating the documents and providing appropriate presentations to the user. 1.1 The Problem, and Paradoxically, the Solution Recent moves towards a separation of presentation, metadata, and information such as CSS has helped to alleviate some of the problems of access to complicated visual information by visually impaired users, but there are still many issues to be addressed. For example, consider “The Mac Observer”7 (see Figure 1). This site is a model of the state-of-the-art, using standards and a clear separation of content and presentation, resulting in some visually stunning web documents. However, the site still remains relatively inaccessible to visually impaired people. This is because visually impaired users interact with these systems in a “serial” (audio) rather than “parallel” (visual) manner. XHTML content, as opposed to its visual rendering after the CSS is applied, is accessed by assistive technologies from top-left to bottom-right, there is no scanning, and its progress through information is slow. Additionally, the information is rendered in an order defined by the designer and not that required 7 http://www.macobserver.com/

ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Structural-Semantics for Accessibility and Device Independence

•

Article 10 / 5

Fig. 2. The Mac Observer code (see http://www.macobserver.com/ for the original).

by the user. Given this interaction paradigm, we can see that visually impaired users are still at a disadvantage because they have no idea which items are menus, what the document layout is, what the extent is, and where the focus (locus of attention [Raskin 2000]) of the information is. In effect, the implicit meaning contained in the visual presentation is lost and any possibility of enhanced meaning is also not available. Even when CSS concepts look as though they have a meaning with regard to the information, there is no way of relating this, due to the lack of machine interpretable semantics. Consider Figure 2, which shows a view into the XHTML code of the Mac Observer. We can see that the names given by the designer to certain areas of the document are highly structurally semantic. Labels such as: Headlines (Figure 2 at 2), News (Figure 2 at 1), Content (Figure 2 at 4), and Todays (Figure 2 at 3) all have implied meaning on their own, and taken together as suggested by the block structure of CSS, have even more resonance (TodaysContent (Figure 2 at 3 and 4), Headlinelist-Hot (Figure 2 at 5). Allowing these semantics to emerge enables us to use them for sectioning document objects into more appropriate combinations for use by visually impaired people. 1.2 Synopsis The article can be summarized as follows: Background. Adding semantics to an XHTML document is not a new concept. Work dates from the late 1990’s and concrete solutions were proposed as ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Article 10 / 6

•

S. Harper and S. Bechhofer

early as 2002. This technology domain is important to our work, as it places our contribution in context. Here we give a brief overview of annotation, transcoding, and semantics, and go on to investigate emergent semantics in an effort to inform a discussion regarding our work. We describe the problems associated with existing work and give an overview of why our system is both different and unique. Our Emergent Approach. We describe the concepts, rationale, and techniques behind our approach, focusing on the emergent techniques used. We show how these are referenced on XHTML and CSS documents and how our emergent system can contribute to the accessibility of information via semantics already encoded into the XHTML document. As a case study we consider “The Mac Observer” and “Blogger” (both sites created before the SADIe tool) for which we have allowed ontologies to emerge; we also show how our Firefox application can transform documents into more accessible forms. Finally, we present a technical evaluation of our system to support our conclusions. Conclusion. We have discussed our system using emergent semantics which may eventually enable semantic information to be freely accessible by all users. By knowing the meaning of the structure of the information that is being encountered, users can interact with this information more easily. Here, we discuss our conclusions from a viewpoint of the work undertaken and look to the future. 2. BACKGROUND Many solutions exist which attempt to make accessible web documents out of inaccessible ones. These approaches include guidelines, automated validation, and best practice, however, they rely on designers building new web documents and being concerned enough about usability to follow them. Problems still exist with badly built and old documents and so solutions which aim to change bad content to good have been developed. These solutions are known collectively as “transcoding.” Transcoding. Simply, transcoding is a technology used to adapt web content so that it can be viewed on any of the increasingly diverse devices found on today’s market. Transcoding in this context normally involves syntactic changes [Hori et al. 2000] or text-block rearrangements and fragmentation [Myers 2007; Textualize 2005]. Systems are often based along similar lines and address set problems; some are annotation based [Hori et al. 2000], others generate text-only versions [Myers 2007; Textualize 2005], some filter the content [WebCleaner 2004] and others are specifically used for small scale device interaction [Buyukkokten et al. 2000]. Whatever system is used, it invariably does not transform all inaccessible elements [Pontelli et al. 2002] but just a subset, leaving holes in the accessibility of it transcode. Annotation. Most transcoding techniques normally rely on an annotation [Takagi and Asakawa 2000] of the target web document. The goal of such annotation is to provide better support, either for audio rendering and thus for visually impaired users, or for visual rendering in small-screen devices [Takagi ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Structural-Semantics for Accessibility and Device Independence

•

Article 10 / 7

and Asakawa 2000]. There are a number of different ways that transcoding can take place. In one example, the original material (e.g., an HTML document) is analyzed by a program that creates a separate version containing annotations. The annotations include information that will instruct the reformatting process, and inaccessible elements will be removed or altered. However, annotation is expensive in terms of user time because each document must be annotated with information regarding its structural context. This information must be applied to each document manually before the transcoding can begin [Hori et al. 2000]. Of course, some systems exist which attempt to transcode documents based on annotation being automatically included in documents (normally from databasedriven sites with templates for document creation) [Buyukkokten et al. 2000]; but these often rely on tailor-made technologies and solutions only appropriate for one site. In Summary. Each of these types of transformations are fraught with problems with regard to the acceptability of the resulting generation. This is especially the case when sighted users as well as visually impaired users wish to use the same document. Automatic transcoding based on removing parts of the document results in too much information loss and manual transcoding is difficult when applied to dynamic websites. Most systems use their own tailor-made proxy servers or client-side interfaces and these systems require a greater setup cost in terms of user time. Finally, some systems require tailor-made automatic annotation by a content generator and so are not useable by every user and all systems. Transcoding systems often lean towards solving the problems of one user group and so destroy the content/structure/context for other, nontarget groups. Although there may be no solution to this outcome, it directly challenges the nature of the web and the philosophy of hypermedia systems in general. 2.1 Semantics The semantic web vision, as articulated by Tim Berners-Lee [1999], is of a web in which resources are accessible not only to humans, but also to automated processes. General overviews of recent semantic web developments can be found in a number of sources (e.g. Fensel et al. [2003] and McIlraith et al. [2004]). The key idea is to have data on the web defined and linked in such a way that its meaning is explicitly interpretable by software processes rather than just being implicitly interpretable by humans; and a key component of these software processes is the “reasoner.” Reasoners are description logic (DL) classifiers that can also be used for modal logic satisfiability testing. A reasoner service takes the statements encoded (asserted) in an ontology as input and derives (infers) new statements from them. In particular, reasoners can be used to reveal subclass/superclass relationships among classes, determine the most specific types of individuals, and detect inconsistent class definitions. Standard languages such as OWL [McGuinness and van Harmelen 2004] have now been defined for the representation of ontologies and metadata, and both tools and infrastructure to support the editing, maintenance, and deployment of ontologies are appearing. A challenge now facing the semantic web community is to ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Article 10 / 8

•

S. Harper and S. Bechhofer

provide convincing demonstrations of the utility of ontologies, and of their use in real systems and applications. OWL provides a mechanism which supports interoperation of applications at not just a syntactic level (through the adoption of common XML-based formats), but also at a semantic level. It is this provision of well-behaved and well-specified inference procedures based on shared semantics that provides the machine understandability of the semantic web. Semantic transcoding. Semantic transcoding aims to overcome some of the problems of using tailor-made systems and technology. Using this approach, the semantics provide the machine understandability and knowledge reasoning, and the transcoding provides the transformation technique [Yesilada et al. 2004]. Indeed, the “HearSay” system [Ramakrishnan et al. 2004] uses taxonomies of terms to help identify document elements within the page. This approach has met with some success, however, is not flexible, as a unique taxonomy is needed for each knowledge domain. Therefore, current systems are at present limited to document analysis where a document built after a set template can be analyzed and transformed by semantic or semantic-like technologies [Huang and Sundaresan 2000]. 2.2 Semantics in Web Documents—The Standard Solution Adding semantics to an XHTML document is not a new concept. Work dates from the late 1990’s, however, concrete solutions were proposed as early as 2002. Berners-Lee proposed embedding XML RDF8 in HTML documents as part of the tag project Berners-Lee [2002], but these documents would not validate as XHTML and so did not find favor among the community [Kew 2002]. A version was created that did validate through the inclusion of a small DTD using XHTML modularization. However, this was not deemed a good solution, as unique extensions have to be created on a whim. In fact, the work concluded that the RDF specification specifies how to understand the semantics (in terms of RDF triples) in an RDF document that contains only RDF, but does not explain how and when one can extract semantics from documents in other name-spaces which contain embedded RDF. It went on to say that the XHTML specification explains how to process XHTML name-space content, but gives no indication about how to process embedded RDF information [Berners-Lee 2002]. Other methods have been proposed in which the object or script elements are used. However, the code becomes unreadable and therefore less workable, although the RDF can be linked to an external file [Palmer 2002]. Use of the XHTML link element has also been proposed, but the main problem with this method is that the RDF is not actually then embedded in the HTML source, but in a separate file [Palmer 2002]. This file is then at the mercy of changes and synchronization issues with the original and the amount of work needed to create the resource is the same as creating two separate and disjoint files, thus time and effort are not saved. 8 Resource

description framework.

ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Structural-Semantics for Accessibility and Device Independence

•

Article 10 / 9

Connolly proposed the HyperRDF system, in which HTML is used as the conduit to use XSLT9 to transform information into RDF. However, HyperRDF cannot be validated, since the head element does not allow an ID attribute [Connolly 2000]. Augmented metadata for XHTML is an implementation that allows Dublin core metadata to be incorporated in web documents in a way that is compatible with today’s web browsers. The basic premise is that one can take the profile attribute to be a global name-space prefix for all of the rel/meta and name attributes throughout the document. This approach is mainly for those authors that want to use a simple mechanism for producing RDF from their XHTML. It is ineffective from the point of view of anyone that wants to randomly extract RDF from XHTML, since one cannot tell whether the author wanted the assertions to be converted into the triples produced by the algorithm [Altheim and Palmer 2002]. Finally, the most recent thinking on the subject comes in the form of GRDDL (gleaning resource descriptions from dialects of languages). This work is being undertaken by the W3C web coordination group and is a mechanism for encoding RDF statements in XHTML and XML. GRDDL has some commonalities with HyperRDF and works on the principle that the HTML specification provides a mechanism for authors to use particular metadata vocabularies, thereby indicating the author’s intent to use these terms in accordance with the conventions of the community that originated them. Authors may wish to define additional link types not described in this specification. If so, they should use a profile to cite the conventions used to define the link types. GRDDL is one of those profiles which uses XSLT to transform a document to an RDF description. As far as our desires and requirements are concerned, GRDDL does not offer an ideal solution. As discussed in Section 1, a key requirement of ours is to support designers and thus ensure the target group is supported by the designer’s creation. GRDDL is about embedding extra information in a document through a modification of that document. In contrast, our approach differs in that although we are interested in associating semantic information with the document, the aim is not to embed extra metadata or semantics in the document, but to try and make use of the existing information already present and expose it in a more explicit fashion. This is similar to the deep annotation approach proposed by Volz et al. [2004], where annotation of a logical schema can lead to annotation of resources or web documents that are dynamically generated from a database. 2.3 Semantics in Web Documents—The Emergent Solution Emergent semantics is a relatively new field10 being derived from the older one of emergent computing [Forrest 1991], which proposes that complex systems may be discernible from the task environment itself, as opposed to being derived from an architects over-arching view [Brueckner et al. 2005]. The idea behind 9 Extensible

stylesheet language transformations. around 2005.

10 Established

ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Article 10 / 10

•

S. Harper and S. Bechhofer

emergent computation is that systems, can emerge with interesting global behavior created from local interactions among the component parts. In these systems, interactions among simultaneous computations are exploited to improve efficiency, increase flexibility, or provide more realistic models of natural phenomena. Emergent computation uses bottom-up techniques to derive systems, instead of the more normal prescriptive methods [Wermter et al. 2001]. This process is analogous to the so-called self-organization of chemical processes such as the Belusov-Zabotinsky reaction, or to physical processes such as laser oscillation [Dogaru 2003]. In reality, this view is supported by the study of requirements analysis and systems requirements engineering, which is performed at the start of many tailor-made software systems and lets the complexity of the system emerge from the people who actually use and implement it [Hatley et al. 2000]. Emergent semantics uses the layered structure of many complex systems to suggest that semantic heavy systems can be created by using exactly the bottom-up approach as suggested by emergent computation. Letting the semantics emerge by observing interactions between the information and the user/creator of that information can give invaluable insight into both the tacit and implicit processes and interactions between the two. These semantics would normally be lost using a top-down prescriptive approach and so emergent semantics can be useful to capture these interactions. Indeed, because information is highly evolutionary in that existing documents may be updated, added to, or deleted, the semantics of that information may change without any physical amendment to the data. Moreover, semantics that have been created using a top-down approach may become incorrect if applied to subsequently changed data sources [Staab et al. 2002]. Conventionally, semantic solutions start with the creation of an ontology which is then populated with concepts. This top-down approach is very prescriptive and forces conformance of an often preexisting system to a new ontology with that ontology as the heart of the development. Our emergent approach is different in that we observe the natural coding and behavior of designers and authors. By looking for the implicit meaning of the elements of the document structure and visual presentation, we can understand the designer’s intention. This may be obvious by the meaning of the label created to link the XHTML structure to the CSS structure, or by study of the visual presentation combined with an innate knowledge of the meaning of the structural elements within the context of that presentation. In effect, we allow the meaning of the elements to emerge from their web resources and create an ontology from this emergence. 3. OUR EMERGENT APPROACH Our approach can be loosely described as follows. An upper-level ontology provides basic notions that encapsulate the role of document elements. For example, we can characterize an element as a menu or header. In addition, the ontology contains further information describing the characteristics of those elements, along with properties that can be used to describe them. For example, we have the notion of a removableCSSComponent-one which can be removed ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Structural-Semantics for Accessibility and Device Independence

•

Article 10 / 11

Fig. 3. The SADIe system architecture overview.

without significantly impacting on the information carried within the document. This upper-level ontology is defined in isolation from a particular site, providing an abstraction over the document structure. For a particular CSS stylesheet, we provide an extension of this upperlevel ontology giving the particular characteristics of the elements appearing in that stylesheet. We can consider this extension to be an annotation of the stylesheet elements: It provides information telling us, for example, whether particular elements in the stylesheet can be considered to be removable or important [Harper and Bechhofer 2005]. In effect, we are observing interactions between the designer and the design brief as expressed within the implicit structural annotations of the XHTML (see Figure 3 at 2) and CSS (Figure 3 at 1) of the web document. These annotations often show us how and what the designer was thinking at design time, but even in the worst case where the annotations seem to make no sense, the structural segmentation present in the designers mind can still be derived. The SADIe tool is unique in that it uses an ontology11 created from a preexisting CSS to triage XHTML components; an overview of the system can be seen in Figure 3. The CSS (Figure 3 at 1) is used to create concepts in the ontology (Figure 3 at 3). These concepts are marked with attributes like isRemovable, and new container classes like removableCSSComponent are created. By providing appropriate definitions for such concepts along with descriptions of specific concepts in the ontology, we can achieve the effect of classifying all removable concepts beneath removableCSSComponent. Classification functionality is provided by an ontology service 11 Currently,

this is created manually. However, we are in the process of building an automatic generation engine. ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Article 10 / 12

•

S. Harper and S. Bechhofer

(Figure 3 at 5). The SADIe application (Figure 3 at 4) then retrieves the ontology much like Firefox retrieves the CSS document. The ontology is passed to the ontology service12 and SADIe can then ask questions based on the actions required. The ontology and reasoner are both instrumental to SADIe because they give manipulation and transcoding accuracy, concept checking, a common application-ontology interface, and generic tools for semantic creation. But, most importantly, we get scalability, as one ontology is used for all pages referencing the CSS from which the ontology was created (in the case of “Blogger” this is currently 1:2,000,000). Moreover, we get flexibility of manipulation, as neither the SADIe nor site-specific ontology is statically connected to the DOM transcoding machinery. In our current prototype, requests and operations are preconfigured and “anchored” to buttons on the toolbar. Basic functionalities include: Defluff. Remove any unnecessary content from the document. Visually impaired users have problems navigating structures which are really “visual candy” such as banners, blank images that provide visual spacing, and advertisements. Removing these has been shown to increase reading speed and cognition [Ivory and Megraw 2005; Wood et al. 2006]. Toggle menu. Toggle menus on and off to remove the need for skip links. Menus often occur partway through XHTML document structures; the visual rendering of the CSS is then used to anchor them to a defined visual location. As visually impaired users do not access this rendering, menus are often obfuscated. Toggling menus has the effect of grouping all menus and moving them to the top to allow navigation, or to the bottom to allow cognition of the content [Norman 1991; Harper et al. 2003]. Reorder. Bring important document items to the top. The order and groups required for the visual presentation of information is often different from the order in which they would logically occur in the XHTML document. By suggesting a priority for each element within the ontology, we can group and move elements into more appropriate positions at a “document level” in the XHTML structure [Hanson 2004]. When an operation is selected, an appropriate request is sent to the ontology service. In defluffing, for example, all of the removable items are requested. The service complies and SADIe parses the DOM, looking for and discarding removable components (Figure 3 at 4). In this way a transcoded document is produced (Figure 3 at 6). Our solution is not limited to static sites and can cope with dynamic changes in content. Furthermore, our transformations do not break any of the target webpages. Finally, user-side security does not present problems, as transcoding is performed on the DOM residing on the client after it has been requested. In this way password protection and secure transfer protocols do not affect the process. While SADIe can cope with changes to websites (and their semantics), 12 An

ontology service accepts ontologies and reasons over them, often reclassifying the asserted ontology into an infered ontology in which concepts are grouped according to a set of rules and properties.

ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Structural-Semantics for Accessibility and Device Independence

•

Article 10 / 13

Fig. 4. The SADIe system ontologies overview.

there may be problems if there are incorrect semantics encoded within the web document. While the reasoner will catch inconsistencies in the semantics of the ontology, missing or changed XHTML–CSS “glue” will cause the process to fail if the ontology is not likewise updated, just as the CSS will fail if there are changes in the XHTML part of the “glue” which are not cascaded into the CSS. 3.1 Upper-Level Ontology Our approach involves an annotation on CSS elements in order to describe their properties. In this way, all documents created using the same CSS can be modified by using this one simple ontology and tool. The particular ontology is specific to the site. However, the upper-level definitions, used by the tool in order to determine which elements are to be removed, are generic; integrating an additional site into the system simply requires the definition of a mapping from the CSS elements of the site into the base SADIe ontology. Any site’s documents can be defluffed, so long as the removable elements are identified. We do not need to hardwire any information regarding the CSS elements into the application; this is encoded in the ontology, which is then used by the application (see Figure 4) that parses the DOM looking for relevant XHTML “glue” defined in the site ontology. The upper-level ontology describes concepts relevant to the process of triage, for example, menu, and the application’s behavior is specified in terms of these concepts. Site-specific extensions describe their CSS elements in terms of these upper-level concepts, and particular documents are then instantiated against this extension. Any site using a CSS-based approach could be processed in this way, even more so if the sites are dynamically generated using templates (although static sites are still amenable to processing using our approach). Our system is in reality a process for associating ontology concepts with instances encoded within XHTML documents. The instances here are instances of document structure elements rather than real-world objects, however. The system revolves around a software process which converts an XHTML document into a series of instances and ontological concepts. Users view the document in a web browser as normal, but, browsers that are “semantic aware” ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Article 10 / 14

•

S. Harper and S. Bechhofer

Fig. 5. The SADIe upper-level triage ontology.

Fig. 6. SADIe upper-level triage ontology (description logic to be viewed in combination with Figsures 5 and 14). Each symbol has a general English meaning and for our purposes these simplified meanings are: (1)  can be read as “is a kind of ”; (2) ≡ ∃ can be read as “is anything that”; (3) items within { } denote a value; and (4)  can be read as “top level”.

can use the ontological information to provide more intelligent access to the instances of information. The approach is nonintrusive and works hand-in-hand with existing technologies used to control presentation. CSS often contains information which is inherently “semantic,” but which is not necessarily presented in a manner amenable to machine processing. This is, we feel, a clear example of a problem that semantic web technology and approaches are intended to represent and that emergent semantics helps to uncover, as there is no explicit characterization of the semantics of these tags, and they are thus opaque to understanding by machine. By providing a mapping from these elements to a shared upperlevel ontology (pictured in Figure 5 and described in Figure 6) of document elements, we can provide the opportunity for applications to manipulate documents in appropriate ways. We can see from Figure 5 that the SADIe upper-level ontology is a simple taxonomy with some lightweight object properties (see Figure 6). The ontology is split into removable and unremovable components. Components that are not ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Structural-Semantics for Accessibility and Device Independence

•

Article 10 / 15

removable can then be a menu and have priorities low, medium, or high. We can see from further analysis of Figure 6 that there exists a set of object properties. This means that concepts in the emergent ontologies can assert values for these object properties. As we shall see later, these assertions will enable inferred classification of concepts within the site-specific emergent ontologies once they have imported the SADIe upper-level ontology. Because our system works with CSS, our method is unsuitable for legacy sites that do not use CSS to separate presentation from structure. However, it is suitable for all sites, old or new, which do separate style and presentation in this manner. Our system is therefore backward compatible with the vast majority of modern (maintained) websites, and the number of sites that can be transformed in this way is increasing daily. One key feature is that because we do not annotate or modify the actual XHTML document, our system does not force developers into costly and time-consuming reengineering to achieve backward compatibility. 3.2 Building Emergent Ontologies Our solution uses both bottom-up and top-down approaches to create a working system. As we have seen, an initial upper-level ontology is created as a kind of interface between SADIe and the pluggable ontologies created from emergent techniques. The question now is becomes “How can we easily create these pluggable ontologies?” The answer to this question is exactly where the emergent approach helps. By a technologist simply looking at the CSS and making value judgements (in concert with the designer) as to whether the CSS element is either removable, or a menu and its order, an ontological structure emerges. Rather than explicitly building a subsumption (kind-of) hierarchy explicitly, concepts can be described, and the subsumption relationships between these descriptions then inferred. The process of constructing such hierarchies is often referred to as classification. OWL supports the description of concepts in terms of both necessary and sufficient conditions. This is in contrast to traditional frame-based systems where, in general, necessary but not sufficient conditions can be described. For example, we can define the notion of a removableCSSComponent as being one for which the isRemovable property is set to true. Any concepts which are then described as having that property/value combination will then be classified as being kinds of removableCSSComponent. This is a simple example, but more complex situations involving Boolean combinations of concepts and quantification of relationships (e.g., a menu-only document is one which only contains elements that are kinds of menus) can be supported. The semantics ensures that the interpretations of these combinations (and thus the inferences that can be drawn) are consistent. As a case study, we consider two legacy sites for which we have created two sample ontologies. We show how our Firefox application can transform documents into more accessible forms. CSS stylesheets are used to control the presentation of both these sites. In this way, a large number of documents can be delivered with almost identical structure, but with a widely differing “look and feel.” As we shall see, it is this similarity in structure that we exploit, by ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Article 10 / 16

•

S. Harper and S. Bechhofer Table I. Mac Observer CSS Elements (Abridged)

ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14

CSS Element div#topbanner div#mainstuff div#news div.rightbox div.rightbox p, form div#sidebarz div#sidebarz .rightfloat div#sidebarz .sideadvert div#sidebarz .sideadvert img .sideadvert div#footer div#footer img .headlines .extras

Emergent Structural Semantics Top Site Banner (Removable) Main Content (NOT Removable) News Items (NOT Removable) Side Items (Removable) Side Items (Removable) Sidebar (Removable) Sidebar (Removable) Advert (Removable) Advert Picture (Removable) Advert (Removable) Legal, Copyright, Padding (Removable) Legal, Copyright, Padding (Removable) Headline Section (NOT Removable) Additional Information (NOT Removable)

Derived from http://www.macobserver.com/I.

providing a mechanism that allows us to annotate at the CSS level. A single annotation is then applicable to a large number of documents. 3.2.1 The Mac Observer Case Study. Consider the Mac Observer website and the abridged CSS elements listed in Table I and Figure 7. Even on an initial pass, some implicit semantic assumptions/observations can be made. It is obvious that an element named news (ID 3) is a news story item, or that text containing the word “advert” such as div#sidebarz .sideadvert; div#sidebarz .sideadvert img or .sideadvert, is advertising. It then becomes a value judgement as to whether this piece of information is useful (and therefore not removable) or a menu item. Its priority is also decidable by an investigation of the visually rendered document (see Figure 7): Those lower down the visual rendering or rendered as sidebars off to the right or left would have lower priority than those in the main content and at the top. Indeed, so alike are some of the CSS element descriptions across different websites that heuristics can be used to identify these descriptions. For instance, news sites often use CSS element descriptions such as headlines; news; article; author; date; time; etc. These descriptions can often be thought of as not removable. However, descriptions like advert; sidebar; footer; and notes may be considered superfluous to the main narrative or objective of the document when being displayed on devices with limited-screen real estate or when used by accessibility devices. Therefore, by using heuristics we could make an initial attempt to assist the automatic bottom-up generation of pluggable ontologies. Let us consider Figures 8 and 9. Figure 8 is the asserted MacObserver emergent ontology with the SADIe upper-level triage ontology already imported. At this stage we can see how these two ontologies still exist separately with no linkage after owl:Thing. Indeed, the MacObserver section of the ontology is an exact mirror of the CSS block hierarchy.13 However, once passed through the 13 http://altmedia.macobserver.com/tmo_media/css/styles.css

ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Structural-Semantics for Accessibility and Device Independence

•

Article 10 / 17

Fig. 7. Sectioned composite of the “MacObserver” linked to Table I (see http://www.macobserver. com/ for the original).

ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Article 10 / 18

•

S. Harper and S. Bechhofer

Fig. 8. The asserted MacObserver ontology (with the SADIe ontology already imported).

Fig. 9. The inferred MacObserver ontology (with the SADIe ontology already imported). ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Structural-Semantics for Accessibility and Device Independence

•

Article 10 / 19

Fig. 10. The MacObserver after transcoding.

reasoner,14 an inferred hierarchy showing a classified taxonomy is generated (see Figure 9). Here we can see that concepts such as topbanner, footer, and sidebarz are all removable and other concepts such as extras, navlist, and headlines are not. Now by associating a programatic function (actioned from a Firefox toolbar button) to, say, the inferred RemovableCSSComponent, we can obtain a list of all removable document objects present and process them as required; in this case SADIe just deletes them. We can now see the SADIe transformation route from start to finish. Figure 1 shows the MacObserver site before transformation; we then create an asserted MacObserver ontology (see Figure 8), and submit it to SADIe to be inferred (see Figure 9), and any MacObserver document using that CSS for style can be accurately transcoded (see Figure 10). 3.2.2 The Blogger Case Study. To further test our system we created a second pluggable ontology and populated it with web logging terms from “Blogger”15 (see Figure 13). This ontology comprises a small set of concepts and subconcepts derived from the Blogger CSS template (see Figure 12). Some of these concepts were described as being removable, and a measure of importance was assigned using integer values. 14 FaCT++:

in this case http://owl.man.ac.uk/factplusplus/

15 blogger.com

ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Article 10 / 20

•

S. Harper and S. Bechhofer

Fig. 11. Example code from blogger.com.

Again, the hierarchical (subclass) relationships for the class removableCSSComponents were inferred using a reasoner and showed that footer, profile-container, and sidebar can all be removed. Interestingly, the XHTML document contains two concepts (recently and archive-list) which have no CSS entry but which are used as CSS-class identifiers in Blogger (see Figure 11). Thus, there is no extra presentational information associated with elements using these identifiers. These two concepts enclose the recent posts list and archive month lists and so, in fact, act like menus into previous postings. Axioms, asserting that the concepts recently and archive-list are subclasses of menu, can be added to the ontology. Our application can then treat recently and archive-list as kinds of menus and perform appropriate operations upon them. Again, this is an example of the explicit specification of the information content of the document. One final aspect of validation did emerge when creating these kinds of inferred hierarchies. Consider the concept sidebar (see Figure 13), which has a subsumption (read “is-a-kind-of”) relationship with content. Content has been classified as having a HighPriority and is therefore a NotRemovableCSSComponent concept. However, on further investigation sidebar has also been classified as a RemovableCSSComponent concept. Adding axioms to the ontology stating the disjointness of the NotRemovableCSSComponent and ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Structural-Semantics for Accessibility and Device Independence

•

Article 10 / 21

Fig. 12. Example stylesheet code fragments from blogger.com.

RemovableCSSComponent will allow a reasoner to detect such situations. The designer and technologist should then remodel the ontology to address the inconsistency. 4. EVALUATION In order to explore the viability of our proposed approach, we conducted a technical evaluation and a small preliminary user investigation. However, for the purposes of this investigation, we make an assumption that proposed transformations such as removal of unnecessary items or reordering of menus are useful operations in improving accessibility based on previous work within the field (see Section 3 of Ivory and Megraw [2005], Wood et al. [2006], Norman [1991], Harper et al. [2003], and Hanson [2004]). Nonetheless, we do freely accept that a systematic user evaluation is required and is part of our further work plan. 4.1 Technical Evaluation This work is novel and therefore an evaluation of the application of the techniques in a programmatic setting is the first step towards a qualitative user evaluation. After all, if we can support our methodology for web document transformation, we can manipulate our programatic outputs based on user evaluation later. Therefore, the purpose of this technical evaluation was to explore how easy it is to apply our methods to existing websites, and how effective the transformations then are. Using W3C guidelines [Chisholm et al. 1999; Caldwell et al. 2005] and taking into account what SADIe was designed to do, as well as using the findings of the research discussed earlier, a series of benchmarks could be produced against which we could compare the outcome of using SADIe on a web document. We used 8 categories: corporate sites; content and ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Article 10 / 22

•

S. Harper and S. Bechhofer

Fig. 13. The inferred Blogger ontology (with the SADIe ontology already imported). View in combination with Figure 14.

Fig. 14. Abridged Blogger ontology focusing on isRemovableComponent (description logic to be viewed in combination with Figures 6 and 13).

media; search engines; web hierarchies and directories; portals; e-stores; virtual hosting; and universities [Amitay et al. 2003] to gain some confidence that our evaluation uses a reasonable sample of the kinds of websites that potential users of SADIe may access. A total of 5 websites from each category were selected, giving a total of 40 sites in the sample. The W3C guidelines specify that when evaluating a website the entry point should be tested, as this is the first document that the users will access. Therefore, the samples include the site entry point (usually index.html) of the website, plus 4 other randomly chosen documents therein. This gave us a total of 5 documents per website. With 40 websites, we examined 200 web documents in total. We did not separate out CSS and non-CSS designed websites and so included sites that would give an increased error rate to maintain the randomness of our sample. This meant that we had 9 site failures out of 40, giving a 23% failure ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Structural-Semantics for Accessibility and Device Independence

•

Article 10 / 23

rate. However, for CSS-only websites (a subset of these sites can be found at: http://rpc81.cs.man.ac.uk:8080/sadie/Mappings) we achieved an 8% failure rate (mainly occurring through errors in the application of the CSS). We are basing success and failure on how the document is transformed after transcoding. In this case, before SADIe was applied, we noted how we wanted each page to look after transformation using all permutations of the SADIe transformation mechanisms. We then used these expected effects as hypotheses to decide on success or failure. Indeed, our approach is testable and our technical evaluation verifiable by third parties at http://rpc81.cs.man.ac.uk:8080/sadie/. 4.2 Preliminary User Investigations The approach proposed in SADIe involves applying transformations to webpages. However, what transformation techniques are appropriate, and how do they relate to the document structure as identified? In this case, our transformations are based on those from the existing literature with extensions and enhancements that make use of the rich description of page structure provided by the ontology. Visually impaired users often adopt coping strategies [Yesilada et al. 2006] in order deal with information that is hard to access. An analysis of such coping strategies has inform our initial development of appropriate transformation techniques. We started informal investigations within the community by demonstrating our techniques and tools at the 8th International ACM SIGACCESS Conference on Computers and Accessibility [Harper et al. 2006]. By soliciting informal comments from attendees who had a particular expertise in this area, we hoped to assess support for our approach. In general, we found overwhelming support, both for our approach and our initial transformations (20 attendees participated, including two profoundly blind users). While this investigation was in no way conducted under scientific experimental conditions, it did serve as an informal formative investigation which goes some distance towards supporting continued development of SADIe . 4.3 Further Evaluations Preceding further evaluations, we will investigate and document the area surrounding coping strategies. Future SADIe transformations will then attempt to replicate these strategies. Finally, our future evaluations will proceed along two strands. A technical evaluation will attempt to validate the suitability of any proposed solution in terms of its flexibility, scalability, and ability to support the proposed transformations. A user evaluation will attempt to determine how well the resulting transformations help users in accessing information on pages. Success criteria will be measured in terms of the ability of test subjects to access information without resorting to our forthcoming documented coping strategies. Both evaluations will require the identification of a collection of websites to be used, along with a group of evaluation subjects. 4.4 Initial Finding While we can make a valued judgement that the transcoded document will be more accessible based on research in the field, a true user evaluation will be ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Article 10 / 24

•

S. Harper and S. Bechhofer

needed before we can be sure of SADIe’s success. While these assumptions need to be addressed, the initial results are promising. 5. CONCLUSIONS This article describes work towards a more elaborate system to enable structural semantic information to be freely accessible by all users and devices. Only by knowing the meaning of the structural information that is being encountered can users perform their own triage on that information. Annotation and triage can help to make information more accessible via restructuring of documents. Unnecessary items that introduce clutter can be removed, while important items can be promoted to a position on the document where they are encountered earlier by user agents. To do this in a principled manner, however, requires that the implicit structural semantics of the document be made explicit. We have described an approach based on the “hidden” annotation of web documents, which encode implicit semantic information that can then be used by tools in order to manipulate and present web documents in a form that provides easier access to content. The annotations use an emergent ontology describing the basic semantic units found in the documents as described in stylesheets. Annotations are allowed to emerge from these stylesheets, and are made directly to stylesheets, allowing the annotation of large numbers of similar documents with little effort. The approach is minimal in overhead presented to the site designer. No constraints are made on the ways in which the layout and presentation of the site can be produced. This is one of our key requirements; as discussed in the article, designers will ignore, or at the very least fight against, initiatives that compromise their work. Instead, we make use of the fact that CSS elements are identified in the document; in a large number of cases, these elements do in fact correspond to “meaningful” units of information. In addition, the approach makes no impact on the validation of XHTML documents. We plan to expand our work in a number of directions: (1) We plan to extend the upper-level ontology to include more concepts covering document constructs, along with the specification of further triage operations. (2) There is a tremendous commonality between mobile and accessible webs. We intend to extend our work on emergent structural semantics to the mobile web. (3) We plan a user evaluation of the tool to determine how effective the transcoding changes are at really increasing accessibility for visually impaired users. In summary, we propose that allowing the emergence of implicit structural semantic information from the XHTML is the only way to assist users’ access to web documents while neither increasing nor compromising the creation activity of authors and designers. By knowing the meaning of the structural information that is being encountered, users can perform their own triage on that information, and therefore the content. ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Structural-Semantics for Accessibility and Device Independence

•

Article 10 / 25

Our work addresses important real-world issues within the field of emergent data semantics, including the creation of a replicable rationale, along with a prototype tool to enable implicit structural semantics to emerge and become explicit. We believe this work represents a significant contribution to web accessibility, interoperability, and mobile information services. REFERENCES ALTHEIM, M. AND PALMER, S. B. 2002. Augmented metadata in XHTML. http://infomesh.net/ 2002/augmeta/ AMITAY, E., CARMEL, D., DARLOW, A., LEMPEL, R., AND SOFFER, A. 2003. The connectivity sonar: Detecting site functionality by structural patterns. In Proceedings of the 14th ACM Conference on Hypertext and Hypermedia (HYPERTEXT). ACM Press, New York 38–47. ASAKAWA, C. AND LEWIS, C. 1998. Home page reader: IBM’s talking web browser. In Proceedings of the Conference on Closing the Gap. BERNERS-LEE, T. 2002. RDF in HTML. http://www.w3.org/2002/04/htmlrdf BERNERS-LEE, T. 1999. Weaving the Web. Orion Business Books. BRAMBRING, M. 1984. Mobility and orientation processes of the blind. In Electronic Spatial Sensing for the Blind, D. H. Warren and E. R. Strelow, Eds. Dordrecht, 493–508. BRUECKNER, S. A., SERUGENDO, G. D. M., KARAGEORGOS, A., AND NAGPAL, R. 2005. Engineering SelfOrganising Systems: Methodologies and Applications. Lecture Notes in Computer Science / Lecture Notes in Artificial Intelligence. Springer. BUYUKKOKTEN, O., MOLINA, H. G., PAEPCKE, A., AND WINOGRAD, T. 2000. Power browser: Efficient web browsing for PDAs. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press, 430–437. CALDWELL, B., CHISHOLM, W., SLATIN, J., VANDERHEIDEN, G., AND WHITE, J. 2005. Web content accessibility guidelines 2.0. www. http://www.w3.org/TR/WCAG20/. CHEN, C. 1997. Structuring and visualising the www by generalised similarity analysis. In Proceedings of the 8th ACM Conference on Hypertext and Hypermedia. ACM Press, New York, 177– 186. CHISHOLM, W., VANDERHEIDEN, G., AND JACOBS, I. 1999. Web content accessibility guidelines 1.0. http://www.w3.org/TR/WCAG10// CONNOLLY, D. 2000. HyperRDF: Using XHTML authoring tools with XSLT to produce RDF schemas. http://www.w3.org/2000/07/hs78/ DOGARU, R. 2003. Universality and Emergent Computation in Cellular Neural Networks. World Scientific Series on Nonlinear Science, Series a, 43. World Scientific. FENSEL, D., SYCARA, K., AND MYLOPOULOS, J., eds. 2003. Proceedings of the 2nd International Semantic Web Conference (ISWC). Lecture Notes in Computer Science, vol. 2870. Springer (Sanibel Island, FL). FORREST, S. 1991. Emergent Computation. The MIT Press, Cambridge, MA. FURUTA, R. 1997. Hypertext paths and the www: Experiences with Walden’s paths. In Proceedings of the 8th ACM Conference on Hypertext and Hypermedia. ACM Press, New York. GUPTA, S. AND KAISER, G. 2005. Extracting content from accessible web pages. In Proceedings of the International Cross-Disciplinary Workshop on Web Accessibility (W4A). ACM Press, New York, 26–30. HANSON, V. L. 2004. The user experience: Designs and adaptations. In Proceedings of the International Cross-Disciplinary Workshop on Web Accessibility. (WYA) 1–11. HARPER, S. AND BECHHOFER, S. 2005. Semantic triage for increased accessibility. IBM Syst. J. 44. 637–648. HARPER, S. AND BECHHOFER, S. 2004. Lightweight mark-up for low cost semantics. In Proceedings of the ISWC Workshop on Knowledge Markup and Semantic Annotation (Semannot) (Hiroshima, Japan) Siegfried Handschuh et al., eds. HARPER, S., BECHHOFER, S., AND LUNN, D. 2006. Sadie: Transcoding based on CSS. In Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility. ACM Press, New York, 259–260. ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Article 10 / 26

•

S. Harper and S. Bechhofer

HARPER, S., GOBLE, C., AND STEVENS, R. 2005. Augmenting the mobility of profoundly blind web travellers . New Rev. Hypermedia 11, 103–128. HARPER, S., GOBLE, C., AND STEVENS, R. 2003. Traversing the web: Mobility heuristics for visually impaired surfers. In Proceedings of the 4th International Conference on Web Information Systems Engineering (WISE), T. Catarci et al., eds. IEEE Computer Society, Los Alamitos, CA, 200–209. HARPER, S., YESILADA, Y., AND GOBLE, C. 2004. Accessible layout—The tension between accessibility and visual design. In Proceedings of the International Cross-Disciplinary Workshop on Web Accessibility (W4A), S. Harper et al., eds. ACM Press, New York. HATLEY, D. J., HRUSCHKA, P., AND PIRBHAI, I. A. 2000. Process for System Architecture and Requirements Engineering. Dorset, New York. HORI, M., KONDOH, G., ONO, K., ICHI HIROSE, S., AND SINGHAL, S. 2000. Annotation-Based web content transcoding. In Proceedings of the 9th International World Wide Web Conference on Computer Networks. North-Holland, Amsterdam, Netherland, 197–211. HUANG, A. W. AND SUNDARESAN, N. 2000. A semantic transcoding system to adapt web services for users with disabilities. In Proceedings of the 4th International ACM Conference on Assistive Technologies (Assets). ACM Press, New York, 156–163. IVORY, M.Y. AND MEGRAW, R. 2005. Evolution of web site design patterns. ACM Trans. Inf. Syst. 23, 4, 463–497. KEW, N. 2002. Why Validate? http://lists.w3.org/Archives/Public/www-validator/ 2001Sep/0126.html MCGUINNESS, D. L. AND VAN HARMELEN, F. 2004. OWL web ontology language overview. W3C recommendation, World Wide Web Consortium. http://www.w3.org/TR/owl-features/. MCILRAITH, S. A., PLEXOUSAKIS, D., AND HARMELEN, F. V., eds. 2004. Proceedings of the 3rd International Semantic Web Conference (ISWC). Lecture Notes in Computer Science, vol. 3928. Springer, (Hiroshima, Japan). MYERS, W. 2007. BETSIE:BBC education text to speech internet enhancer. British Broadcasting Corporation (BBC) Education. http://www.bbc.co.uk/education/betsie/. NORMAN, K. L. 1991. The Psychology of Menu Selection: Designing Cognitive Control at the Human/Computer Interface. Ablex. PALMER S. B. 2002. RDF in HTML: Approaches. http://infomesh.net/2002/rdfinhtml/. PONTELLI, E., GILLAN, D., XIONG, W., SAAD, E., GUPTA, G., AND KARSHMER, A. I. 2002. Navigation of HTML tables, frames, and XML fragments. In Proceedings of the 5th International ACM Conference on Assistive Technologies (Assets). ACM Press, New York, 25–32. RAMAKRISHNAN, I. V., STENT, A., AND YANG, G. 2004. Hearsay: Enabling audio browsing on hypertext content. In Proceedings of the 13th International Conference on World Wide Web (WWW). ACM Press, New York, 80–89. RASKIN, J. 2000. The humane interface (book excerpt). Ubiquity 1, 14, 3. REGAN, B. 2004. Accessibility and design: A failure of the imagination. In Proceedings of the International Cross-Disciplinary Workshop on Web Accessibility (W4A). 29–37. RNIB. 1996. A short guide to blindness. Booklet. http://www.rnib.org.uk. SEEMAN, L. 2004. The semantic web, web accessibility, and device independence. In Proceedings of the International Cross-Disciplinary Workshop on Web Accessibility (W4A). ACM Press, New York, 67–73. http://doi.acm.org/10.1145/990657.990669. STAAB, S., SANTINI, S., NACK, F., STEELS, L., AND MAEDCHE, A. 2002. Emergent semantics. IEEE Intell. Syst. 17, 78–86. ISSN: 1541-1672 . TAKAGI, H. AND ASAKAWA, C. 2000. Transcoding proxy for nonvisual web access. In Proceedings of the 4th International ACM Conference on Assistive Technologies. ACM Press, 164–171. TEXTUALIZE. 2005. Textualize manual. Codix.net;. http://codix.net/solutions/products/ textualise/index.html. VOLZ, R., HANDSCHUH, S., STAAB, S., STOJANOVIC, L., AND STOJANOVIC, N. 2004. Unveiling the hidden bride: Deep annotation for mapping and migrating legacy data to the semantic web. J. Web Semantics 1, 2 (Feb.), 187–206. WEBCLEANER. 2004. WebCleaner. http://webcleaner.sourceforge.net. WERMTER, S., AUSTIN, J., AND WILLSHAW, D. 2001. Emergent Neural Computational Architectures Based on Neuroscience: Towards Neuroscience-Inspired Computing. Lecture Notes in Computer Science/Lecture Notes in Artificial Intelligence. Springer. ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.

Structural-Semantics for Accessibility and Device Independence

•

Article 10 / 27

WOOD, S., COX, R., AND CHENG, P. 2006. Attention design: Eight issues to consider. Comput. Hum. Behav. 22, 4, 588–602. YESILADA, Y., HARPER, S., GOBLE, C., AND STEVENS, R. 2004. Screen readers cannot see (ontology based semantic annotation for visually impaired Web travellers). In Proceedings of the Web Engineering 4th International Conference (ICWE), N. Koch et al., eds. Lecture Notes in Computer Science, vol. 3140, Springer, 445–458. YESILADA, Y., STEVENS, R., HARPER, S., AND GOBLE, C. 2006. Evaluating DANTE: Semantic transcoding for visually disabled users. ACM Trans. Comput.-Hum. Interact. To appear. Received April 2006; revised November 2006; accepted November 2006

ACM Transactions on Computer-Human Interaction, Vol. 14, No. 2, Article 10, Publication date: August 2007.