Towards the Use of Ontologies for Improving User ... - CiteSeerX

Towards the Use of Ontologies for Improving User Interaction for People with Special Needs Shuaib Karim, A. Min Tjoa Vienna University of Technology {skarim, amin}@ifs.tuwien.ac.at

Abstract. Formal description of concepts so that it may be processed by computers has great promises for people with special needs. By making use of ontologies, improved user interaction with personal information management system is possible for these people. An ontology using semantic web technology is proposed which formally describes the mapping information about user’s impairments, and the available interface characteristics. Effort is made to enhance accessibility at a generic level by making it possible to enrich the ontology for a diverse range of users. Consequently users with all types of special needs are able to get already customized interfaces. Especially, the possible adaptation to our prototype Personal Information Management System SemanticLIFE [1] is the trigger for this investigation.

1 Introduction The end user can conveniently perform the tasks through UI (User Interface) provided mental models of both the designer and the user, and the implemented system image are coherent with each other [2]. Practically, this triangle is never perfect due to lack of shared semantics at each point. The situation is aggravated in case of users with special needs. Accessibility for these people can be improved in many ways, e.g., improving physical access (mobility impaired people using wheel-chairs require information on accessible hotel rooms, lifts, ramps), sensory access (hearing or visually impaired people need tactile markings or hearing augmentation) & communication access (people having difficulties with written text, vision, speech, and language) [3]. Ontologies are one of the key enabling technologies for exploring the information in numerous possible ways [4]. Their exploitation for accessibility would accelerate the inclusion & improve the eQuality for people with special needs in the digital world. It is possible to formally specify the concepts in ontologies & hence remove the ambiguities so that the interaction is made conformant with the needs of the task at hand, & user abilities. The output is presented using additional devices & techniques called the Assistive Technology like screen reader, electronic Braille, screen magnifier, usage of Alt text, sticky keys etc according to the Web Content Accessibility Guidelines (see WAI1). It is noticeable that these guidelines mainly focus on vision impairment & to some extent on motor impairments. Even for visually impaired people mostly total blindness is assumed, whereas, there are many types of vision

problems, e.g., low vision, color blindness, problem in which the person can not recognize the images even his own. Other impairments are not significantly addressed by these guidelines [5], e.g., person with memory problems, mobility impairments etc. The implementation of existing guidelines, work well for visually impaired people provided the information is well structured. For static textual information contents on web pages this strategy is already practiced but still there are unresolved issues. Currently the mapping of impairments with accessibility guidelines is hard coded either directly in HTML documents or in style sheets. The problem is severe where one has to navigate the ever changing information like on the web & explore the hyperlinks with possibility of each time going into a different navigational structure. The issue is similar in our system SemanticLIFE which is managing the information using ontologies for making associations of concepts. These associations represent the additional knowledge which was not present before. The human users had to read the contents & make sense of it themselves. Now all users, especially the people with special needs are in a better position to access this added explicit knowledge. The accessibility guidelines will be fed in the ontology along with the impairments and interface data. The link between them is automatically managed by the ontology. The ontology management is made under user control, be it the user himself or his / her caregiver. In the next section, related work is discussed. Next, the accessibility requirements for our system are briefly described, followed by our suggested approach for building “Impairment-User Interface” ontology. Then some important consequences of our ontology, and conclusions and future work are described.

2 Related Works Haystack project [6] has successfully used RDF for designing a component architecture that provides rich & uniform UI. It consists of four parts, i.e., layout, informative, decorative, and view. The UI components are populated based upon the ontology contents, the metadata for whom is stored in RDF. The semantic information items to be presented are associated with appropriate view part in RDF. Here we can incorporating our accessibility ontology while implementing components. Ontologies are used in [7] for providing accessibility in context of web page annotations for conveniently navigating the visual structure of web pages. Their ontology consists of semantics about mobility (travel objects such as way points, orientation points, & travel assistants), authoring (header, logo, label, heading, footnote, section) & context (information seeking, surveying, orientation, navigation, browsing). The WAI1 provides WCAG guidelines to encode some of the above information semantics but not all. Also, the accessibility is provided only for visually impaired users. The recent work by [8] for increasing accessibility of web pages, suggests an approach to encode the semantic information of the page directly into the page itself by introducing lightweight markup, without compromising the creative activity of authors and designers. An ontology is created representing the meaning of data in XHTML metatags and then encoding this meaning into the data. The relevant CSS 1

Web Accessibility Initiative of the W3C http://www.w3.org/WAI/ (4th Mar’06)

remains unaffected, while the semantics become implicit part of data. It works for pages with available CSS. Only simple instantiations with property assertions is possible. Usage of metatags reserved for other purposes is not a stable solution. But it highlights the possible XHTML enhancement with new tags for including semantics. Different kinds of user policies (navigation, exploration, presentation, etc.) are introduced in [9] for making the presentation & interaction of XML contents accessible. The user policies are supposed to take into account the user’s physical & cognitive impairments by encoding the relevant information in XML style sheets. This is a nice and practical approach for keeping separate the original document contents & the user preferences. This can be combined with Haystack’s design to improve accessibility. An organized set of steps is described in [10] for designing ontological-driven semantic applications such as our prototype system SemanticLIFE. An additional step could be to incorporate accessibility using ontologies. The WWAAC2 project by the European Union is trying to integrate the assistive technology, the web, and the signs and symbol language used traditionally by people with communication problems for text interpretation. The accessible UI complexity is highlighted as a challenging issue to tackle due to conflicting needs by different users. Also, ontologies are used to exchange semantics between the concepts databases of symbols and the assistive technology. The framework is composed of Concept Code Definitions (a plain concepts list), Base Reference Ontology (concepts mapped from WordNet3 to used symbols) and (Complementary Reference Ontology (specifies missing concepts). Different user groups have been successfully tested against this framework in [11]. The proprietary ontologies like Assistive technology Ontology need to communicate with the framework by mapping against Reference, and Complementary ontologies using APIs. The role of user impairments is fixed. Our impairments ontology can be used for associating concepts from Concept Code Definitions to specific impairments so that presentations are customized at run time. The human disease is conceptualized around Type (disorder types), Symptom (signs/indicators), Cause (genetic/environmental), and Treatments (surgery, drug therapy, physiotherapy, etc.) in [12]. The therapy can be extended to incorporate the notion of interface therapy / adaptation for convenience and rehabilitation of impaired user. The concepts of our interest are type (impairment type with associated properties to determine the severity of the impairment), and the treatment in context of UI.

3 Accessibility Requirements In order to make an accessible UI, one has to list down nature of the system, the user group, nature of data, tasks users need to perform, and their ability or disability to perform their tasks. Although this list is not exhaustive, however it serves our purpose for demonstrating the positive impact of ontologies on the UI.

World Wide Augmentative & Alternative Communication http://www.wwaac.org/(4th Mar’06) 3 http://wordnet.princeton.edu/ (4th Mar’06) 2

The input to our prototype system is user’s lifetime information items captured by independent data feed modules. Those range from emails, documents of various types, calendar data, contacts, process state data, browsed web pages, chat sessions etc. The primary goal of our visualization is to explore the associations of information items. The secondary goal is to visualize their contents to explore further associations. The associations are of three types. Firstly, these are based upon the already available metadata of an information item, for example Exif header fields for pictures, and email header information. Secondly, the annotations given by the user, for example email1 is related with file1. Thirdly, the associations discovered from the contents of the information items, e.g., finding the string “eclipse svn problems” in an email body, and associating it with web page “http://subclipse.tigris.org/”. Users of the system may vary from normal user to a user having some special needs. Since it is not a collaborative but a personal system, therefore a personalized UI would fulfill the user’s requirements better than a universal interface. However, using appropriate style sheet the UI can be adjusted for any user. The UI should be conformant with the accessibility checklist mentioned in WAI1.

4 Suggested approach The mapping between the information about user’s impairments & available interface characteristics is formally described in an ontology. Then simple subsumption mechanism can be applied based on class-subclass hierarchy & sophisticated inference mechanisms by description logic statements for automatic interface adaptation. According to our requirements we have followed suitable guidelines from various ontology development approaches [13] [14]. The activity is divided in two steps, i.e., finalizing general ontology characteristics, and designing its contents. However the ontology development, especially the content design phase is to be iterative in nature. General characteristics: It is important to know the purpose, and scope of the ontology. Then its possible interaction with other ontologies is specified, followed by a description of users, some motivating scenarios, and a list of competency questions. Purpose and Scope: The ontology is about user’s physical and cognitive impairments, and the related consequences on the UI. It will be used in context of user’s personal information management system. The objective is to exploit the impairments data so that user’s interaction with the system is optimized in terms of improved UI. The aim of this ontology is to make improvements for all the impaired users in a generic way instead of focusing only on the stereotypical cases of impaired users. By exploiting this generic ontology, improved personalized interaction for all could be possible to accommodate diversity. It is a domain specific ontology and will not cover general world or common sense knowledge in the initial attempt. Interaction with other ontologies: In our current prototype, the impairments ontology will be required to interact with existing UI ontologies. Other possible interac-

tions could be with DOID4, UMLS5, ICD9CM6, and MeSH7. Sample extracts from DOID and MeSH are: [Term] id: Doid: 3203, name: Blindness of both eyes, impairment level not further specified, rank: 7, xref_analog: Umls_Cui: C0271217, xref_analog: Icd9cm_2005: 369.00, xref_analog: Umls_Icd9cm_2005_Aui: A0241434, is_a: Doid: 3204 [Term] id: Mesh: A.01.456.505.420, name: eye, synonym: "eye" [], synonym: "ophthalmologic_effect" [], synonym: "ophthalmological_effect" [], is_a: Mesh: A.01.456.505 ! face, is_a: Mesh:A.09 ! sense_organ Users and User Scenarios: The ontology will be used by the end user or his / her caregiver in case of severe impairments like cognitive problems. User interaction is not only limited to traditional UI part. This ontology is usable in many ways, e.g., a. The ontology will be used for customizing / adapting the UI according to user impairments. Each interface component has a certain affordance of use [15]. If the user is impaired to use it in that way, then it should not be part of the UI. Instead a component with better affordability would be suggested for this user, e.g., • Color blindness can be Red-Green, Blue-Yellow, and Monochromacy (complete inability to distinguish any color). The confusing colors on an interface can be avoided for a particular type of user. • Visual acuity refers to the clarity of one’s vision, a measure of how well a person sees. Font size can be adjusted according to the user’s visual acuity. • Blindness in one half of the visual field normally due to stroke. For this user, the information should only be presented on the better half of the screen. b.

In tourism domain, the ontology can provide accessibility for a better travel planning. But, it is not the major focus for this paper. If user is mobility impaired, then while searching route from point A to point B the availability of accessible lifts and restrooms can be shown on transits, & also the corresponding time calculations for the journey (e.g., accessibility of http://www.vor.at/)

Questions to be answered: Some competent queries are framed which must be answered by our ontology. For example, is UI component (such as vertical scrollbar) suitable for the user? Otherwise find the alternate interface component. What is suitable font size? Which colored control buttons are suitable? What is the most suitable screen area for presenting information (right, left or central)? Is lift facility available on a specific underground station? Are textual descriptions available using Braille for a certain museum etc? Finally, as part of evaluation, all the queries should be satisfied and should give correct, consistent, and reliable results without any regression. 4

http://obo.sourceforge.net/cgi-bin/detail.cgi?disease_ontology (4th Mar’06) 5 http://www.nlm.nih.gov/research/umls/ (4th Mar’06) 6 http://icd9cm.chrisendres.com/ (4th Mar’06) 7http://cvs.sourceforge.net/viewcvs.py/obo/obo/ontology/vocabula ries/mesh.obo (4th Mar’06)

Design of ontology contents: Three approaches for designing the ontology contents are mentioned in [14], i.e., top-down, bottom-up, & mixed. We followed the topdown approach which calls for collection of terms, finding/making hierarchy of terms, & finding synonyms. This is followed by identification of concepts, their attributes, restrictions, cardinality, & interrelations of concepts. Important terms and their organization: The intention of this activity was to find the concept hierarchies & their interconnections. We also interviewed some physicians8 for this activity. A sample taxonomy is made & is available in technical report9.

Fig. 1. Part of Impairment-User Interface ontology

Classes and the class hierarchy: Extending the work from [12] an ontology is created using OWL-DL in Protégé 3.1. The main extension is the InterfaceAdaptation (see Fig. 1) as another treatment for impaired users of computer systems.

8 9

Private communications with Zubair Kareem, M.D. Phy-Neurologist (Holyoke, MA) Impairments Taxonomy http://storm.ifs.tuwien.ac.at/publications/ImpTax.pdf (2nd May’06)

“Impairment” is a type of disease which needs to be treated. As a result of impairment the concerned person can become partially or completely reliant on some one. The sub-classes of “Impairment” and other classes are managed with the help of given properties. As a sample case, visual acuity is taken care of by binding it to appropriate text sizes. The sub-classes of “VisualAcuity” are associated with “AvailableText” by specifying restriction for its property “recommendedText”. While entering the user’s impairments data (using instance of “Person”) user’s visual acuity instances would be linked with suitable text size. Then it can be used for displaying text on the interface for this user. A very simple case is chosen as a proof of concept.

5 Consequences The consequences of this ontology are numerous. Primarily, it will be helpful in adapting the UI for a specific user, e.g., specifying a suitable text size. It can also be used in deducing the best match for a user with multiple impairments, e.g., if user has a very low visual acuity and also he can not focus on left part of the screen then it is a composition of impairment classes and related interface options. Also, the historical data generated by users would help to get an insight about the evolving cause-effect relationship between the impairments and the computer interfaces.

6 Conclusions and Future Work Presenting information according to user’s special needs is a complex task. In absence of a generic solution each type of special need results in different interface adaptation effort. The contributing factors, i.e., the impairments (medical domain), interfaces (computer and assistive technology domain), and the consequences of impairments on interface (accessibility guidelines) are brought together in form of Impairments-User Interface ontology. The demonstration by using a test case shows that the ontology could provide a generic step in improving universal accessibility. Our future work will focus on incorporating other impairments. Available ontologies will be used after transformation, if their format is not OWL. Currently, reasoners only work on ontology classes. Later, we also plan to apply those on individuals. Our approach & projects like USERfit [16] (generating usability specifications using assistive technology), & GADEA [17] (UI development framework adapted to human cognition diversity) may also help & complement each other in future.

Acknowledgements The work is supported by Higher Education Commission of Pakistan, and ASEANEU University Network Initiative. Special thanks to Zubair Kareem, M.D. PhyNeurologist for useful remarks on impairments & its potential use for rehabilitation.

References 1. Ahmed M., Hoang H.H., Karim M.S., Khusro S., Lanzenberger M., Latif K., Michlmayr E., Mustofa K., Nguyen H.T, Rauber A., Schatten A., Tho M.N. and Tjoa A.M., ’SemanticLIFE’ - A Framework for Managing Information of A Human Lifetime, Proceedings of the International Conference on Information Integration, Web-Applications and Services, IIWAS’04, 27-29th Sep. 2004 Jakarta-Indonesia. 2. Dix A., Finlay J., Abowd G., and Beale R., 1993. Human-Computer Interaction, PrenticeHall, New Jersey. 3. Pühretmair F., It’s Time to Make eTourism Accessible, Proceedings of the 9th International Conference on Computers Helping People with Special Needs ICCHP’04, LNCS 3118 (2004), pp. 272–279. 4. Garcia E. and Sicilia M-A., User interface tactics in ontology-based information seeking. PsychNology Journal, volume 1(3) December 2003, pp 242–255. 5. Poulson D. and Nicolle C., Making the internet accessible for people with cognitive and communication impairments. International Journal Universal Access in the Information Society, volume 3(1) March 2004, pp 48 – 56. 6. Karger D.R., Bakshi K., Huynh D., Quan D. and Sinha V, Haystack: A Customizable General-Purpose Information Management Tool for End Users of Semistructured Data, In CIDR, January 2003, California-USA. 7. Plessers P., Casteleyn S., Yesilada Y., Troyer O.D., Stevens R., Harper S. and Goble C., Accessibility: A web engineering approach. Proceedings of the Fourteenth International Conference on World Wide Web, Sep, 2005, pp 353–362, Chiba-Japan. 8. Harper S. and Bechhofer S., Semantic Triage for Increased Accessibility. IBM Systems Journal issue 44-3, 2005, pp. 637-648. 9. Encelle B. and Baptiste-Jessel N., Adapting Presentation and Interaction with XML Documents to User Preferences, Proceedings of the 9th International Conference on Computers Helping People with Special Needs ICCHP’04, LNCS 3118 (2004), pp. 143–150. 10. Sheth A. and Avant D, Semantic visualization: Interfaces for exploring and exploiting ontology, knowledgebases, heterogeneous content and complex relationships, NASA Virtual Iron Bird Workshop, (2003), CA-USA. 11. Nicolle C., Osman Z., Black K. and Lysley A., Learning from Internet Requirements of People with Communication Needs, Proceedings of the 9th International Conference on Computers Helping People with Special Needs ICCHP’04, LNCS 3118 (2004),pp.121-128. 12. Hadzic M. and Chang E., Ontology-based Support for Human Disease Study, Proceedings of the 38th Hawaii International Conference on System Sciences, HICSS’05. 13. Kalyanpur A., Hashmi N., Golbeck J. and Parsia B., Lifecycle of a Casual Web Ontology Development Process, The 13th International WWW’04 Conference: Workshop on Application Design, Development and Implementation Issues in the Semantic Web, (May 2004). 14. Noy N.F. and Hafner C.D., The State of the Art in Ontology Design, AI Magazine, 18(3), 1997, pp. 53-74. 15. Norman D., the Design of Everyday Things.Doubleday: New York, 1990. 16. Abascal-González J., Arrue-Recondo M., Garay-Vitoria N., Tomás-Guerra J. and VelascoNuñez C.A., USERfit Tool: A Design Tool Oriented towards Accessibility and Usability, UPGRADE The European Journal for the Informatics Professional – Human-Computer Interaction: Overcoming Barriers, vol. IV, issue 1, 2003, pp. 5-11. 17. González-Rodríguez M., Moral-Pérez E.D., Paule-Ruiz M.D.P. and Pérez-Pérez J-R, GADEA: a Framework for the Development of User Interfaces Adapted to Human Cognition Diversity, UPGRADE The European Journal for the Informatics Professional – Human-Computer Interaction: Overcoming Barriers, vol. IV, issue 1, 2003, pp. 27-30.