Semantic Matching Framework for Personalizing ubiquitous Environment dedicated to people with special needs Rachid Kadouchel, Mounir Mokhtari2, Sylvain Giroux', Bessam Abdulrazak 2
1 DOMUS Lab Universit6 de Sherbrooke, Sherbrooke, Qu6bec, Canada Handicom Lab, Institut National des T6l6communications-GET, Evry, France
[email protected]
Abstract:
User adaptability and acceptability in heterogeneous environments are very important issues, mainly for people with disabilities. Traditional technology-push approaches fail in overcoming the needs. It becomes crucial to adopt a user-centric approach, both from methodological and points views. technological of In this paper we present a semantic matching framework for personalized service delivery in assistive environments. This framework is based on matching two models: (1) environment model, describing environment factors and (2) user model, describing the human factors. The Framework was implemented using semantic web technologies and integrated into a demonstrator, which has been used to validate the concept in laboratory conditions. Our contribution is three folded: first, the formalization of the user model and the environment model, second, the matching between these two models combining the human factors and the environmental factors, and third the description of these models and their association with ontology based
languages.
1. Introduction
of Personalization the environments accessibility is an important factor to enhance autonomy and quality of life for people with special needs (PwSN') in daily living. The quality of life would benefit from smart homes and pervasive computing designed under the "assistive environment" paradigm and can experience significant enhancements due to the increased support received from the environment. This support includes facilities for environmental control, information access, communication, monitoring, etc., and built over various and emerging existing technologies. Nevertheless, PwSN are usually confronted to accessibility barriers due to the user's disability. These problems include both, PwSN elderly people and people with disabilities 978-1-4244-2020-9/08/$25.00 02008 IEEE 582 -
-
physical difficulties to handle input devices, and cognitive barriers to understand and reach suitable functionalities. The main goal of our research study is focused on the human deficiency study that allows to identify the user's limit according to each environment, then personalize the environment access. We mean by environment, the infrastructure and the devices that are around the users. We believe that novelty is not located at an engineering level, where most of technological problems could be solved, but mainly on the usability issue where human behavior should interfere with available functionalities of any system. This multidisciplinary approach focused on user modeling, which should guide future research work on context awareness in the pervasive computing community. The challenge is to understand the interaction between the human and his environment to design an ubiquitous assistive environment dedicated to PwSN (people with disabilities and ageing). This interaction was well defined in the model proposed by Fougerollas [1]. He defined the sociological parameters linked to the user, and the technical parameters associated to the environment. The paper focuses on formalizing this interaction that could be supported by any technological system, based on clinical, sociological, and usage analysis studies.
1.1. Handicap situation:
Disability is not only a physical deficiency, but it results mainly from the inadequacy of the environment infrastructure to fulfil the user needs. This is defined as "handicap situation" which indicates that disability doesn't design the individual, but mainly to the inadequacies of the living environment in regard to the user's requirements. Main questions are how to identify the differences between a person having severe disabilities and a one without a disability in terms of daily activities? Or even which deficiency of the user is causing the handicap situation and what are the environmental elements becoming an obstacle for the user?
And finally, how can an ubiquitous environment compensate, partly or totally, disability of each person. Each of these questions are strongly dependent on human behavior that should be supported by any assistive system. In our work, we aim at designing a user modeling methodology that allows defining the user limitation capabilities. Our goal is first to define the accessibility rate of each user and second to increase this rate by an adapted personalization process.
2. Related Work:
Personalization in smart spaces often focuses on customizing the interface of a terminal (screen color, font size, etc.) and adapting environment's services to the user preferences and location. MyCampus [2] is one of the first personalizing systems in smart environment. According to the location and profile, students access online course materials, check account balances, view grades.. .etc. The CoolAgent system [3] aimed at providing intelligent services to mobile users in their professional and personal lives such as user's agenda. These systems do not advise users on devices that can be used, they just inform users about services provided by the devices. In general, these systems focus on a specific application targeting a specific disability, such as enhancing the contrast of the user interface screen in case of people with color blindness or other visual impairments. In our case, constraints are not only the adaptation of the service to the user, but also studying the usability issue of the service provided by the environmental devices. The ultimate goal is to propose a generic personalization framework dedicated to developers dealing with application sensitive to the parameters related to the user (human factors) and to the environment (environment factors).
3. General approach and concept
We designed an architecture that allows to identify the interaction facts that generate the handicap situation in the user's daily activities. This architecture handles two types of factors (1) User factors and (2) Environmental factors: 1) User factors include user's characteristics that enable the handicap situation (fig 1) it is represented by the user model. 2) Environmental factors include devices characteristics defined as effectors, which belong to an environment, which is defined by the environment model.
Fig 1. Handicap Production Process
The user's daily living activities is a set of tasks and actions involving the different environment effectors. It is the interaction between user and Environmental (figure 1). This interaction is defined as matching relation. Particularly it is related to the user's characteristics and to particular effectors properties that leads to the handicap situation. It also defined by the inadequacy between the capacities of the user (user factors) and the environment infrastructure (environment factors).
3.1. Motivational scenario:
In order to highlight the technical problems and related issue, let's consider the following scenario for the rest of the paper: Pierre is a person on wheelchair, with a spinal cord injury involving four limbs impairments. He has a hand force of 5 Newton and a hand workspace of 3 cm3. In this paper, we consider the hand force, as the user Pushing Hand force and hand workspace as the volume space of the hand posture. Pierre met his new friend Francois a month ago and he went to visit him at home. He is a very active and social person. When he arrives, he would prefer knowing all accessible tasks and services he could perform in the different area inside the home. He also would like to know the different effectors that lead him to a handicap situation.
3.2. The user model
The user model gathers the human factors and supports different user profiles, particularly users with special needs. This model is built upon categories to represent different user model attributes (fig 2a). In order to support users with disabilities, and to perform the matching, our user-model based attributes, is extensible and is designed to capture two main categories: features and behaviors. The features are the peculiarities and distinctive aspects that differentiate one user from another while the behaviors are the actions or reactions of the user in response to external or internal stimuli. Figure 2b presents a user model instance which is a particular attributes for - 583 -
Pierre's user profile. For readability of the figure, we have limited the number of attributes to the type of disability, type of technical aids, hand force and hand workspace, all of them enable to perform the matching and susceptible to provide a handicap situation.
structured into several categories (fig.3a). Each category represents an environment domain, classified into effectors or into other subcategories. We have defined two main categories related to indoor and outdoor environments. Environment model -Category Name string 0..
Category -Category Name: string -Effector Name: string
O..
0..*
1
Effector
-Attribute: string -Tasks Name: string
Fig. (3a). Environment model
Fig. (2a ). User Model
Figure (3b) represents an example of an indoor home environment (the outdoor environment can be represented similarly). It underlines the different categories and effectors that constitute the Indoor environment. It contains subcategories bathroom, living room, toilet, etc. and the subcategory "bathroom" contains the subcategories washbasin, bathtub, etc., with the effectors mirror, lamp, door, etc. Tap Cold Water -T ask:open -T ask:close +Required hand workspace= 5 + Required hand force =1 0 Lampe -Task:switch on -Task:switch Off -.................
()
+Required hand force =15
washbasin -Effector:tap cold water -Effector:hot tap water
BathRoom
-Effector:BasinStopper
-washbasin
-Effector:Basin -.
-bathtub -Effector:Lampe
BathTub
-Effector:tapcold
In-DoorCategorie D q r D
m
water
-
.................
Tap H-ot W ater -Task:open -Task:close -.................
Fig .( 2b ) .User Profile for the example scenario
3.3. The environment model
The environment model provides all devices (ex. Doors, windows, sensors, etc.) or services (Internet access, Video on Demand, remote alarm, etc.), defined as effectors, that constitute the ubiquitous environment. This model is - 584 -
+Required hand workspace= 5 +Required hand force=10
+...................
Fig. (3b).Indoor environment description
In order to perform the matching, we define two groups of effecter's attributes: 1- Attributes that define effectors' physical characteristic, such as the weight of the
effector, the dimension, size, etc. This group contains also special attributes that define the required elements allowing the effectors to be usable. For example, the required hand force (in Newton), the required hand workspace (in cm3). Those values are mainly provided by the manufactures. They also can be measured in the laboratory. In this paper, we only consider the two attributes: (1) required hand force and (2) required hand workspace. These attributes are chosen because of their ability to lead to a handicap situation. 2- Attributes that define effectors' related tasks and services. These attributes are numbered according to each effector. For example, tasks related to the effector tap-cold-water are open and close (Figure 3b), and the physical characteristics of tap-cold-water are: required hand workspace (ex. about 5 cm3) and required hand force (ex. 10 N).
3.4. Matching between user model and the environment model
Once attributes of both user and environment models are defined, we determine the matching or the relationship between this attributes that leads to the handicap situation. For that, we search if this attribute leads to a handicap situation? For example, does the hand force attribute generate a handicap? For example, Pierre has a hand force of 5 N, all effectors that require a force greater than 5 N, become obstacles, and provide the handicap situation. That is what we call a matching between the attribute hand force and the attribute required hand force. By the same reasoning, Pierre has a wheelchair as a type of technical aids and the stairs (effector) are not accessible for him, consequently it leads to the handicap situation. There is a matching between the attribute type of technical aid and the attribute type of disability. We represent the matching relations by a set of rules, exp:
4. Implementation
Different approaches have been proposed in the literature for implementing semantic environments [4-5] or for using web semantic to model users [2, 6]. We adopted an approach that integrates both works and to build a semantic framework that can be used to model the user and the environment structure. We implemented User and Effector concepts that define respectively an OWL description of the user model and the environment model. The matching is defined by an object property in the ontology based language OWL [7], named Handicap-situation.
4.1.The property Handicap_situation:
It is related to the effector that leads a user to a handicap situation. If we have the following relation (called triples): User(i) and Effectors(j) represent, respectively, an instance of User concept and Effectors concept. This means that the Effectors (j) leads User (i) to a handicap situation. The OWL description of the property Handicap-situation is:
4.2. The Semantic Matching Framework (SMF)
SMF is the semantic matching between the and the environment attributes. The Framework architecture is showed in Figure 4. user
User Defined
Quelry engine
If (hand force user > required hand
force of an effector) Then user can use this effector, Else this effector leads to handicap situation.
In the next section, we present a mechanism based on an SMF framework to automatically define the effectors that lead to a handicap situation for a PwSN in an environment composed on several devices and effectors. We also presents how we can easily define the tasks/services that user can perform in this environment.
rules
Inference Reasoner enine
Context Manager
I Tasks/Services
User profile
External Data
Internal Infrastructure
Provided data
j
Fig 4. The SMF architecture
SMF is composed of several collaborating a components: knowledge base, an instantiation manager, inference reasoner and a query engine. It operates with external data
-
585
-
(user profile and context manager) and provides tasks and services.
4.3. The knowledge base ontology
The knowledge base ontology (KBO) implements the effectors and the user concepts, using OWL standard web semantic language to define, instantiate and store the different instances of our ontology. The KBO allows us to interpret the concepts effector and user and the properties Handicap_situation described above. The knowledge base links the different concept instances. It provides interfaces for the query engine and inference reasoner to manipulate correlated instances.
4.4. Instantiation manager
The instantiation manager obtains raw data from the two components: context manager which provides the contextual information such as the user position in the environment and the User profile module which provide the user the real profile, representing characteristics of the user's environment.
4.5. Inference reasoner engine and the user defined rules
This component infers and filters the property
Handicap-situation and the knowledge base
with user and environment instances over a set of rules. We used JENA 2 generic rule engine [8] to perform hybrid reasoning over the knowledge base. The inference reasoner engine module processes iteratively the rules which are inferred from the matching defined in section 3 between the instances of both concepts (User, Effectors).
5. Experimentation
Two users with motor disability have participated in the experimentation (Table 1).
Table. 1: User's diagnostics User name
Francois
Hand force (Newton) 5
Hand work space (cm3) 10
Pierre
5
3
SELECT Effectors(i).tasks WHERE (User
=
"Pierre")
AND (Environment="Env(i)") AND Effector(i)
belongs into "Env(i)"
AND(User have Property
(Handicap-situation(
Effectors(i))
The symbol "-" defines the negation property
-
586
-
of Type technical aids
Muscular dystrophy SCI C5/6
Wheelchair
Wheelchair
(SCI C5/6 = spinal cord injury with level C5/6) The user profile is formatted as XML standard and integrated on a RFID tag. We used our laboratory's workspace to build a prototype of a smart house. It was composed on two spaces Envl and Env2 (figure 5). Envl is a living room including two windows, Tv, Radio and living room light. Env2 is a kitchen including a coffee machine and a kitchen light (table 2). Table.2: Environment description
Environment Environment Required Required name effectors Hand Force hand (Newton) workspace (cm3) 0 5 Kitchen Kitchen environment light 2 Coffee3 (Envl) machine 0 Living-room Living room 5 environment light 5 1 Radio (Env2)
Windowl
4.6. Context query engine
The context query engine (CQE) provides desired data from the knowledge base. We adopted the RDF Data Query Language (RDQL) [9] in order to support expressive queries. In order to provide "on demand" relevant environment tasks, the CQE asks the KBO with set of queries. In our case, we formatted queries regarding user's profile and environment effectors. The next example show the query that provide appropriate tasks that a user (Pierre in this case) can use in an environment i (Env(i)).
Type of disability
25
Window2
25
TV
5
10 10 5
70 c-
Figure 5. Experimentation demonstrator
Each user was invited to fellow next scenario: The users have to badge before coming inside the house. The RFID reader transmits the user profile to the server using Wifi wireless protocol, which allows the system to identify the user and display a personalized welcome
message. Within the house, infrared motion sensors track user's position. SMF was integrated in a server. The user uses a mobile handhelds integrating an interface to interact with the server. According to each location (Envl&Env2) the server discovers the effectors belonging to this environments and provides a service discovery message to the user (figure 6) New Discowrred Serbces: -
-LngRboomLight -Radio
-Teloesion
OK'
Figure 6. Service Discovery Message for env 2
After that, the server runs the SMF Framework and provides the user with adapted services according to his profile (figure 7). In the case of Francoise, the required hand force of the windows of Env2 (25 N) is superior to the user's hand force (5 N). The SMF filtering process hides these windows and delivers the tasks and services associated to the living room light, radio and TV.
Figure 7. Assistive Services clients
The system can also presents effectors that make a user in a handicap situation in the specific environment. In the case of Francoise, windows 1 and windows 2 of env2 (figure 7). The user interface is automatically updated in the mobile handheld according to the user environment interaction.
6. Conclusion
We presented in this paper the semantic matching framework that provides information to easily personalize environment services to user having special needs. SMF is sensitive to the environment and users (people with disabilities). We defined the matching between
both human factors and environmental factors and their impact on the user's daily living activities. This developed framework was integrated in a general demonstrator, representing a smart assistive environment for people with physical disabilities. The detection of handicap situation is done by an inference reasoner involving rules. This method allows to define the tasks that each user can perform, according to his capacity (defined in his profile), and his environment. In the near future, we are working on extending the SMF with service provision platform. We believe this is logical next step to enable non-deterministic managing environments, since living environment are multiple and unknown by the system (ex. Friend house, public space, etc.).
7. References [1] Fougeyrollas, P.: The process of cultural production of disability. A study of socio-historical backgrounds in the evolution of knowledges between body function and structures. ISBN 29803811-0-1. [2] Norman Sadeh, Fabien Gandon and Oh Buyng Kwon, "Ambient Intelligence: The MyCampus Experience" Book Chapter in "Ambient Intelligence and Pervasive Computing", Eds. T. Vasilakos and W. Pedrycz, ArTech House, ISBN: 1- 58053- 9637, 2006 [3] Griss, Martin; Letsinger, Reed; Cowan, Dick; Sayers, Craig; VanHilst, Michael; Kessler, Robert CoolAgent: Intelligent Digital Assistants for Mobile Professionals - Phase 1 Retrospective. In HP Labs 2002 Technical Reports HPL-2002-55R1 [4] Choi J., Shin D., Shin D., :Research and implementation of the context-aware middleware for controlling home appliances,: IEEE Trans. on Consumer Electronics, vol.51, nol (2005). 301-306 [5] Ranganathan A. Campbell, R.H., :A Middleware for Context-Aware Agents in Ubiquitous Computing Environments, ACM/IFIP/Usenix Int'l Middleware Conf., Springer-Verlag, (2003). [6] Pretschner, A. Gauch S., : Ontology Based Personalized Search, Proc. 11th IEEE Intl. On Tools with Artificial Intelligence, Chicago, (1999) 391398 [7] Mike Dean, Dan Connolly, Frank van Harmelen, James Hendler, Ian Horrocks, Deborah L. McGuinness, Peter F. Patel-Schneider, and Lynn Andrea Stein. OWL web ontology language reference. W3C Working Draft, 31 March 2003. [8] Carroll J.J. et al., Jena: Implementing the Semantic Web Recommendations, tech. report HPL2003-146, Hewlett Packard Laboratories Bristol, (2003) [9] Miller L., Seaborne, A., Reggiori, A.,:Three Implementations of SquishQL, a Simple RDF Query Language, Proc. 1st Int'l Semantic Web Conf., LNCS 2342, Springer-Verlag, (2002)423-435.
- 587 -
