for the support of a domotic homecare system are: Self-management, Context .... User tracking systems are currently widely used in the implementation of ...
Image Processing Based Services for Ambient Assistant Scenarios ´ Elena Romero, Alvaro Araujo, Jos´e M. Moya, Juan-Mariano de Goyeneche, Juan Carlos Vallejo, Pedro Malag´on, Daniel Villanueva, and David Fraga Universidad Polit´ecnica de Madrid, Dpto. Ingenier´ıa Electr´ onica, ETSI de Telecomunicaci´ on, Ciudad Universitaria s/n, 28040 Madrid, Spain
Abstract. Guaranteeing ubiquity and appropriateness of security and monitoring services provision to the users constitutes a priority issue for the authorities. This paper presents an innovative Wireless Personal Area Network architecture that takes advantage of some of the features provided by Intelligent Environments -large number of devices, heterogeneous networks and mobility enhancement- in order to adapt and personalise ambient conditions to the user profile. This system is based on image processing and its main aim is to provide an AAL solution that is integrated with other control devices for the home to make everyday tasks easier for users.
1
Introduction
The ageing of the population and, consequently, the growing of chronic diseases in a change in the way home services (i.e. monitoring, assisted living, emergency help) are provided. New technology developments could be a potential answer to the requirements of a society demanding better assistance [1]. Previous experiences urge for an improvement in human- machine interfaces enabling users (i.e. mostly elderly people, and disabled) to deal easily and non-intrusively with technology [2]. The latest innovations in this area come from the field of Ambient Intelligence(AmI) [3]. New technologies, based on the design of easy to use and easy to learn devices that work transparently and pervasively [4], will help users overcome the current existing limitations. Traditionally, Smart Homes have been designed for the rich and famous. Devices such as door openers, activated by telephone, have not been considered for use within the care sector. Ambient Assistant Technology (the use of products or equipment, to help maintain or improve functional capabilities) has begun to be accepted by the care sector and as a result environmental control systems are being used. Smart Home technology uses the same basic devices used in Assistive Technology to build an environment in which many features in the home are automated and devices can communicate with each other. Smart Homes are often the ideal solution for individuals with differing levels of disabilities.
Therefore, next generation follow-up and homecare systems have several important challenges to face. On the one hand, there is a need to develop nonintrusive systems that require a minimum human machine interaction and provide homecare services on a highly-usable basis, helping his daily tasks, enhancing the augmented environment while making computers disappear in the background. On the other hand, the big issue is to implement these necessities by means of affordable solutions for Public and Private Institutions, only achievable with low cost infrastructures easy to install and maintain. Wireless Personal Area Networks (WPAN) allow several sensors to be connected within the same network. They provide information about behavioural patterns and helping ageing, disabled and ill populations with everyday activities and healthcare follow up. WPAN’s capabilities that make them optimal, compared to other networks, for the support of a domotic homecare system are: Self-management, Context awareness, Heterogeneity and dynamism, Low cost, Performance with limited resources, Security and Trust. In order to provide extra features for our system, one of the most interesting option is to introduce a sensor based in image processing. This kind of sensors is based on filter combinations, mathematical operations, frames comparisons, images mixers, etc. They are able to detect movements and activity parameters such as speed and direction of the mobile target object. This paper presents an innovative Personal Area Network architecture. Several sensor nodes such as video cameras, presence and door crossing have been developed, monitoring the user’s status and its house, and providing information about his habits. This data conform the user profile, which serves as input for an actuators network with distributed intelligence that adapts ambient conditions accordingly. The proposed solution can be implemented and validated in a real home system scenario for the monitorization of elderly users.
2
Architecture
WPANs are constituted by several nodes. Each node - i.e. sensing device- can be divided in three different parts. First, the sensor itself: an ad- hoc module (accelerometer, PIR, camera...). All these sensors have low data rates (8bits when detect something new). Camera has processing capacity and therefore only sends processed instructions. Sensors have a second component, an autonomous power supply. As power requirements of each sensor are different, customized supply stages have been designed to use standard interface with commercial AAA batteries and always looking for the highest efficiency that maximizes sensors’ lifetime. Finally, the third part added provides the ”intelligence”: awareness of patient’s willingness, network functionality (e.g. routing, reconfiguration), etc. The connection between sensor’s and intelligence’s layer depends on each sensor: parallel I/O, SPI, A/D converter, etc. All these interfaces are implemented within the intelligence module (Figure 1).
Fig. 1. Intelligence module
The key component of this stage is a radio transceiver in the ISM band of 2,4 GHz that implements the physical layer and the MAC layer of the IEEE 802.15.4 wireless standard. The network requirements and functionality suit the given standard perfectly. In order to implement the remaining layers of the ZigbeeTM protocol stack, a low-cost 8051 microcontroller that connects to the radio transceiver through an suit the given standard perfectly internal SPI communication interface has been used. The commonly used OSI architecture has been discarded in favour of a service-based architecture with ZigbeeTM as protocol support. The aim of this design is to implement applications that arise as a mere aggregation of services and relations between services, rather than a node- focused orientation, simplifying the development of user applications [5, 6].
3
Image processing Sensor
This sensor is a vision systems that are conceived to be implemented as lowcost and very compact platforms capable to solve complex vision tasks at very high speed and with very low power consumption. It has capabilities in image processing and incorporates enough computing power to perform functions such as pattern recognition, objetct tracking or movements detection. In this sense we can take advantage in two ways: event detection and pattern recognition. This vision systems integrates all levels to carry out the tasks: Capturing (sensing) images, enhancing sensor operation, performing spatiotemporal processes on the image flow, interpreting the information contained in the image flow, supporting decision-making based on the outcome of that interpretation. This variety of structures is arranged into a bio-inspired architecture and powered by embedded RISC processor cores. Some relevant features of our vision systems are: – Truly mixed-signal architecture, with processing following a hierarchical flow. At early stages, processing is done by focal- plane mixed-signal programmable processors. Focal-plane means that there is a processor per data channel. At later stages, processing tasks are undertaken by digital processors – Huge computational power with low-power consumption. Such computational power (in the range of TeraOPs) can be exploited to handle frame rates of up to 10,000 F/s for images with medium spatial resolution. It can
also be used to perform very complex processes at the standard video frame rate with large resolution images – General-purpose, all-in-one architecture including processors, memories, data conversion, control and communication peripherals – Large operational flexibility. Through software programming the chips can be reconfigured and parameterized to meet specific customer needs Roughly, we can say that this vision sensor is:”A system whose input is a two-dimensional set of analog signals representing a visual flow, which internally converts these signals to the digital domain, and which processes these digital data afterwards” [7]. In a conventional vision system, data conversion to the digital domain occurs right after the sensors. Hence, all processing is realized in the digital domain. But in this system, we employ a different strategy that follows a hierarchical processing approach with two main levels: – Early-processing. This level comes right after signal acquisition. Inputs are whole images. This means huge amounts of data; proportional to the spatial resolution of the sensor; and to the number of bits used to code the pixel values. Much of this data is redundant and the basic tasks at this level are meant to extract the useful information while reducing the data set. Outputs at this level are reduced sets of data comprising image features such as object locations, shapes, speed, etc. In many cases these outputs consist of binary data whose dimension is generally proportional or smaller than the image size. – Post-processing. Here the amount of data is significantly smaller. Inputs are abstract entities, and tasks are meant to output complex decisions and to support action-taking. These tasks may involve complex algorithms within long and involved computational flows and may require greater accuracy than early processing. Thanks to this sensor, we can perform data processing in a more rapid and efficient way. Based on this architecture and using the sensor described above, we can develop services such as those described below.
4
Services
The proposed services are built on top of an AP layer that abstracts network or hardware specific tasks to the application. This middleware allows sending data to any node of the network without considering its location, sending broadcast messages, etc, and all this made possible by a simple and unique interface (based on query-response) that can be dynamically instantiated in any node at any time. This allows easily compose applications to take into account many complex aspects, such as privacy, safety, load balancing, low energy consumption, etc. [8, 9] in a totally transparent way to the user.
The idea behind this approach is to isolate common behaviour that can be reused from other applications and to encapsulate it as system services, so that once the system is running the addition of a new application becomes easy. An interoperable, reliable system, abstracting the hardware from the applications has been designed, developed and implemented. It is important to highlight that the size of the prototype is not comparable to that of wearable devices, but the aim of this work was just to emulate the behaviour of future wearable sensors, not to miniaturize existing ones. The architecture described above allows supporting services that provide an added value to the system, such as user modelling, context adaptability, user and object tracking, face and 2D recognition, tangible interfaces, common sense or reputation. According to the defined system requirements, the most appropriate services for the design process are described below. 4.1
User Modelling (UMS)
These systems use the existing technologies to recognize interests, habits, behaviours and needs of the users in a concrete spatiotemporal situation. This information, named context in AmI, is stored in a data base specifically designed for his kind of services as an attribute-value pair constituting users profiles. During the execution time, the system processes the input data, matches coincidences in the database, acquires new data pairs, establishes new links between input data and determines possible systems responses. This service allows providing specific and personalized services according to the user needs [12]. The designed system includes a set of pre-programmed procedures which evolve to self-adapt to user routines and preferences depending on the patient behaviour. The system will gradually improve its level of understanding and knowledge of the user’s habits and it will therefore have a more complete model of the user behavioural patterns. Besides, this system might offer in the future, by means of user tracking like identification, and broad context information; multi-user responses, depending on several factors such as week day, season, presence of third users, etc. Thus, the system is completely integrated in the life of the user in a transparent way, easing he daily tasks and taking care of the user health in a pervasive mode [13][14]. 4.2
User Tracking Service (UTS)
User tracking systems are currently widely used in the implementation of security platforms and applications for elderly people. This kind of systems allows basic movement patterns identification and future actions prediction. In the described system, these features have been implemented through the integration of the image processing-based sensors. The utilization of this kind of device offers a wide range of possibilities for the above scenario since it is able to detect intrusions, to perform easy daily tasks or to identify alarms produced by falls or by long user inactivity periods.
The followings lines illustrate one example of the system used in the user’s daily routine tasks. This system is able to detect the user behaviour patterns when he goes to bed and as a result, it conditions the room according to the user preferences: light and temperature level, alarm activation, security requirements and, most importantly, the system activates the necessary procedures that mobility impaired people gets into bed in a easy and comfortable way. 4.3
Reputation Service
By sharing information within the sensor’s network, where each sensor acquires multiple contextual data, the system is able to detect unusual behaviours in its own functioning / operating. Thus, the network is capable of self-evaluating and discovering possible malfunctioning sensors. In this case, the system informs the user and ignores the data sensed from the suspicious device if the error persists. This service enhances the system’s reliability and security, both imperative features since the system handles confidential data and critical applications such as home security systems. Taking into account that nodes can fail and are vulnerable to attacks because their low resources, the system has been designed to minimize the consequences of such problems. To achieve this, the network implements redundant information from different sources and sensing technologies, while the reputation system continuously evaluate the trust level of each node. Moreover, the failing node can be isolated and ignored if the problem cannot be solved.
5
AAL Scenario
The deployed system is an example of implementation of the concepts and services described in previous sections. This system mounted in a residential environment is a WPAN featuring environmental sensors (vision system, presence detectors, InfraRed sensors, etc.) and several ambient actuators (alarms, light switches, appliances, etc.) which can adapt to the situation according to the context and the collected data. The main goal of the system is to promote independent living of the patient. The system offers a home care service, providing complete information about patients’ behaviour. These parameters are used to control the patient health, to react in emergency cases (falls, loss of orientation, gas leaks...) and to set the most appropriate conditions in the environment, taking the user state into account. The wireless sensor network only consists of two different elements: environmental sensors (used to monitor home conditions and events) and home actuators (used to change home conditions). The functionality of the different elements is explaining below: The sensors have been developed as stand alone wireless devices, powered with batteries, and have been provided of an automatic start of the measure, auto-discovery, and
auto-configuration facilities. That means that the patient does not need to take care of configuring or connecting them. It is important to note that the user does not need to ensure either that the sensors are powered on and off, as they are suspended and resume automatically after a given time. Sensors included in this scenario and data, parameters and signals associated to them are: Passive InfraRed (PIR) (presence), InfraRed (pass through), door opening (doors or windows opening/closing), and camera (user movement direction/speed). The actuator nodes are designed in the same way as the sensor. The functionality in this case is to change the status of the house, resulting in a change that can be of several types (activation of light, sound, operating switches, sending data. . . ). Actuators included in this scenario are alarms and a power control switchs that allows to power on or off all the devices of the house. The home care system needs an easy-to-use interface providing the following information. In the home conditioning part, it is fundamental to let the system interact with the user proactively, warning the user for security or medical reasons. When the system detects anomalies, either in the user’s general activity or in a concrete physiological parameter, it warns the patient as transparently as possible: colour codes in walls, acoustic interfaces, extra-sensorial aspects, etc. In other cases, where the user doesn’t need to be informed, the system is designed to provide a completely transparent interface, so that he system can act autonomously with initiative, evolving, learning and making decisions, letting the user rectify its actions. The rectification is done in a natural way, and he user never needs to tell the system that the decision made was wrong. With the user modelling system, the environment has its own methods to take rectifications into account and evolve to improve its performance. This design avoids the need of manual introduction of parameters or data to the platform; it is the reiteration of actions what will lead the user to feel completely satisfied with the performance of the system. Using easy-to-learn or even completely transparent interfaces, an accessible environment can be achieved. This environment is adaptable to any user, avoiding the communication man-machine barriers and obtaining a very intuitive system appropriate and open to any kind of user.
6
Conclusions
The proposed Wireless Personal Area Network architecture takes advantage of its main features: large number of devices, heterogeneous networks and mobility enhancement, in order to adapt and personalise ambient conditions to the user profile for an Ambient Assistant Scenario. This approach is based on the aggregation of services with different capabilities. The basic services presented (User Modelling Service, User Tracking Service and Reputation Service, and their implementation in a real world) along with the image processing based sensor, allow the Ambient Assistant Scenario development. As we have seen, this system is
based on Intelligent Environments which have been proven to provide valuable tools to assist people with chronic health conditions or disabilities helping them to gain independence and overcome barriers.
7
Acknowledgment
This paper has been supported by the DGUI of the Comunidad de Madrid and the Universidad Polit´ecnica de Madrid under the GR-1703-2007 project. The authors would like to thank all LSI members for their support during the development of the paper.
References 1. Gareis, K.: Towards user orientation and social inclusion in the provision of elearning services. In: eLearning Conference 2005. (May 2005) 2. Hernandez, C., Casas, A., Escarrabill, J., Alonso, J., Puig-Junoy, J., Farrero, E., Vilagut, G., Collvinent, B., Rodriguez-Roisin, R., Roca, J.: Home hospitalisation of exacerbated chronic obstructive pulmonary disease patients. Volume 21. Eur Respiratory Soc (2003) 3. Trivedi, M., Huang, K., Mikic, I.: Dynamic context capture and distributed video arrays for intelligent spaces. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on 35(1) (2005) 145–163 4. Remagnino, P., Foresti, G.: Ambient intelligence: A new multidisciplinary paradigm. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on 35(1) (2005) 1–6 5. Sgroi, M., Wolisz, A., Sangiovanni-Vincentelli, A., Rabaey, J.: A service-based universal application interface for ad-hoc wireless sensor networks (draft). Whitepaper (November 2003) ´ 6. Alvaro Araujo: Metodolog´ıa para el desarrollo de aplicaciones basadas en servicios sobre entornos inteligentes. PhD thesis, Universidad Polit´ecnica de Madrid (2007) 7. Anafocus: Eye-RIS vision system evaluation kit. Hardware description 8. Coleri, S., Puri, A., Varaiya, P.: Power efficient system for sensor networks. In: Computers and Communication, 2003. (ISCC 2003). Proceedings. Eighth IEEE International Symposium on. (2003) 837–842 vol.2 9. Rabaey, J.M., Ammer, M.J., da Silva, J.L., Patel, D., Roundy, S.: PicoRadio supports ad hoc Ultra-Low power wireless networking. Computer 33(7) (2000) 42–48
