eMonitoring for eHealth Research Projects for Assisted Living Thomas, A.M., Evans, C., Moore, P., Sharma, M., Patel, A., Shah, H., Chima, P., Abu Rmeileh, S., Dubb, G., Bhana, R. Faculty of Technology, Engineering and the Environment, Birmingham City University, Birmingham, United Kingdom.
[email protected] Abstract— Education and research exist hand-in-hand, and todays' students are likely to rely heavily on tomorrows' technologies for assisted living when they reach the later stages of their lives. That makes them primary stakeholders in terms of current eHealth research, but below PhD level student project time is limited. Therefore, involving them to a greater extent in their own futures requires care in considering how smaller projects can be created that facilitate progression of overarching eHealth work. Therefore, this paper considers the future need for eHealth monitoring and describes some projects that, while allowing independent learning over short timescales, can add significant value to related academic research. Keywords-component; health monitoring, assisted living, context-awareness, Kansei sensing, affective computing, sensors.
I.
INTRODUCTION
The majority of students are young with the largest part of their lives ahead of them. By the time they reach the later stages of their lives the world can be expected to be a very different place from where we currently exist. For instance, it is expected that initiatives such as the Internet of Things will mean significant integration of technology, and particularly sensors, into our daily lives. Much of that ubiquity of technology will relate to eHealth, including systems that ensure living is assisted within the home environment for patients with conditions their clinicians consider do not warrant institutionalized care. Therefore, todays' students may become tomorrows' cared-for in smart and intelligent home care spaces. Also, as todays cared-for often desire life within the familiar environs of their homes (or those of family and friends) over formal care, in future similar desires may relate to care using familiar technologies. Therefore, involvement of students in eHealth projects, at all academic levels, allows them to develop a high degree of ownership of the IT systems involved, including in understanding how they can be developed and implemented. The former allows for increased motivation and the latter provides real-world skills for future employment. In order to achieve this, within over-arching academic research projects, consideration is required of a wide range of eHealth system areas. For instance, sensor node, and monitoring system, development is obviously important, along with a knowledge of communications systems so critical to progression of the Internet of Things. However,
sensor data cannot be considered standalone, and so requires knowledge of how to extract metadata, such as to determine behavior, activity types and person-location. Often this requires concern for privacy of the cared-for, which requires data fusion, blind cameras and shadow monitoring, which can be accomplished over short-duration student projects. It also must therefore consider emotion and context, as such projects all revolve around vulnerable humans, allowing even non-technical students to get involved in eHealth. Similarly, mobile computing devices are very ubiquitous now, and certainly may be almost completely so in the future. Therefore, projects based around handheld devices, and even RFID, barcodes, and suchlike offer ideal areas within which students can base interesting and relevant projects. However, we must also remember that all eHealth systems require power and so Green ICT for Sustainability is therefore important to temper the desire for power-hungry systems that may impact on the technology-rich future of today's students through squandering of natural resources and production of unnecessary carbon emissions. Therefore, this paper considers all of the above factors, and describes areas where students, from undergraduate to doctorate level, can contribute significantly to the quality of their own futures, over short project-durations, while also benefitting eHealth academic research on a wider level. II.
ENVIRONMENT MONITORING
Environment variables are central to work in areas such as monitoring carbon emissions relating to power consumption, as well as to provide automated climate control as part of building management systems. In eHealth, the need for such monitoring, and associated control of heating, lighting and suchlike, may be more critical as it could be required to facilitate recuperation, for instance. Also, it has been said that many patients with Dementia express their dissatisfaction with temperature through problematic behavior, and light levels can affect headaches as well as disrupt circadian rhythms. Those examples, which are just two of many, illustrate how environmental monitoring projects can be directly related to real-world applications (see, e.g. [1]). Measurement of many environment variables can be achieved using microcontrollers [2][3][4] (e.g. the widely used Arduinos, see Figure 1), making them ideal project candidates.
invasion of privacy. Under those circumstances, 'blind camera' systems can be constructed from a wide range of indirect sensing (e.g. light levels, pressure pads, etc.). However, data fusion is then necessary to build the greatest probability of particular behaviors occurring in sensor data, and power consumption monitoring can provide useful probabilities not only that a certain appliance is in use, but also therefore the location at which activity is occurring. For instance, boiling a kettle, or making toast, indicates kitchen activity, while indications of a water boiling being active over an extended period, with localized elevated humidity, may indicate bathroom activity. IV.
Figure 1. An inexpensive environment-sensor system prototype.
III.
POWER MONITORING
Monitoring data for mains electricity consumption is very useful for eHealth monitoring, largely because it provides a wealth of behavior data from a single sensor node [5][6][7]. As shown in Figure 2, power use data can differentiate between a wide range of domestic electrical appliances, in this case based on a simple microcontroller circuit with an inexpensive current clamp. With sub-metering, accuracy of behavior extraction will obviously increase as part of a house-wide sensor system. Therefore, this kind of sensing could enhance the studies, and projects, of a wide range of students including those undertaking electronics, communications, networking, electrical and even social science courses.
'SHADOW' MONITORING
With the introduction of mass-market gaming solutions, it is possible to deploy cost effective depth-sensor technologies in a variety of fields. Game controllers, such as the Microsoft Kinect, have a wide variety of applications in eHealth [8] including those on the Microsoft website: Jintronix – movement tracking, NConnex – 3D room visualisation and Zebcare – care monitoring. One of those uses of depth-camera devices is tracking of users positions and movements, for instance to measure activity levels, location and for person-down alarms. As an example, Figure 3 shows output data from a Kinect-based prototype monitor (darkness of the skeletal data increases closer to the current time) [6].
Figure 3. Skeleton time-tracking using a Microsoft Kinect.
Figure 2. Variations in power consumption over a 48-hour period.
For people with significant cognitive decline intrusive sensors, such as CCTV cameras, may be warranted. However, for other patients, and those in earlier stages of cognitive disability, they may be considered a significant
Deployment of the Microsoft Kinect has seen many applications (see e.g. [9]) and Birmingham City University have successfully used it within a project to detect human and animal movement in a room. That project will eventually infer from shadow images the difference between a healthy person and a person with health issues. The intention is to deploy three Kinect devices around a lab to obtain skeletaljoint data at different times of the day, in a similar manner to [9]. This work is intended to expand on [10] by collecting full body data rather than gestures of the upper body, allowing more heath issues to be detected. The Kinect databases will be used to compile a standard model of people. Eventually students and staff will be asked to act out various impairments and collect this data. This will
be used to develop an algorithm to categorize and predict impairments in real health cases. For example, as show in [11], the data collected from the sensors can also be used to infer body mass, this can be applied to eating disorders but can also be used for cases such as Alzheimer's where the patient is forgetting to eat. The Kinect is also ideal for maintaining dignity and privacy [12] because instead of real-time images, only shadow data is collected. The unit is discrete and can be positioned easily around rooms. Their inbuilt microphones can be used to detect noise from falls, trips and accidents, as well as vocal changes. This can be used again to develop a data set of familiar and unfamiliar sounds in a room, therefore allowing prediction of non-standard behaviors. For example with intruder detection, normally there would be no noise if the house is empty. However, if there is noise and an intruder enters the room, the Kinect microphones can detect the noise changes and then be set to detect humans in the room via the 3D camera technology. If a human is detected an SMS message can be sent, or emergency services contacted. It can also be used to detect elderly patients and the noise can be used trigger video analysis to assess the well-being. If this is outside the normal tolerance band, again an SMS alert can be sent to the relative, health-care worker or warden. V.
BODY MOUNTED MONITORING
Microcontrollers and mobile computing devices (e.g. phones and tablets) now allow a significant degree of portability to be achieved in student projects. For eHealth applications they can be used for physiological monitoring, such as measurement of brain activity (e.g. EEGs), galvanic skin response and heart beats [13] (e.g. ECGs). As an example, ECG measurements can be undertaken using a microcontroller with a simple measurement circuit and metal-plate electrodes. As shown in Figure 4, such a setup, in this case based on an Arduino platform, can provide acceptable data for ongoing heart rate measurements.
as being able to interface to a mobile phone, over Bluetooth, or within a personal-area network. While many electronics students may prefer to construct their own activity monitors, use of commercially-available game controllers allows costeffective eHealth applications to be developed by a wider range of students.
Figure 5. Accelerometer data from a Nintendo Wii game controller.
Where mobile phones are used in body-mounted systems, the sensors they often contain can also be put to good use enhancing projects. Many such devices now contain at least an accelerometer, gyroscope and light sensor. Logging of their data, and transmission to web services for additional processing and storage, forms an interesting project in its own right. However, consideration of frequency-components in accelerometer and gyroscope data over time (e.g. Figure 6) can be used to determine types of activities being undertaken, as well as providing longitudinal data to link changes in gait to condition-progression.
Figure 6. Frequency-domain analysis of mobile phone sensor data.
VI.
Figure 4. Output from a simple microcontroller-based ECG.
Other examples include activity and gait monitoring, for which the Nintendo Wii remote control provides a sophisticated measurement platform including an accelerometer and Bluetooth communications [14]. As shown in Figure 5, it can provide useful activity data, as well
ICT FOR SUSTAINABILITY
The changing landscape of ICT today indicates that energy consumption has become one of the key milestones of good governance in businesses, the home and Higher Education Institutions. The majority of green ICT projects relate to electricity consumption [15] and green ICT projects typically relate to sustainability, cost analysis, Smart City and Smart Home initiatives [16]. The internet and peer-topeer networking have driven the need for power consumption of private home owners significantly, as bandwidth increases. This is supported by [17] where it is argued that a growth in the usage of audio-video downloading of files will increase traffic by 103 Tb/s. Therefore, technology and related communications systems are rapidly becoming a significant aspect of global
energy use and carbon emissions. However, they also hold the promise of improving the efficiency of existing powerhungry systems, thus helping reduce emissions. That dichotomy of interests is the domain of ICT for Sustainability (ICT4S), which attempts to minimize ITrelated energy use while also finding applications for IT in development of low-carbon technologies. Given that students of today will suffer greater difficulties due to environmental impacts and peak-oil, for example, projects that improve their understanding of ICT4S are obviously highly relevant. ICT4S projects can take many forms, including simply imparting an understanding of the implications of equipment choice. For instance, all of the eHealth projects described in this paper are likely to depend on some form of computer, mobile device or microcontroller-system. Measurement of the power consumption of those devices can be simply achieved using commercial, low-cost, devices. As an example, Figure 7 shows instantaneous power usage for a small range of computer devices. While many students may assume that choosing more powerful computing devices is preferable, use of WiFi enabled microcontrollers, or system-on-a-chip devices such as the Raspberry Pi, can result in significantly lower power consumption than a traditional desktop PC (even if that PC is used in a headless mode). Even the low-power 'web book' laptop shown in Figure 7 offers c.80% reductions in power use compared to a headless PC. For sensor-based monitoring systems, which may need to operate 24/7, those are important facts to understand.
nodes. For networking students, a greater understanding of server power usage, for a range of data serving scenarios, code-libraries and algorithms could help ensure network (and internet) related projects actively seek to reduce the negative impacts of IT on the environment. VII. INTELLIGENT AGENTS Intelligent software agents have been applied in many domains to model roles and practices in areas such as eCommerce, financial advice in banking, and fraud detection in insurance. However, they have not been widely adopted in e-Health, which can be facilitated through related projects where students benefit from state-of-the-art networking facilities. One reason is that software agents tend to be autonomous, while in healthcare decision making is a critical task that usually must be performed under the control of health professionals [18]. Nevertheless, software agents can also be intelligent and collaborative, allowing the obstacle of “decision making in healthcare” to be overcome. Furthermore, software agents and multi-agent systems can utilize other technologies and middleware, such as ontologies and context aware systems to achieve their objectives. Figure 9 shows an e-Health monitoring middleware system based on the CloudBDI.Net platform. The CloudBDI.Net, is a FIPA [19] compliant BDI [20] multiagent systems platform for developing multi-agent systems based on BDI and reactive agents. Applying autonomy and intelligence, provided by intelligent software agents and multi-agent systems, to Smart Care Spaces will help to maintain people longer at home or in their preferred environment, support maintaining health and functional capability of patients and organize healthcare at home. Moreover, it will enhance the security of patients, prevent social isolation and support maintaining the multifunctional network around the individual while reducing the costs of current healthcare systems.
Figure 7. Example IT power consumption values.
Other aspects of ICT4S suitable for student projects include use of the sensor systems described herein (particularly environment and energy monitoring) with actuators and interfaces. For instance, projects could be developed for automatic power saving in consumer devices, controlling room temperature, and suchlike. Shorter undergraduate projects could simply gather power consumption data for inclusion in product selection databases and algorithms, allowing future students easy access to data on which to judge their project systems in terms of energy and emissions. That could include power consumption of electronic components and sensors, for developing very-low-power and energy harvesting sensor
Figure 9. e-Health Monitoring Middleware using CloudBDI.Net MultiAgent System Platform
VIII. INTERFACING AND VISUALISATION The quality of care provided in medical systems depends largely on the quality of related electronic communications and graphical user interfaces [21]. Therefore, projects considering how clinicians, carers, and patients can access Smart Care Space data is very important. Such projects can include use of tablet computers, such as the prototype shown in Figure 10 which has been used for a Masters project at Birmingham City University (see Figure 11, illustrating access to patient-specific data). It is capable of accessing static and dynamic content (internet and local storage) using RFID, barcodes and voice recognition. Live sensor data is accessed as text and simple visualizations. The RFID tags shown have a wide range of uses including patient tagging, equipment tagging and carer visit recording. As cloud architectures minimize dependence on individual mobile devices and allow service-personalization [22], the system relies heavily on web services and HTML5, making it largely platform independent.
responses into computer hardware and software enabling interaction. Bringing Kansei into a Smart Home requires applications to be human friendly but at the same time focus on the task assigned to the device. Adjectives are a good way of creating semantic differentials such as modern --> conservative; beautiful Æ ugly and provide a helpful way to communicate between the human and machine (device). The application of Kansei Engineering with Semantic differentials in User Interface Design could make a useful area for researching mobile applications for Demented patients, including by students. The response to particular stimuli could be evaluated based on the adjective used to understand the emotional state of the person. UID and Kansei Engineering coupled together could help to support independent living of a person who is demented in their own home [24]. Patients who are demented tend to have difficulty acclimatising to change in their own environment, so a case study could be designed around the patient as a Kansei Metaphor. Using visual techniques such as images of a person’s household items could provide a framework of the mental state and support that person as their symptoms worsen. The semantic differentials for a person with the early onset of dementia, inhabiting their own home, could include a range of semantic adjectives such as comfort [inviting Æ elegant], ornateness [cheerful Æ loud] etc. and really depends on the individual person to be supported [25]. An example architecture for a Kansei-based system is shown in Figure 12.
Figure 10. Prototype tablet software and equipment.
Figure 11. Accessing eHealth data on an Android tablet using RFID.
IX.
KANSEI ENGINEERING
Kansei monitoring is a technique which introduces emotional and intuitiveness between human and machines [23]. Such is the need of humans to integrate emotional
Figure 12. An example system architecture for a Kansei-based system.
X.
CONCLUSIONS
The future of eHealth is a technology-rich area of research and implementation that todays' students are likely, one day, to rely on heavily for at home health care in
sensored smart spaces. To a great extent, they will be responsible also for research, development and implementation of eHealth systems, either as part of academic research or future work in industry. Therefore, it can be argued that they should hold significant ownership of eHealth technology development, during academic studies. That can be partly achieved through development of projects than can be undertaken over short durations, at all academic levels, within larger over-arching research work. In so doing, the students gain a wider appreciation of how technology will integrate into their future lives. Also, and of equal importance, they can learn how their decisions on smart care space implementation can impact the wider environment and how sustainability may be ensured as a central thread through the development of eHealth. This paper has described some, but by no means all, of the potential projects, some of which are already being undertaken by students at Birmingham City University. It is therefore hoped that this paper will motivate new and exciting areas of research that students can engage in to increase their ownership of technologies critical to their future living. The authors also welcome discussions at the workshop on the issues discussed herein. REFERENCES [1]
[2] [3]
[4]
[5]
[6]
[7]
[8]
Negru, S., 2012, “SemaKODE: Hybrid System for Knowledge Discovery in Sensor-Based Smart Evironments”, ICWE, LNCS, 448451. Mittal, R.M., 2010, “Wireless Sensor Network for Monitoring the Environmental Activities”, IEEE, 978-1-4244-5967. Das, A., Bhadri, P., Beyette, F.R., Jang, A., Bishop, P., Timmons, W., 2006, “An Integrated Amperiometric Sensor for in situ Environmental Monitoring”, U.S. Government. Cherubini, C., Gaisi, C.I., Pastore, N., 2009, “Application of Modelling for Optimal Localisation of Environmental Monitoring Sensors”, Advances in sensors and Interfaces, IWASI, 3rd International Workshop on Date of Conference: 25-26 June. Itoh, T., Fujimoto, J., Maeda, R., 2011, “Application of Wireless Sensor Nodes to Commercial Power Consumption Monitoring”, DTIP, ISBN: 978-2-35500-013-3, 11-13th May France. Dasgupta, S.; Ibeawuchi, S.-R.C.; Ding, Z., 2009, “Optimum Sensor Placement For Source Monitoring Under Log-Normal Shadowing in Three dimensions”, ISCIT 2009: 9th International Symposium Communications and Information Technology, Digital Object Identifier: 10.1109/ISCIT.2009.5341223, 376 – 381. Chen, Y., Mu, X., 2010, “The Software Subsystem of Building Power Consumption Monitoring and Management System”, Power and Energy Engineering Conference (APPEEC), Asia-Pacific, 28-31 March. Parajuli, M. Tran, D., Ma, W., Sharma, D., 2012, “Senior Health Monitoring Using Kinect”, Communications and Electronics (ICCE), 2012 Fourth International Conference.
[9]
[10] [11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19] [20]
[21]
[22]
[23]
[24]
[25]
Chaves. T., Figueiredo. L., Da Gama, Araújo. C., Teichrieb. V., 2012, "Human Body Motion and Gestures Recognition Based on Checkpoints", 14th Symposium on Virtual and Augmented Reality, IEEE. Dammak. M., Ammar., Alimi. A.M., 2012, "Real-Time Analysis of non-Verbal upper-Body Expressive Gestures", IEEE. Velard. C., Dugelay. J., 2012, "What can computer vision tell you about your weight?", 20th European Signal Processing Conference, IEEE. Stone. E.E, Skubic. M., 2011, "Passive In-Home Measurement of Stride-to-Stride Gait Variability Comparing Vision and Kinect Sensing", IEEE. Whitchurch, A.K., Abraham, J.K., Lonkar, M.A., Varadan, V.K., 2007, “Design of a Compact Amplifier and Signal Conditioning Module for Wireless EEG Monitoring”, Region 5 Technical Conference, 153 – 156. Nambu, M., 2007, “ Body Surface Mounted Biomedical Monitoring System using Bluetooth”, Engineering in Medicine and Biology Society, EMBS, 29th Annual International Conference of the IEEE, August 23-26, France, 1824-1825. Riaz, M.T., Gutierrez, J.M., Pedersen, J.M., 2009, “Strategies for the next generation green ICT infrastructure”, Applied Sciences in Biomedical and Communication Technologies, ISABEL 2009, 2nd International Symposium. Ochiai, H., 2012, “Power Data Management on the Internet Space: Green ICT Projects in Japan”, Communications Conference (COLCOM), IEEE Colombian. Lu, Shyue-Ching, 2011, “Dual Thrust in ICT Innovation: Green and Application”, VLSI Design, Automation and Test (VLSI-DAT), International Symposium. Najafi, M., Aghtar, S., Sartipi, K. & Archer, N., 2011, "Virtual Remote Nursing system", Consumer Communications and Networking Conference (CCNC), 2011 IEEE, 13-17. FIPA, 2012, "The Foundation for Intelligent Physical Agents", http://www.fipa.org [Accessed 08 March 2012]. Rao, A. S., 1996, "AgentSpeak(L): BDI agents speak out in a logical computable language", MAAMAW '96 Proceedings of the 7th European workshop on Modelling autonomous agents in a multiagent world: agents breaking away, 42 - 55. Beuscart, R., Renard, J.-M., Delerue, D. and Souf, A., 1999, "Telecommunication in healthcare for a better coordination between hospitals and GP's: routine application of the 'ISAR-Telematics' project", IEEE Transactions on Information Technology in Biomedicine, Vol. 3(2), 101-108. Hakkani-Tür, D., Tur, G. and Heck, L., 2011, "Research Challenges and Opportunities in Mobile Applications", IEEE Signal Processing Magazine, July 2011, 108-110. Nagashima, T.; Sawai, M.; Nagashima, Y., 2011, "Conceptual Cognitive Scheme of Kansei on Referring to the Use of Human Language", Biometrics and Kansei Engineering (ICBAKE), International Conference, DOI: 10.1109/ICBAKE.2011.23, 14 – 19. Ota, Y. 2011, "The Societal Role and Design of Pictograms as Kansei Language (Perceptual Language)", Biometrics and Kansei Engineering (ICBAKE), DOI: 10.1109/ICBAKE.2011.53, 1 - 13 SuKyoung K., Youngil C., Kazuhisa N., Toshimasa Y., 2011, "The Relationship between Preference and Information Types: An Approach to a Preference Model in Intuitive Decision-making", Biometrics and Kansei Engineering (ICBAKE), 10.1109/ICBAKE.2011.36, 23 – 26.
