Open Research Issues in Vision-Based Fall Detection ...

1

Open Research Issues in Vision-Based Fall Detection Systems (FDS) X. Kolovou1, C. Doukas5, D. Vergados3, I. Anagnostopoulos4, I. Maglogiannis2 1,2

University of Central Greece/Computer Science and Biomedical Informatics, Lamia, Greece 3 University of Piraeus/Informatics, Piraeus, Greece 4,5 University of Aegean/Information and Communications Systems Engineering, Samos, Greece Abstract— One of the most important parts in the development of an AAL (Ambient Assisted Living) system is to create a Fall Detection System (FDS). A large number of patients and senior citizens suffer from falls every year. This paper refers to research and techniques that have been made on fall detection with video surveillance. State of the art methodologies have been surveyed to estimate the background and the moving objects with great accuracy. However fall detection needs a combination of different techniques. A person could walk, squat or fall, it is difficult to distinguish these activities 100%. Another significant challenge is the optic angle of the fall, associated with camera overview. Most algorithms of fall detection aren’t reliable enough to detect a side fall of the person. This work summarizes the open research issues in this important topic. Keywords— Elderly assisted living, fall detection, video processing. I. INTRODUCTION

Millions of elderly people all over the world have health problems and live alone. They need support with daily activities and psychological support to feel safe because they are alone. The latest statistics 10 refer to a large percent of elderly people who have accidents every year, which can have serious or instantaneous consequences for their lives. More dangerous is a fall in their house because the person can lose his senses or he can’t ask for help. The results of this are they lay on the floor for many hours, and this could be crucial to their health if the accident is serious. The research in Ambient Assisted Living (AAL) field involves developing the fall detection system. The Rajendram et al. 10 sorts the fall detection techniques into four main categories: 1. 2. 3. 4.

User-activated alarms and pendants Floor Vibration-based fall detectors Automatic wearable fall detectors Video monitoring-based fall detectors

The 1st method is simple and low cost. These devises is require the user to press an alarm button when the accident happens. It isn’t automatic and this is the main disadvantage. After a fall the elderly person can not press the alarm button either because he hasn’t got his sense or because he

has forgotten to wear the device. In the second method the fall detection becomes automatic from a floor vibrationbased fall detector. Generally vibration-based systems depend on the material that the floor is made of. The third method is the most popular commercial brand of fall detection. The automatic detector is wearable, small and low cost. It is based on accelerometers, tilt sensors and gyroscopes which determine the orientation of the fall setting off an alarm. Although this method is better than 1st and 2nd , elderly people may forget to wear the device or they may feel discomfort in wearing it. The last method is based on video fall detection. These systems don’t require the user to activate or wear any device; they are automatic and detect room continuously. This paper presents an overview of the techniques and challenges of video-based fall detectors. The rest of the paper is organized as follows: in section II an overview of existing vision-based fall detection systems is presented. In subsection A of section II the capturing of video data is described, while part B discusses the main techniques for background estimation and moving object detection. The useful feature extraction of a moving human is referred to in subsection C and the next step is the determination of fall detection algorithm in subsection D. The failures and shortcomings of existing vision based systems are described in section III and finally we conclude in section IV. II. ARCHITECTURE OF VISION BASED (FALL DETECTION SYSTEM) FDS

A. Video Acquisition The starting point of the development of a FDS is the determination of the way that data is collected. Debard et al. [18] give an overview of the different approaches that may be used in order to build a camera system for capturing and processing the real-life videos, as well as simulated videos. More specifically the authors propose three different approaches to monitoring all rooms: a) by using only one Omni-directional camera, b) or one infrared camera or c) many wide-angle cameras. In order to process real time video, the minimum resolution must be at least 320x240 pixels. Then, two options can be followed: a) if a central-

2

ized system is used; the video processing comes from a process unit, usually a PC or a Digital Signal Processor (DSP) b) if a decentralized system is used, each camera is connected with its own embedded PC. Another main factor in building a camera system is the cost. Infrared cameras are not used in research because they are too expensive. Rougier et all 12 use a monocular USB camera for 3D head tracking; although in 4 present a system which covers a large space with wall-mounted cameras for 2D analysis of data. In general 3D analysis leads to more reliable systems. Another important point is the place, where the camera is located and the area of surveillance. To overcome the problem of the person’s view being obscured by furniture for example 13 is to use ceiling-mounted, wide-angle cameras with vertically-oriented optical axes. In another approach Miaou et all 14 use a MapCam (omni-camera) to capture images and to superimpose them over other images. Also the same authors in 16, use again a Map-Cam, to solve problems such as the occurrence of light source glimmer, turning a light on and off, or the hidden human over static abandoned objects.

The global Motion Estimation (GME) is used in 3 for object extraction. The global motion and local object motion(s) are separated, and then those macro blocks with significant local motions are grouped together to obtain a rough object mask. A real-time foreground-backround segmentantion algorithm (CB codebook) is introduced by Kim et all [17]. The CB algorithm adopts a quantization/clustering technique to construct a background model from long observation sequences. Bobick and Davis 6 introduce another way to estimate the moving object, the ”Motion History Image” (MHI). The MHI is an image where the pixel intensity represents the recent motion of an image in sequence, and therefore give the most recent movement of a person during an action. Rougier et all use the MHI method in 4 to overcome the problems of optical –flow in real time applications for detecting the motion, parrallel to using the CB [17] algorithm for foreground extraction. In [19] is described a general method that combines statistical assumptions with the moving object, apparent objects (ghosts) and shadow detection.

B. Moving object Tracking

C. Video processing and feature extraction

The second step after collecting data is the use of an appropriate method for estimating the background and detecting the moving object. Background subtraction is used to identify the person in the image. Difficulties in human detection occur when there are different states of motion (slow- quick), conditions of light and objects which detect as moving object. In 1 are presented the main subtraction estimation algorithms based –on the change of each pixel of the image sequences. The most popular are the a) frame differencing, b) Median filtering 2 c) linear predictive filter d) Running average e) approximate median filtering f) Kalman filtering g) Mixture of Gaussians (MOG). The a), b) and c) algorithms require the use of a buffer. On the other hand, these buffer functions will require a significant amount of memory, especially when a large buffer is used. The d), e), f) and g) algorithms don’t require a buffer. Instead they update their background image recursively. The advantage is that there should only be one frame stored and this image will be updated every time a new frame is received. The MOG is more popular than the other three for background estimation. The mixture of Gaussians method does not use one image of values as a background model, instead each pixel is modeled by a number of Gaussians. Methods employing MOG have been incorporated into algorithms that utilize Bayesian frameworks, color and ingredient information. The disadvantages of MOG are the computational complexities and the non-detection of very fast motion.

The FDS, after the moving object-human detection, focuses on moving objects. Measurements of useful parameters of objects are essential for the fall recognition. As follows, it becomes a reference to the important features of moving objects. Human shape: a bounding box [20] or an approximated elliptical blob around the human. Rougier et all 4 give a method for making an approximated ellipse. An approximated ellipse is better than a bounding box to analyze the human shape. Main axes, vertical Histograms, human centroid, are the characteristics from the analysis of human shape. These measurements give information about the orientation and the position of the body. Fall angle is defined as the angle between the person’s main axis and the ground (horizontal axes). If the angle is less than 45 degrees or more than 135 degrees it is presumed a fall 2. Aspect ratio is the height/width ratio of the bounding box around the object. Inactive period: After a fall, follows a small period without motion about 5sec. In case where a fall is detected, the system waits for a short period of time and if there isn’t any motion the alarm is activated 4. Duration of motion: Some research shows that a fall accident holds less than one second - approximately 0.4-0.8 sec. So if there is a large movement and a large variation in the main axe and fall angle, a fall is very possible. On the

3

other hand, a fall’s duration isn’t standard, so this may cause the system to be less reliable. Head Detection: Many fall detection systems track the head because it is usually visible in the scene (no occlusion problems), and has a large movement during a fall. In 7 are described two approaches for head detection: a) head detection using skin color and b) head detection using shoulder/neck profile. Also in 12 is proposed another and evolutionary approach for head detection and tracking. In Error! Reference source not found. is shown the characteristics of motion and the human position that are used for a successful fall detection.

grams

8

aspect ratio

9

Quick motion before the fall

10

Inactivity period after a fall

11

Head detection

Table 1 Features that used by references for fall detection

Counting

Category of features

Features

Papers which use the corresponding feature

5

major and minor of main axis

6

Fall angle

[Error! Reference source not found.] [Error! Reference source not found.],[Error! Reference source not found.],[Error! Reference source not found.],[Error! Reference source not found.],[Error! Reference source not found.],[Error! Reference source not found.] [Error! Reference source not found.] [Error! Reference source not found.],[Error! Reference source not found.],[Error! Reference source not found.],[Error! Reference source not found.] [Error! Reference source not found.],[Error! Reference source not found.],[Error! Reference source not found.],[Error! Reference source not found.] [Error! Reference source not found.],[Error! Reference source not found.]

7

Vertical Histo-

[Error! Reference

1

blob

shape

2

fix ellipse

3

bounding box

Shape analysis

4

Human centroid

Duration of event

Other 12

HMM

13

Inactivity Zones

14

Foot location

15

Motion History Image

source not found.],[Error! Reference source not found.],[Error! Reference source not found.] [Error! Reference source not found.],[Error! Reference source not found.],[Error! Reference source not found.],[Error! Reference source not found.] [Error! Reference source not found.],[Error! Reference source not found.],[Error! Reference source not found.] [Error! Reference source not found.],[Error! Reference source not found.],[Error! Reference source not found.] [Error! Reference source not found.],[Error! Reference source not found.],[Error! Reference source not found.] [Error! Reference source not found.],[Error! Reference source not found.] [Error! Reference source not found.],[Error! Reference source not found.] [Error! Reference source not found.] [Error! Reference source not found.],[Error! Reference source not found.]

D. D. Fall Detection Different combinations between the above referred features are used by fall detection system. Measurements of each of the above feature, determine the motion of humans, the body orientation, the variation of aspect ratio and the fall angle. Statistics and repeated methods are used to specify thresholds of each feature. The fall recognition systems present difficulties in distinguishing falls from other daily

4

activities e.g. laying on the floor, so it is critical in the thresholds and the combination of each moving human feature. In all research the orientation of the human body is used with other measurements. In most cases, when a human falls the body lies in a horizontal position on the ground. On the other hand, a fall has many possible causes, so it is difficult to develop a system with 100% accuracy. When a fall happens in a normal situation and in a surveillance area the fall is possible to detect, but there are countless cases of abnormal falls. The performance and reliability of a FDS is measured from the successful fall detection, although the percentage of errors in fall detection is important for the system’s effectiveness. As referred in [21] all research results show that there are four possible cases: •

True positive (TP): a fall occurs, the device detects it.

•

False positive (FP): the device recognizes a fall, but it is an error.

•

True negative (TN): the device does not detect a fall and this is correct. False negative (FN): a fall occurs but the device does not detect it.

•

And 2 criteria are introduced for measuring the response of the system in the four cases. TP TP + FN

•

Sensitivity:

•

Specificity: TN + FP

TN

In Error! Reference source not found. the percentage of false positives and false negatives for some research is presented. At this point, it can be observed that the scenarios of each research are so different that the comparison can not exist. Besides the scenarios, the research has differences in system architecture (type of camera, surveillance space, special circumstances etc) and in abnormal situations. III. RESULTS FROM EXISTING SYSTEMS AND OPEN ISSUES RESEARCH

As it is shown in Error! Reference source not found. the percentage of fall detection doesn’t approach 100% in any of the above referred research. Although when a fall happens in a vision-area of the camera the system can detect it but depending on the circumstances, the algorithm may present variations in performance. In a real-time system many different factors affect fall detection. First of all, the fall can be obscured by other objects or furniture. Another factor is the side, or the orientation of a fall as regards the camera. A fall in slow motion or by contrast a fall in fast

motion may not be detected by the system (negative false). Also, more importantly is the high percentage of positive false. It is difficult to distinguish a fall from other daily activities e.g: laying on the floor or sofa, sitting down, walking or squatting. So, it is required by the system to develop methods for activity recognition and detection of abnormal situations. In 2 the system tracks not only the human but also the large object that can transfer a human. In 3 2 4 4 8 9 16 methods are used for activity recognition and distinguishing fall situations. Different cases of fall direction associated with the camera are inclusive in 3 4 9. In [22], a 3D orientation vector of the human shape with an inactivity period is used for developing a fall detection context. A combination of audio and visual analysis is introduced in [20] and reduces the false positive of abnormal situations. This may be a solution for problems which face a vision based system only. The mixture of audio and optical video surveillance and sensors analysis is produced in a computational complex system, but the audio analysis or the data processing of sensors can cover the range of inaccuracy in a vision based system. As it is referred to in the Introduction a video-based system does not require the patient to wear any device and is more flexible because it detects the home automatically and continuously, but it may be undesirable for the elderly if he feels that someone is watching him. For this reason the camera must be placed in an unobtrusive place. Another point is privacy. The personal data that is collected from the camera and processed from algorithm does not need to be stored anywhere. So the person is comfortable in his home with the existence of one or more cameras. Table 2: Sequences of parameters for fall detection.

Fall Detection Refere -nces [Error! Reference source not found.] [Error! Reference source not found.]

Feature of table1

6,8,7,9

4,7,9

Description

Performance False positive

False negative

5.50%

13%

2%

0%

Abnormal situation

Both side view and front view

in a normal situation

5 [Error! Reference source not found.] [Error! Reference source not found.] [Error! Reference source not found.] [Error! Reference source not found.]

6. 6, 8, 11, PCA method for estimate the variance ratio

1%, 45%

-

in a normal activity, occluding activity respectively

7. 8.

shape analysis, 9,10

16%

9%

normal situation and falls backward total percentage of daily activities including falls

9.

10. 2, 11,shape analysis,Four layered MLP network

7.20%

2,4%

6, 5, 15,

11,3%

13,4%

in a normal situation

IV. CONCLUSIONS

11.

12.

13.

In this article the main issues of a generated FDS based on video and image processing are presented. As a model of a fall has infinite situations and factors, it is difficult to predict all the cases. In most cases in a vision-based system it is difficult to distinguish the fall from other close activities like the laying on the floor. Different techniques need to be applied for a reliable fall detection system but not only in a vision-based method. It is required to apply assistive methods for activity recognition and detect abnormal behavior.

14.

REFERENCES

17.

1.

2.

3. 4.

5.

Jared Willems, Glen Debard, Bert Bonroy, Bart Vanrumste, Toon Goedeme, 2009. How to detect human fall in video? An overview. Positioning and Context-Awareness International Conference, POCA 2009, Antwerp, Belgium (May 2009). J.Willems, G.Debard, B.Bonroy, B. Vanrumste, T. Goedeme 2009. A Video-based Algorithm for Elderly Fall Detection. Medical Physics and Biomedical Engineering World Congress, WC2009, Munchen, Germany, 10 Sept 2009 Chia-Wen Lin and Zhi-Hong Ling 2007. Automatic Fall Incident Detection in Compressed Video for Intelligent Homecare. icccn 2007: 1172-1177. Lykele Hazelhoff, Jungong Han, Peter H. With 2008. Video-Based Fall Detection in the Home Using Principal Component Analysis. Lecture Notes In Computer Science; Vol. 5259. Proceedings of the 10th International Conference on Advanced Concepts for Intelligent Vision Systems. C.Rougier, J.Meunier, A.St-Arnaud, J.Rousseau 2007. Fall Detection from Human Shape and Motion History using Video Surveillance. 21st International Conference on Advanced Information Networking and Applications Workshops 2007 IEEE.

15.

16.

18.

19.

20.

21.

A.Bobick, J.Davis 2001. The Recognition of Human Movement Using Temporal Templates. vol 23, NO.3 Transactions on Pattern Analysis and Machine Intelligence, 2001 IEEE. A. M. Tabar, A. Keshavarz, and H. Aghajan 2006. Smart home care network using sensor fusion and distributed vision-based reasoning. In Proc. of VSSN 2006, October 2006. Xinquo Yu, Xiao Wang, Panachit Kittipanya-Ngam, How Lung Eng, Loong-Fah Cheong 2009. Fall Detection and Alert for Ageing-atHome of elderly, ICOST 2009, LNCS 5597, pp. 209-216 2009. Homa Foroughi, Baharak Shakeri Aski, Hamidreza Pourreza 2008. Intelligent Video Surveillance for Monitoring Fall Detection of Elderly in Home Environments, 11ht International Conference on Computer and Information Technology (ICCIT 2008). Prabhu Rajendran, Amy Corcoran, Bruce Kinosian, Majd Alwan 2008. Falls, Falls Prevention, and Fall Detection Technologies. Eldercare Technology for Clinical Practitioners pages 187-202 SprinkerLink. M. Alwan, P. J. Rajendran, S. Kell, D. Mack, S. Dalal, M. Wolfe, R.Felder 2006. " A smart and passive floor-vibration based fall detector for elderly", The 2nd IEEE International Conference on Information & Communication Technologies: from Theory to Applications ICTTA’06, April 24 - 28, 2006, Damascus, Syria. C. Rougier, J. Meunier, A. St-Arnaud, and J. Rousseau 2006. Monocular 3d head tracking to detect falls of elderly people. In International Conference of the IEEE Engineering in Medicine and Biology Society, pages 6384–6387, August 2006. Nait-Charif, H. and S Mckenna 2004.Activity Summarisation and Fall Detection in a Supportive Home Environment. in 17th International Conference on Pattern Recognition, 2004. S.-G. Miaou, Pei-Hsu Sung, Chia-Yuan Huang, 2006. A Customized Human Fall Detection System Using Omni-Camera Images and Personal Information, In Proc. of 1st Transdisciplinary conference on Distributed Diagnosis and Home Healthcare, pp. 39-42, 2006. Bernd Schulze, Martin Floeck, and Lothar Litz 2009. Concept and Design of a Video Monitoring System for Activity Recognition and Fall Detection. Lecture Notes In Computer Science; Vol. 5597 Proceedings of the 7th International Conference on Smart Homes and Health Telematics: Ambient Assistive Health and Wellness Management in the Heart of the City, France 2009. Huang, Miaou, Liao 2009. A Human Fall Detection System Using an Omni-Directional Camera in Practical Enviroments for Health Care Applications, MVA2009 IAPR Conference on Machine Vision Applications, May 20-22, 2009, Yokohama, JAPAN. K. Kim, T.H. Chalidabhongse, D. Harwood and L. Davis 2005. Realtime foreground-background segmentation using codebook model. Real-Time Imaging 11 (3) (2005), pp. 172–185. Glen Debard, Jonas Van den Bergh, Bert Bonroy, Mieke Deschodt, Eddy Dejaeger, Koen Milisen, Toon Goedeme, Bart Vanrumste 2009. FallCam: Practical Considerations in Implementing a Camera-based Fall Detection System. PoCA conference 2009, Antwerp Belgium, May, 2009. Rita Cucciara, Costantino Grana, Massimo Piccardi, Andrea Prati 2003. Detecting Moving Objects, Ghosts, and Shadows in Video Streams. Transactions on Pattern Analysis and Machine Intelligence, VOL. 25, NO. 10 2003 IEEE. Ugur Toreyin, Yigithan Dedeoglou, Enis Cetin : HMM Based Falling Person Detection, N. Sebe, M.S. Lew, and T.S. Huang (Eds.): HCI/ICCV 2005, LNCS 3766, pp. 211–220, 2005. Springer-Verlag Berlin Heidelberg. N. Noury, A. Fleury, P.Rumeau, AK. Bourke, G.O Laighin, V.Rialle, J.E Lundy : Fall Detection- Principles and Methods, Proceedings of the 29th Annual International Conference of the IEEE EMBS, 2007 Lyon, France.

6 22. Jansen, B., Deklerck, R. 2006.Context aware inactivity recognition for visual fall detection. In: Proceedings of the first international con-

ference on pervasive computing technologies for healthcare 2006, Innsbruck, Austria (2006).