Pyroelectric IR sensor arrays for fall detection in ... - Semantic Scholar

J. Phys. IV France 128 (2005) 153–160
C EDP Sciences, Les Ulis DOI: 10.1051/jp4:2005128024

Pyroelectric IR sensor arrays for fall detection in the older population A. Sixsmith1 , N. Johnson2 and R. Whatmore3 1

Department of Primary Care, Whelan Building, Brownlow Hall, University of Liverpool, Liverpool L69 3GB, UK 2 IRISYS Ltd., The Mill, Towcester, UK 3 Department of Advanced Materials, Cranfield University, Cranfield, Beds MK43 0AL, UK

Abstract. Uncooled pyroelectric sensor arrays have been studied over many years for their uses in thermal imaging applications. These arrays will only detect changes in IR flux and so systems based upon them are very good at detecting movements of people in the scene without sensing the background, if they are used in staring mode. Relatively-low element count arrays (16 x 16) can be used for a variety of people sensing applications, including people counting (for safety applications), queue monitoring etc. With appropriate signal processing such systems can be also be used for the detection of particular events such as a person falling over. There is a considerable need for automatic fall detection amongst older people, but there are important limitations to some of the current and emerging technologies available for this. Simple sensors, such as 1 or 2 element pyroelectric infra-red sensors provide crude data that is difficult to interpret; the use of devices worn on the person, such as wrist communicator and motion detectors have potential, but are reliant on the person being able and willing to wear the device; video cameras may be seen as intrusive and require considerable human resources to monitor activity while machine-interpretation of camera images is complex and may be difficult in this application area. The use of a pyroelectric thermal array sensor was seen to have a number of potential benefits. The sensor is wall-mounted and does not require the user to wear a device. It enables detailed analysis of a subject’s motion to be achieved locally, within the detector, using only a modest processor. This is possible due to the relative ease with which data from the sensor can be interpreted relative to the data generated by alternative sensors such as video devices. In addition to the cost-effectiveness of this solution, it was felt that the lack of detail in the low-level data, together with the elimination of the need to transmit data outside the detector, would help to avert feelings intrusiveness on the part of the end-user. The main benefits of this type of technology would be for older people who spend time alone in unsupervised environments. This would include people living alone in ordinary housing or in sheltered accommodation (apartment complexes for older people with local warden) and non-communal areas in residential/nursing home environments (e.g. bedrooms and ensuite bathrooms and toilets). This paper will review the development of the array, the pyroelectric ceramic material upon which it is based and the system capabilities. It will present results from the Framework 5 SIMBAD project, which used the system to monitor the movements of elderly people over a considerable period of time.

1. INTRODUCTION Recently, there has been increasing recognition of the potential of new technologies to improve health and social care services and enhance the independence and quality of life of older people [1,2,3]. Falls represent a major health hazard for the older people [4], as well as being one of the major obstacles to independent living [5,6]. Although the numbers vary throughout the literature, the estimated incidence of falls for both institutionalized and community-living persons aged over is at least 30% [7], with some figures quoting as high as 49% per year. As one would expect, the frequency of falling is considerably higher amongst more dependent older persons, such as those who live in nursing homes. It has been estimated that up to 50% of residents fall each year, and more than 40% of these may have more than one fall each year [8]. It is also important to look at the social and service dimensions of the problem of falls. Increasingly, older people are living alone with only limited support from formal and informal sources of assistance. In nursing homes, there has also been a considerable drive towards single occupancy rooms.

Article published by EDP Sciences and available at http://www.edpsciences.org/jp4 or http://dx.doi.org/10.1051/jp4:2005128024

154

JOURNAL DE PHYSIQUE IV

While the privacy of should be respected, it also has to be recognized that this increases the risk of being alone when a fall occurs. Technological solutions to the problem of falls and other emergencies have been around for many years. Community alarm systems, allow someone in difficulties to raise an alarm in a control centre by pushing a button on a special telephone or on a pendant device worn on their person. However, this is of little value if the person is incapacitated and unable to raise the alarm themselves. Recent research and development [9,10] has addressed this problem by developing intelligent systems that will trigger an alarm and an appropriate response, even when the person is incapacitated. In this sense the alarm system is a proactive, rather than a passive monitoring system. SIMBAD (Smart Inactivity Monitor using Array Based Detectors) achieves this by using low-cost, array-based passive sensor technology based on the use of pyroelectric infrared (IR) detector arrays. Pyroelectric IR detectors have used for many years in applications ranging from intruder sensing through environmental monitoring to flame and fire detection [11]. In the last 15 years there has been a growth of interest in using 2D arrays of very small pyroelectric elements for uncooled thermal imaging [12,13]. In the SIMBAD application, high image resolution is not required and may, indeed, be seen as an invasion of privacy by the subject. The array on which the system is based has been designed for adequate resolution (16 x 16 elements) coupled with low cost of manufacture. The ultimate aim of the SIMBAD fall detector is to enhance the quality of life of older persons while affording a greater sense of security and facilitating independent living. This paper reviews the SIMBAD system, the array and pyroelectric materials technology upon which it is based and preliminary results from a user trial. 2. SIMBAD SENSOR USER REQUIREMENTS For the SIMBAD system to be useful and commercially viable, it is important that the fall detector fulfils the following criteria, drawn from previous research studies [14]. The sensor should be able to detect reliably different types of falls, e.g. differentiated by factors such as rate, position, posture, etc. It should be robust and operate reliably and immediately in a range of different home environments and patterns of client activity. The issue of false-alerts was considered to be important, as was the need for low cost and simplicity of use. It was important that the solution should not be seen as an invasion of privacy. The thermal IR sensing technology was seen to be particularly appropriate in that it was an environmental installation with no requirement for the person to wear a device. The use of a sensor array would provide more detailed information on movement and activity than conventional single or dual element PIR detectors, allowing sophisticated data analysis and higher levels of confidence in fall/alert identification. It would be less intrusive and more acceptable to users than camera-based sensors and installation and maintenance was likely to be cheap, avoiding problems associated with contact sensors, etc. The cost was likely to be viable, given sufficient production volume. 3. PYROELECTRIC ARRAY AND MATERIALS The SIMBAD project uses a pyroelectric array-based passive infrared technology developed by Irisys Ltd. Fig. 1 shows a schematic diagram of a pyroelectric infra-red detector. If a thin piece of pyroelectric (the “element”) is electroded as shown, then the pyroelectric current, ip which flows through the gatebias resistor RG will be given by ip = ApdT/dt. p is the pyroelectric coefficient and dT/dt is the rate of change of the element’s temperature with time. Thus, pyroelectric devices are coupled to the changes of any input energy flux – such as infra-red radiation - which generate a change in element temperature with time. The physics of pyroelectric detectors has been reviewed previously [11]. If CE >> CA the voltage response is proportional to a figure-of-merit FV = cp

o where c = volume specific heat,
o is the permittivity of free space and
g the permittivity of the pyroelectric material. If CA >> CE the voltage response is

EMSG 2005

Incident radiation (Pow er W(t), modulated at frequency ω)

155

Electrode (Area A, Emissivity η)

VS

Vo CA

Polar axis of element CE

ip

RG

RL

d 0V

Thermal Conductance (GT)

Figure 1. Schematic diagram of a pyroelectric detector element.

proportional to Fi = cp , which would apply to maximising the response of very small elements. Hence, it can be seen that the choice of the appropriate material will depend both on the amplifier to be used and on the size of the detector element. Small area detector elements will tend to favour high dielectric constant materials and vice versa, to produce a good match between CA and CE . It is evidently insufficient to consider only the response of a detector when analysing its usefulness for a particular application. It is generally necessary to analyse both intrinsic and extrinsic noise signals and compare them with the response. A parameter frequently used in the discussion of detector performance is the specific detectivity D∗ [15]. To maximise D∗ in the region where Johnson Noise due to the loss tangent (tan
) of the pyroelectric material dominates, it is desirable to maximise the figure of merit (FoM) FD : p FD =  √ . c

o tan  The structure of a typical pyroelectric array is shown schematically in Fig. 2. These arrays are microsystem-type devices in which each pyroelectric element is interfaced to its own integrated FET amplifier, these being linked by arrays of switches for multiplexing. The devices have been made possible through the flip-chip hybridization of thin pyroelectric ceramic sensor layers with silicon application specific integrated circuits (ASICs) – known as read-out integrated circuits (ROICs). IR Absorbing Electrode

Pyroelectric Layer

ROIC Conductive bumps

Contacts

Figure 2. Schematic of pyroelectric element.

There are two main classes of pyroelectric ceramics that are used commercially in infra-red detector arrays: ceramics with a Curie temperature TC well above ambient (typically >200◦ C), so that the pyroelectric coefficient is reversible and stable and ceramics with a TC around room temperature, for which the pyroelectric coefficient must be stabilised by the application of a DC electric field. The latter group of devices are frequently referred to as dielectric bolometers. The arrays used in the SIMBAD sensor use high TC ceramics. The most widely used high TC ceramics based on modified lead zirconate (PZ) [15,16,17] and those based on modified lead titanate [18]. The morphotropic phase boundary compositions of the PZT system are not used for pyroelectric applications because they have high permittivities. The FoM for a selection of typical pyroelectric ceramics are shown in Table 1.

156

JOURNAL DE PHYSIQUE IV

Table 1. Properties of some pyroelectric ceramic materials (dielectric properties measured at 33 Hz, c’= 2.9 x 106 Jm-3 K-1 ). Material

Mod. PZ (a) Mod. PZ (b) Mod. PT

Meas. T ◦ C

p

25 25 25

4.0 3.56 3.5

10-4 Cm-2 K-1

Dielectric Properties
tan
3000.014 2180.007 2200.03

c’

FV

FD

Ref.

106 Jm-3 K-1

m2 C-1

10-5 Pa-1/2

2.9 2.9 2.9

0.054 0.063 0.063

2.33 4.57 1.61

15 17 18

Compositions a = Pb(Zr0.58 Fe0.2 Nb0.2 Ti0.02 )0.995 U0.005 O3 b = Pb(Zr0.8 [Mg1/3 Nb2/3 ]0.125 Ti0.075 )0.99 Mn0.01 O3

-1

log conductivity(Sm )

There have been various experiments to try to improve the figures of merit of modified PZ compositions. One prospect is through exploitation of the step in the spontaneous polarisation at the FR(LT) to FR(HT) phase transition [19]. This leads to a significant increase in the pyroelectric coefficient over a relatively narrow temperature range, with very little change to the dielectric permittivity or loss in the same range. This produces a significant improvement in the pyroelectric FoM. Shaw et al have studied this phase transition in the PbZrO3 -PbTiO3 -PbMg1/3 Nb2/3 O3 system [17]. An alternative technique to potentially improve the performance of pyroelectric materials is to structure them in an advantageous way. Navarro et al [20] have studied functionally-gradient materials in which a porous layer is included in the centre of a laminated stack. This reduces the average volume specific heat of the material while also reducing the thermal conductivity. In many cases, it is desirable to control the electrical conductivity of the pyroelectric ceramic. This will determine the electrical time constant of the detector element by fixing RG in Fig. 1 and fix the bias point of the FET. It will also determine the time that the array takes to “settle” when the system temperature is changed. Whatmore [21] and Stringfellow et al [22] have shown that uranium is a highly effective dopant for conductivity control in two families of modified lead zirconate ceramics and that the behaviour can be modeled in terms of hopping conduction of carriers, probably electrons, between localized sites in the ceramics. This model predicts that log (where  is the ceramic conductivity) is proportional to z-1/3 , where z is the amount of U-dopant added. Fig. 3 [22]

-17.00 -18.00 -19.00 -20.00 -21.00 -22.00 -23.00 -24.00 -25.00 3.50

PZFNTU data PZT-PMN-U data Linear (PZFNTU data) Linear (PZT-PMN-U data)

4.50

5.50

Z

6.50

7.50

-1/3

Figure 3. Electrical conductivity as a function of doping level in two uranium-doped pyroelectric ceramics (PZFNTU = Pb(Zr0.76 Fe0.10 Nb0.10 Ti0.04 )1-z Uz O3 and PZT-PMN-U = Pb{(Mg1/3 Nb2/3 )0.025 (Zr0.825 Ti0.175 )0.975 }1-z Uz O3 ) [22].

EMSG 2005

157

illustrates this for these two ceramic families. Whatmore, Molter and Shaw [23] have shown that a similar type of behaviour is followed for the Cr-doping, except that this would be expected to act as an acceptor. The fact that pyroelectric sensors are only sensitive to changes in the IR intensity means that they can be used in applications where there is a need to sense moving warm objects against a relatively unchanging background. These include people counting and sensing systems for use in retail applications. Stogdale et al [24] have reported the use of a 16 x 16 array [25] in a system for people counting and queue length monitoring applications. Fig. 4 illustrates the visible and infra-red view of people walking beneath a sensor system using this array.

Figure 4. IR and visible images using 16 x 16 array based system.

4. SIMBAD SYSTEM AND TRIALS In this application, the array sees only the moving warm objects and not the static background, greatly simplifying the signal processing required. This novel, low-cost, technology is capable of reliably locating and tracking a thermal target within the sensor’s field of view, providing size, location and velocity information. Thus, the sensor provides a much richer source of data than current home monitoring systems that rely on sensors such as door switches and conventional PIR movement detectors. Intelligent inactivity monitoring and fall detection is achieved by considering two distinct characteristics of observed behaviour. Firstly, target motion is analysed to detect characteristic dynamics of falls. Secondly, target inactivity is monitored and compared with a map of acceptable periods of inactivity in different locations within the field of view. The prototype sensor has been implemented as an embedded system, executing on a processor within the device. The system communicates with a host PC to report alarm conditions and obtain a user-defined risk map. In addition the PC interface can be used to obtain system parameters and output debugging information, allowing limited adjustment of the system during the trials. The major components of the system are:

r r r

Target tracking: The target tracking system is an elliptical gradient contour tracking system designed to identify and track an elliptical target using data from the IRISYS sensor. The system is capable of tracking targets exhibiting both positive and negative contrast with the background, provides real-time estimates of target position, velocity, and shape/size. Subtle motion detection: This is a relatively simple mechanism for the identification of small movements within the sensor’s field of view. Such movements generate insufficient responses to activate the target tracking system, and would thus otherwise be ignored. Inactivity monitoring: This uses output from the target tracking system and subtle motion detector to monitor periods of inactivity within the field of view of the detector. Once a target is no-longer visible, two distinct types of inactivity are monitored in the neighbourhood of the last known position. Coarsescale inactivity identifies the period of time since the tracker last tracked the object, whilst fine-scale inactivity identifies the period of time since subtle motion was last detected in some neighbourhood of the last know object location.

158

JOURNAL DE PHYSIQUE IV

The fall detection system employs a neural network to classify falls using vertical velocity estimates derived from IRISYS sensor data. The multi-layer perceptron neural network has been implemented as an optimised sequence of floating-point operations. A number of modifications have been made to the velocity estimation mechanism from an initial model in order to improve the robustness of the system when used within the current IRISYS detector architecture to monitor realistic scenarios (e.g filtering of spurious velocity estimates). The high-level reasoning module performs the reasoning required to monitor the output of the fall detection, inactivity monitoring, and subtle motion detection systems and generate alarm signals if required. As described below, two distinct classes of alarm are generated by the system – those triggered by excessive periods of inactivity and those triggered by the detection of a fall. Laboratory-based trials were undertaken at Liverpool University, using a prototype detector system monitoring an actress who performed a number of predefined scenarios, covering a variety of fall types, viewpoints, and degrees of obscuration in a laboratory context (Fig. 5).

Figure 5. Typical fall scenario.

Table 2. Fall classification summary. fall classification

type

(scenarios 1-14)

classification results alarm

non-alarm

alarm (14)

35.7% (5)

64.3% (9)

non-alarm (30)

3.3% (1)

96.7% (29)

Preliminary results are presented in Table 2. These were, at first sight, not very encouraging, indicating that an alarm was not triggered in almost two thirds of scenarios requiring an alarm (false negatives), although only one of the falls followed by subsequent activity did, incorrectly, cause an alarm (due to obscuration of post-fall activity). In addition, examination of the detailed results indicates that only 30% of the falls were identified by the fall detection system, well below the 70% predicted from the classifier’s ROC data, although the post-fall activity monitoring mechanism did successfully prevent alarms when activity was subsequently observed. There are a number of reasons why the classifier may have performed sub-optimally: the detector mounting position was different to that used to train the classifier; the distribution of fall types observed in the trial may be significantly different to the

EMSG 2005

159

population used to train the classifier; the modifications to the velocity estimation mechanism may affect the output of the classifier. The classification results for non-fall scenarios were, however, very encouraging, indicating that none of the non-fall scenarios resulted in alarms. Examination of the detailed results indicates that this is due mainly to the improved robustness of the fall detection system, whilst postfall activity monitoring removes any remaining potential alarms caused by rapid vertical motion such as jumping. The preliminary results provide evidence of the effectiveness of the post-fall activity monitoring mechanism, whilst indicating that the fall detection system itself requires some further development. In particular, the failure of the system to detect certain types of fall must be addressed, velocity estimation made generally more accurate and more robust, and the classifier’s characteristics re-assessed such that the output threshold may be sensibly chosen. In addition, it will probably become necessary re-train the classifier if the characteristics of the data are considered to have changed significantly. Finally, it is important to recognise that in cases where the fall detection mechanism failed, the inactivity monitoring mechanism would, given sufficient time, actuate an alarm due to the cessation of activity. A limited user trial of the sensor was undertaken over a two-month period in the living room of a single-person apartment in Merseyside, UK. The sensor was attached to a host PC which collected data including time stamps, centroid data and status flags indicating major events. The system was set up to send issue an alarm call in the event of a fall being detected. Initial testing logged a false positive triggered by the occupant leaving the room and where a failure in the velocity estimation mechanism produced a false fall classification. Subsequent inactivity prevented the negation of the alarm. The development of inactivity “risk” regions within the field of view was an important step in ensuring a low false alarm rate. Low risk areas were eventually set to allow 30 minutes of fine-scale inactivity and 1 hour course grain inactivity and for high risk areas it was set at 5 minutes and 10 minutes. This gave a final false alarm rate about 1 alarm every three days. In such a small trial it was not possible to determine the whether the system would be effective in raising an alarm due to inactivity while maintaining an acceptable level of false alarms. However, the trial did demonstrate the robustness and stability of the system. 5. CONCLUSIONS It has been successfully demonstrated that a pyroelectric movement sensor system based on a 16 x 16 sensor array, and using a ceramic sensor element based upon a modified lead zirconate material can be used for a prototype inactivity monitoring and fall detection system. Fall detection trials have demonstrated that the combination of the prototype dynamic fall detection mechanism with a post-fall activity monitoring mechanism yielded relatively poor detection rates (around 30%) for real falls but has a low false alarm rate (only a single false alarm was triggered). There is evidence, however, to suggest that the selected output threshold was inappropriate, and it is reasonable to expect that lowering this threshold would improve the detection rate without significantly increasing the false alarm rate. User trails of the entire prototype system have again demonstrated the robustness of the fall detection mechanism, with only a single false alarm, whilst demonstrating that the false alarm rate of the inactivity monitoring mechanism is rather sensitive to parameter choice. Of particular importance is the selection of regions within the risk map, where a system based, at least in part, on learning from observation is likely to be required. The research presented in this paper indicates that the SIMBAD sensor offers a potentially significant enhancement to the functionality and effectiveness of existing monitoring and community alarm systems. The commercial potential is also significant, given that the sensor is self-contained and could be relatively easily integrated into existing alarm systems. References [1] United Nations Assembly on Ageing (2002) International Plan of Action 2002 www.un.org/ageing accessed 12th May

160

JOURNAL DE PHYSIQUE IV

[2] J. Graafmans, V. Taipale and N. Charness, Gerontechnology: A Sustainable Investment in the Future, (IOS Press, Amsterdam) (1998) [3] European Commission (1996) Telematics Applications Programme, Disabled and Elderly Sector: Project Summaries. DGXIII-C5, Brussels [4] H. K. Edelberg, Evaluating Fall-Prevention Strategies, Medicine and Behaviour. www.medinfosource.com (1998) [5] J.S. Fulks, Generations Review, 12, 2 (2002) [6] K. Doughty, R. Lewis and A. McIntosh The Design of a Practical and Reliable Fall Detector for Community and Institutional Appliances. Presented at TeleMed 99, London, November (1999) [7] E. Duthie, Medical Clinics of North America, 73, 1321 (1989) [8] R. Tideiksaar, Falling in old age: prevention and management. 2nd ed. (NewYork: Springer) (1998) [9] A. Sixsmith, Journal of Telemedicine and Telecare, 6, 63 (2000) [10] Doughty et al (1999) Ibid [11] R.W. Whatmore, Rep. Prog. Phys., 49, 1335 (1986) [12] R. Watton, Ferroelectrics, 91, 87 (1989) [13] R. Watton, P.N. Dennis, J.P. Gillham, P.A. Manning, M.C. Perkins and M.A. Todd, Proc. SPIE, 2020, 379 (1993) [14] Sixsmith (2000) Ibid [15] R.W. Whatmore and F.W. Ainger, Proc. SPIE, 395, 261 (1983) [16] R.W. Whatmore and A.J. Bell, Ferroelectrics, 35, 155 (1983) [17] C.P. Shaw, S. Gupta, S.B. Stringfellow, A. Navarro, J.R. Alcock and R.W. Whatmore, J. European Ceram. Soc., 22, 2123 (2002) [18] M. Nakamoto, N. Ichinose, N. Iwase and Y. Yamashita, J. Ceram. Soc. Jpn., 110, 639 (2002) [19] R. Clarke, A.M. Glazer, F.W. Ainger, D. Appleby, N.J. Poole and S.G. Porter, Ferroelectrics, 11, 359 (1976) [20] A. Navarro, J.R. Alcock and R.W. Whatmore, J. Electroceramics, 13, 413 (2003) [21] R.W. Whatmore, Ferroelectrics, 49, 201 (1983) [22] S.B. Stringfellow, S. Gupta, C. Shaw, J.R. Alcock and R.W. Whatmore, J. European Ceram. Soc., 22, 573 (2002) [23] R.W. Whatmore, O. Molter and C.P. Shaw, J. European Ceram. Soc., 23, 721 (2003) [24] N. Stogdale, S. Hollock, N. Johnson and N. Sumpter, Proc. SPIE (2003) [25] M. Mansi, S.G. Porter, J.L. Galloway and N. Sumpter, Proc. SPIE (2001)