Human detection using OT-MACH filter in cluttered ...

Human detection using OT-MACH filter in cluttered FLIR imagery Ahmad Alkandri(Author), Nagachetan Bangalore, Akber Gardezi, Philip Birch, Rupert Young, Chris Chatwin Department of Engineering and Design School of Engineering and Informatics University of Sussex, Brighton, United Kingdom, BN1 9Q

Abstract

a. Correlation peak localisation. b. Distortion tolerance.

An improvement to the Optimal Trade-off Maximum Average Correlation Height (OT-MACH) filter with the addition of a Rayleigh distribution filter has been used to detect humans in FLIR imagery scenes. The Rayleigh distribution filter is applied to the OT-MACH filter to provide a sharper low frequency cut-off which improves the OT-MACH filter performance in terms of target discrimination. The OT-MACH filter has been trained using a Computer Aided Design (CAD) model and tested on the corresponding real target object in high clutter environments acquired from a Forward Looking Infra Red (FLIR) sensor. Evaluation of the performance of the Rayleigh modified OT-MACH filter is reported for the recognition of humans present within the thermal infra-red image data set. Keywords: MACH filter; OT-MACH filter; pattern recognition; FLIR; Rayleigh distribution filter, human detection

I. INTRODUCTION

c. Suppression of noise/clutter. In this paper we will use a frequency domain Optimal Trade-off Maximum Average Correlation Height (OTMACH) with transfer function [2]:

h

m *x C Dx  Sx

where α, β and γ are OT-MACH non-negative parameters, m* is the average of the training images (in the frequency domain), C is additive noise and Dx is the  diagonal average power spectral density of the training images defined as: Dx 

1 N *  Xi Xi N i 1

Sx is the similarity matrix of the training images and is defined as: *

The ability to detect, classify and identify targets in a scene or video sequence is one of the main goals in image processing and pattern recognition. Correlation filters are one of the preferred methods used in such applications due to their ability to discriminate objects from a cluttered background. They are well known for their ability to provide shift-invariance and distortion tolerance, which also makes their use attractive for pattern recognition applications. The Maximum Average Correlation Height (MACH) filter [2] is one of the most effective filter design algorithms since it permits the control of three important features:

Sx 

1 N X i  M x  X i  M x  N i 1

where Xi is a diagonal matrix of the ith training image. The additive noise has been automatically adjusted in power using the coefficient of variation technique [1]. II. THE FLIR DATA A. FLIR imagery Some examples of the data used to test the OTMACH filter’s ability to discriminate and detect a predefined human from a noisy cluttered background are shown in Figure 1 below.

ultimate solution will be the capability to design and render 3D models of any target description, which can then be used in any desired detection scenario.

(a)

(b)

(c)

III. HUMAN DETECTION USING THE RAYLEIGH DISTRIBUTION MODIFIED OTMACH FILTER

Detecting humans in cluttered backgrounds is a demanding task, (d) Figure 1

(e)

(f)

FLIR imagery from multiple sensors showing human activity

The filter has been tested with real data sets from different platforms and application scenarios. The Nissan patrol data has been acquired by the Ranger HRC FLIR imager with a 640x480 focal plane array operating between 3-5 μm wavelengths and is used for border security in Northern Kuwait. The test images vary in resolutions and the time of day of acquisition (i.e. day and night). B. CAD Traning set The OT-MACH filter has been trained with twodimensional views derived from rendered CAD models of a human CAD model. Several out-of-plane rotations (0° to 360°) of the selected humans have been used. Figure 2 below shows a few examples of the training set images.

C. Rayleigh distribution filter A correlation filter design has to make a compromise between to in the filter transfer function. The filter then will be band pass in nature; however, recent work has shown that by explicitly forcing a band pass structure on the filter to ensure, in particular, that the high amplitude low frequency spectral components are fully suppressed, can enhance further the performance of the OT-MACH filter [10-12]. In this paper we improve the OT-MACH filter with a band pass filter constructed using a Rayleigh filter described by the Rayleigh probability density function [13]: f r ,  

r



r 2

e 2 , r  0 2 2

where a radial co-ordinate, r, is used to produce an isotropic, circularly symmetric 2-D distribution and σ2 is the variance parameter that controls the filter width. The Rayleigh filter is applied to the OT-MACH filter transfer function.

Figure 2. Example training images - 3D CAD models of human.

The advantage of using FLIR images over visible band images is the ability to discriminate the humans from the background due to their thermal difference. The use of two-dimensional projected views (derived from CAD models) as the training images enables the OTMACH filter to efficiently detect humans in the scenes an is due to the presence of enhanced and complete edges in the training set as compared to the actual target data set. The rendering of the CAD models to match the FLIR signature of the target allows them to be effectively correlated against the FLIR data. The

Figure 3. Plot of the Rayleigh probability density function showing the sharp cut-off at low spatial frequencies.

As illustrated in Figure 3, the Rayleigh distribution has the characteristic of a sharp fall at the center of the

(circularly symmetric) distribution which when applied in the frequency domain will remove the zero and low spatial frequency content of the filter transfer function. The filter falls smoothly towards higher spatial frequencies and so acts as a band pass pre-processing filter to the OT-MACH filter. D. Correlation plane measurement methods Several measures have been used to quantify correlation filter performance [14]; in this paper, peakto-sidelobe ratio has been used to quantify the correlation peak quality. The peak height is compared to the surrounding side lobe peak values and is given as:

PSR =

Peak - m

sc

where μ is the mean and σc is the standard deviation of the correlation plane pixel intensities. It is a common measure used to describe the peak quality and the degree of match between the target and the input image.

Figure 4. detection result of image (a) as shown in figure 1. The top plot is the OT-MACH correlation plane and the bottom is the OTMACH Rayleigh modified filter correlation plane. TABLE I. PSR RESULT FOR IMAGE (A) SHOWNIN FIGURE 1 Image

β

γ

OT-MACH PSR

OT / RAY PSR

A

0.001

0.001

24.93

39.45

IV. RESULTS DEMONSTRATING HUMAN DETECTION IN VARYING CLUTTER BACKGROUNDS E. General Tests have been conducted on the sample FLIR images both with, and without, the use of the Rayleigh distribution filter. The OT-MACH parameters β and γ have been fixed to a value of 0.001. This value has been found in testing to be near optimal for all the sample images. Although we can get better discrimination ability by adjusting the parameters between tests, for this comparison we set both filters with the same parameter values to unify the benchmark set for both filters. Initially, we have tested the FLIR images without the addition of the Rayleigh distribution filter and then the same tests have been conducted with the use of the Rayleigh distribution filter, the results being presented in the following section.. F. Test result Multiple tests have been conducted on the sample images, example correlation plots and PSR results being shown in the figures and tables below.

Figure 5. detection result of image (b) as shown in figure 1. The top plot is the OT-MACH correlation and the bottom is the OT-MACH & Rayleigh filter correlation.

TABLE I.

PSR RESULT FOR IMAGE (B) SHOWN IN FIGURE 1

Image

β

γ

OT-MACH PSR

OT / RAY PSR

B

0.001

0.001

24.93

31.88

Figure 6. detection result of image (c) as shown in figure 1. The top plot is the OT-MACH correlation and the bottom is the OT-MACH & Rayleigh filter correlation.

TABLE II.

PSR RESULT FOR IMAGE (C) SHOWN IN FIGURE 1

Image

β

γ

C

0.001

0.001

OT-MACH PSR -

OT / RAY PSR 28.44

Figure 7. detection result of image (d) as shown in figure 1. The top plot is the OT-MACH correlation and the bottom is the OT-MACH & Rayleigh filter correlation.

Figure 8. detection result of image (e) as shown in figure 1. The top plot is the OT-MACH correlation and the bottom is the OT-MACH & Rayleigh filter correlation.

TABLE IV. PSR RESULT FOR IMAGE (E) SHOWN IN FIGURE 1 Image E

β 0.001

γ 0.001

OT-MACH PSR -

OT / RAY PSR 25.24

Figure 9. detectoin result of image (f) as shown in figure 1. The top plot is the OT-MACH correlation and the bottom is the OT-MACH & Rayleigh filter correlation. TABLE V. PSR RESULT FOR IMAGE (F) SHOWN IN FIGURE 1

TABLE III. Image D

β 0.001

PSR RESULT FOR IMAGE (D) SHOWN IN FIGURE 1 γ 0.001

OTMACH PSR 18.04

OT / RAY PSR 26.16

Image F

β 0.001

γ 0.001

OT-MACH PSR 10.05

OT / RAY PSR 24.7

The above results clearly show how the addition of the Rayleigh distribution filter improves the OT-MACH overall performance. It can be observed that the PSR result is higher with the addition of the Rayleigh

distribution filter. In the results derived from the images shown in Figure 1 (c) and (e) the OT-MACH filter did not manage to suppress the background clutter enough which resulted high false peaks exceeding half of the detection peak which thus affected the PSR result. However, the addition of the Rayleigh preprocessing filter to the OT-MACH filter has resulted in quite noticeable improvements to the PSR results. The addition of the Rayleigh distribution filter results in sharper correlation peaks and a smoother correlation plane that makes the detection of a human in the cluttered scenes by detecting a clear correlation peak more reliable.

for FLIR imaging in high clutter environments", Proc. SPIE 8055, 805508 (2010). [3]

Mahalanobis, A., Vijaya Kumar, B. V. K, Song S., Sims, S. R. F., Epperson, J.F. “Unconstrained correlation filters,” Appl. Opt. Vol. 33, pp. 3751–3759 (1994).

[4]

Zhou H, and Chao, T.-H., "MACH filter synthesizing for detecting targets in cluttered environment for grayscale optical correlator", Proc. SPIE 3715, 394 (1999).

[5]

Tan, S. Young, R. C. D., Chatwin, C. D., “Markovian and Auto-regressive Clutter Noise Models for a Pattern Recognition Wiener Filter”, Applied Optics Vol. 41, No. 32, p. 6858-6866, (2002).

V. CONCLUSION

[6]

Johnson, O. C., Edens, W, Lu, T. T., and Chao, T-H., "Optimization of OT-MACH filter generation for target recognition", Proc. SPIE 7340, 734008 (2009).

In this paper we have shown that humans can be detected in cluttered FLIR imagery. The OT-MACH filter parameters have been modified to find the optimal configuration that was determined to be 0.001 for both the beta and gamma parameters. The frequency domain application of the Rayleigh filter to the OT-MACH filter function was employed to enhance the discrimination capability of the filter design, leading to a reduction in the number of peaks present in the correlation plane by tuning the filter to an appropriate band pass to provide a suitable compromise in filter response between distortion tolerance and resistance to clutter. The overall result has shown a significant improvement to the correlation plane and subsequent target detection. The proposed technique thus allows a good starting point for the further optimization of the OT-MACH filter design to allow better detection and recognition of human targets using infra-red imagery of targets in highly cluttered scenes.

[7]

Bahri, Z., Kumar, B. V. K., “Generalized synthetic discriminant functions”, JOSA, Vol. 5, pp. 562-571 (1988).

[8]

Refregier, Ph., “Optimal trade-off filters for noise robustness, sharpness of the correlation peak and Horner efficiency”, Optics Letters Vol. 16, pp. 829-831 (1991).

[9]

Mahalanobis, A., Vijaya Kumar, B. V. K., Casasent, D. , “Minimum average correlation energy filters”, Appl. Opt. Vol. 26, pp. 3633-3640 (1987).

[10] Kumar, B. V. K., Hassebrook, L., “Performance measures for correlation filters,” Appl. Opt. Vol. 29, pp. 2997–3006 (1990). [11] Brundstorm, K., Schenkman, B.N., Jacobson, B., “Object detection in cluttered infrared images”, Opt. Eng. Vol. 42, pp. 388-399 (2003). [12] Mahalanobis, A., Vijaya Kumar, B. V. K., “Optimality of the maximum average

correlation height filter for

detection of targets in noise”, Opt. Eng. Vol. 36, pp. 26422648 (1997).

REFERENCES

[13] Strehl, A., Aggarwal, J.K., “Detecting moving objects in [1]

Ahmad Alkandri, Akber Gardezi, Philip Birch, Rupert

airborne forward looking infra-red sequences”, Journal of

Young and Chris Chatwin, "Parameter optimization of the

Machine Vision Applications, Vol. 11, pp.267-276

optimal trade-off maximum average correlation height filter (OT-MACH) for FLIR imaging in high clutter environments",

[2]

Proc.

SPIE

8055,

805508

(2011);

(2000). [14] Goyal, S., Narveen, N., Beri, V., Gupta, A., “Waveletmodified maximum average correlation height filter for

doi:10.1117/12.882823.

rotation invariance that uses chirp encoding in a hybrid

Al-Kandri, A., Gardezi, A., Birch P., Young, R., Chatwin,

digital-optical correlator”, Appl. Opt. Vol. 45, pp. 4850-

C., "Parameter optimization of the optimal trade-off

4857, (2006).

maximum average correlation height filter (OT-MACH)