Effective Infrared Small Target Detection Utilizing a ... - IEEE Xplore

1890

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 13, NO. 12, DECEMBER 2016

Effective Infrared Small Target Detection Utilizing a Novel Local Contrast Method Yao Qin and Biao Li Abstract— Effective detection of small targets plays a pivotal role in infrared (IR) search and track applications for modern military defense or attack. Consequently, an effective small IR target detection algorithm based on a novel local contrast measure (NLCM) is proposed in this letter. Initially, difference of Gaussian band-pass filter is employed to enhance target and suppress background clutter. Then, a segmentation operation is implemented to obtain IR local regions of fixed size larger than general IR small target size. Finally, the salient map is obtained using the NLCM, and an adaptive threshold is applied to extract the target region. Experimental results on two real sequences show that the proposed method has better detection performance compared with conventional baseline methods. Index Terms— Infrared (IR) image, novel local contrast measure (NLCM), small target detection.

I. I NTRODUCTION FFECTIVE and robust infrared (IR) small target detection is of great significance to IR search and track (IRST) applications, such as precise guidance and early warning. However, particular characteristics of IR images make small target detection a difficult and challenging work. For example, due to the required long working distance of IRST systems, targets usually occupy a few pixels without discriminating shapes or texture characteristics [1]. Meanwhile, small targets are always immersed in a complex background, in which many pixel-sized noises with high brightness (PNHB) are usually included [2]. In addition, real-time target detection becomes more urgent in modern military applications. Consequently, research on real-time detection algorithms of small targets becomes essential and urgent. The algorithms dealing with small targets detection can be generally categorized into two groups, namely, the sequential detection methods and the single-frame detection methods [1]. Conventional sequential detection methods have well performance in small target detection with a prior knowledge of targets, as well as assumptions of static background or consistent targets in adjacent frames [1]. However, practical applications of IRST systems usually fail to attain preset assumptions or prior knowledge. Thus, small targets detection using a single frame has important meaning for practical applications. The single-frame detection methods concerning filtering mainly include adaptive [3] and morphological filtering [4]–[6]. Other representative methods have been developed to detect small targets, such as the combination of

E

Manuscript received May 11, 2016; revised August 13, 2016; accepted October 2, 2016. Date of publication October 28, 2016; date of current version December 7, 2016. The authors are with the Electronic Science and Engineering College, National University of Defense Technology, Changsha 410073, China. Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/LGRS.2016.2616416

empirical mode decomposition and modified local entropy [7], probabilistic principal component analysis [8], and a sparse ring representation model [9]. Additionally, approaches based on human visual system (HVS) have attracted much attention in recent years [2], [10]–[15]. At the moment, however, and to the best of authors’ knowledge, best reported algorithm among the few works tackling high-speed small targets detection problem using HVS was done by Han et al. [2]. Based on the works of local contrast measure [12], an improved local contrast measure (ILCM) on subblocks of IR images was proposed in [2]. The key parameter, namely, sliding window size in ILCM is supposed to be approximated to the size of small target. However, target size is usually changing and unpredictable in practical applications, where the IRST systems need to search and track target with high velocity; thus the detection capability of ILCM would degrade. Furthermore, since ILCM is based on mean and maximum of gray values in subblocks, ILCM would cease to be effective once clutters with high brightness of complicated background exist in the IR image. In order to design an effective and robust high-speed small target detection algorithm of complicated IR images, a novel local contrast measure (NLCM) method is presented in this letter. Concretely, the proposed algorithm is divided into three phases, namely, preprocessing, segmentation, and detection. After the preprocessing phase, a sliding window is employed to segment IR image into local regions. Then, obtained local regions are utilized to calculate the saliency map based on NLCM, and an adaptive threshold is adopted to extract the target region. II. P REPROCESSING AND S EGMENTATION P HASES The 2-D difference of Gaussians (DoGs) band-pass filter turned out to be very consistent with HVS contrast mechanism [2], [11]. Thus, the DoG filter is utilized to enhance target and suppress background clutter. Notably, salient regions at the scale adaptive to the target size are first extracted in the target detection process of HVS. Similarly, a segmentation operation is implemented in this letter to operate target detection on a region level. Specifically, local regions of the preprocessed IR image are obtained by using a sliding window from left and top to right and down. Given multiple scales of target size in practical IRST applications, the sliding window size here is set to be larger than general maximal size of small targets. In this way, a single region is hoped to contain a whole (or most part of) target of various sizes. Note that a small target is defined to have a total spatial extent of less than 80 pixels [12], ranging from 2 × 2 to about 9 × 9 pixels. Consequently, square sliding window is employed with size ranging from 10 × 10 to 12 × 12 pixels. The moving step is advised to set to 1/2 of the window’s side length. If the IR image size is M × N and the square sliding window length is W , then local region series Pi (i = 1, . . . , N0 ) are obtained, and the corresponding

1545-598X © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

QIN AND LI: EFFECTIVE IR SMALL TARGET DETECTION UTILIZING A NOVEL LOCAL CONTRAST METHOD

1891

gray matrix series are denoted by G i (i = 1, . . . , N0 ), where N0 equals to multiplication of ((M − W )/(W/2)) + 1 and ((N − W )/(W/2)) + 1. III. D ETECTION P HASE A. Saliency Map Calculation

Fig. 1.

After the segmentation phase, the key of accurate small target detection is to obtain a salient map, which can enhance regions containing the whole (or the most part of) target and suppress regions composed of background or PNHB. These three kinds of regions are denoted as target region (TR), background region (BR), and PHNB region (PR), respectively. For the purpose of enhancing targets, gray mean of local regions has been utilized in the calculation of salient map in [2], [11], and [12]. However, since background pixels are always included in local regions larger than general target size, gray mean would no longer work well in some cases. For example, if a TR with a small target occupying several pixels and a PR are obtained, gray mean of the PR and TR is scarcely discriminative on the assumption that the spatial distribution of background gray values in PR and TR is similar. Moreover, gray mean fails to enhance TR with a large region size. Consequently, an NLCM is presented to obtain salient map. Due to high brightness of target or PNHB and large size of local regions, gray variances of BR and TR (or PR) are discriminative. That means gray variance is suitable for BR suppression. Furthermore, for the sake of TR enhancement and PR suppression, an improved variance (IVar) and mean (IMean) of region gray values are then introduced. Since gray values of target pixels are usually maximal in one region, IVar and IMean of region Pi are defined as follows: IVari =

K  

j

G i − mi

2

(1)

j =1

1 j IMeani = Gi K K

(2)

j =1

where K is the number of maximal gray values considered, j G i is the jth maximal gray value of Pi , and mi is the gray mean of Pi . The optimal value of K is required to be larger than PNHB pixels number and less than target pixels number. Considering that the minimal size of target is 2 × 2 pixels and PNHB usually emerges as single pixel, K is generally set to be 2 or 3. Now, we reconsider the case of PR and TR with a target of several pixels. As mentioned above, the gray mean of the PR and TR is nearly the same. Since K is set to be 2 or 3, the K j maximal gray values of TR (G TR , j = 1, . . . , K ) are then j all target pixels, while that of PR (G PR ) is composed of both PNHB and background pixels. Consequently, IVarTR should be much larger than IVarPR because of gray differences between j j G TR and G PR . Obviously, IMeanTR is much larger than IMeanPR . Thus, IVar and IMean are efficient in discriminating TR and PR. Moreover, IVar and IMean work well in BR suppression. Therefore, an NLCM is derived from IVar and IMean. Here, u denotes the central region, where the target could appear; then eight surrounding regions can be found to calculate the NLCM of u (see Fig. 1). The NLCM of region u is

Nine local regions obtained by sliding window.

Algorithm 1 Saliency map calculation Input: Preprocessed IR image. Output: Salient map. 1: for i = 1, ..., N0 do i) Slide the window to obatin one local region. ii) Compute the IVari and IMeani . end for 2: for u = 1, ..., N1 do Compute the NLCM of region u. end for

defined as NLCM = min

IVaru × IMeanu (i = 1, . . . , 8). IMeani

(3)

If region u is TR, usually min(IMeani ) < IMeanu ; thus NLCM(TR) > IVaru , then target can be enhanced. In addition, IVarPR and IMeanPR should be smaller than IVarTR and IMeanTR , respectively; thus PNHB is suppressed. If region u is BR, min(IMeani ) is close to IMeanu . However, since IVarBR is quite small, NLCM(BR) can hardly be large; then background is suppressed. In order to distinguish NLCM from ILCM, there are two points need to be explained: First, the sliding window of ILCM is supposed to be approximated to the size of the small target, whereas region size larger than general small target size in NLCM is more generalizable. Second, gray maximal value (L n ) and mean value (mi ) are utilized in ILCM, while IVar and IMean are employed in NLCM. For each region, compute its NLCM according to (3), and form them as the salient map. In summary, the process of salient map calculation is shown in the following. Since marginal regions in IR images scarcely have eight surrounding regions, they are excluded in NLCM computation; thus N1 is smaller than N0 . B. Threshold Operation It is clear that TRs have larger IVar and IMean values than BRs or PRs. Consequently, it is likely that the most salient point in the final map is supposed to be a TR. Based on this fact, TR can be extracted by an adaptive threshold operation [2], [12]. The threshold of saliency map can be written as T =μ+k×σ

(4)

where μ and σ are the mean and standard deviation of the final saliency map, respectively. The parameter k ranging from 5 to 15 is practically proved effective in our work. Generally, several regions will be extracted after threshold operation. It is essential to select regions that are most likely to be TRs. Therefore, a winner-take-all mechanism similar to [11] is finally utilized. The largest element will be processed

1892

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 13, NO. 12, DECEMBER 2016

Fig. 2. IR small target detection results of two representative images using NLCM method. (a) Raw IR images. (b) Images after DoG preprocessing. (c) Saliency maps after NLCM calculation. (d) Detection results. TABLE I D ETECTION P ERFORMANCE OF THE P ROPOSED M ETHOD W ITH R ESPECT TO D IFFERENT R EGION S IZES AND K S ( Fa = 0.005/S EQUENCE )

first, then other elements will be processed from large to small until it is smaller than threshold [2]. In addition, once a TR is detected, the elements belonging to the same cluster will all be inhibited. IV. E XPERIMENTAL E VALUATIONS In this section, real data experiments are carried out to test the performance of the NLCM method for small targets detection in IR images. Denoted by P and Q, two sequences with single aircraft are gathered under different conditions. Specifically, backgrounds of P are composed of sunny sky, sea, and ground, whereas Q mainly contains sky cloudy background. P contains 100 images with a resolution of 250 × 300, while Q is composed of 75 images with a resolution of 194 × 310. Furthermore, an independent data set with an aircraft in the sky is utilized for validation, namely, R. R contains 73 images with a resolution of 250 × 310. All experiments are implemented by MATLAB 7.0 software on a personal computer with 2.99-GHz Pentium CPU and 1.89-GB memory. In the experiments, probability of detection Pd and falsealarm rate Fa are used as the metrics of evaluating detection performance. They can be defined as number of true TR detections (5) Pd = number of actual TRs number of false TR detections . (6) Fa = total region number − number of actual TRs Meanwhile, the average time consuming for each image is employed to describe TR detection speed.

1) Results Using NLCM Method: We select two images to evaluate the proposed method, for each sequence, one image is selected. The general parameter setup is region size being 10 × 10 or 12 × 12 pixels and K being 2 or 3. Therefore, the region size is 12 × 12 pixels, while K and k are 3 and 12, respectively. Fig. 2 shows the detection results, in which TRs are marked with rectangles. Fig. 2(a) is the raw IR images. Fig. 2(b) is images after DoG preprocessing. Clearly, both TRs are enhanced with the most saliency in Fig. 2(c); then TRs are precisely screened out in Fig. 2(d). It is obvious that the values of region size and K exert great impact on the detection performance of the NLCM method. In order to validate generalizable parameters setup, two experiments are then performed, in which receiver operating characteristic (ROC) curves are obtained by changing the segmentation thresholds for each sequence. Also, Pd for each sequence is utilized with false-alarm rate Fa fixed to 0.005 per sequence. In the first experiment, region size is fixed as 12 × 12 pixels and parameter K is set as 1–8 and then NLCM method is tested, respectively. The results of Pd and average timeconsuming for P and Q (denoted by K1 to K8 ) are illustrated in Table I. It can been seen that Pd of P reaches maximum at K = 2, whereas that of Q is maximizing when K is 7. In fact, the target size of P varies from three to ten pixels, while that of Q is more than seven pixels. That is why optimal K for P and Q is 2 and 7, respectively. However, K values of both 2 and 3 perform well in both P and Q, validating the generalization of such parameters setup. Furthermore, a smaller value of K requires less gray values of each local

QIN AND LI: EFFECTIVE IR SMALL TARGET DETECTION UTILIZING A NOVEL LOCAL CONTRAST METHOD

1893

Fig. 3. ROC curves of simulation experiments with respect to K , region size, and comparison. (a) Different K values for P. (b) Different region sizes for P. (c) Comparison to baseline methods for P. (d) Different K values for Q. (e) Different region sizes for Q. (f) Comparison to baseline methods for Q.

Fig. 4.

Detection results comparison of the two images using (a) NLCM method, (b) TopHat filtering method, (c) ILCM method, and (d) Wang’s method.

region, producing a smaller time consuming (see Table I). In addition, the ROC curves of sequence P and Q are shown in Fig. 3(a) and (d), respectively. There are two points that can be concluded from the curves: 1) NLCM method for P with K being 2 achieves best performance, while optimal K for sequence Q is 7 and 2) NLCM method with K equivalent to 2 or 3 performs quite well for both P and Q. Apparently, the two points are consistent with the results in Table I. In the second experiment, we set K as 2 and region size is 6 × 6, 8 × 8, 10 × 10, 12 × 12, 14 × 14, and 16 × 16 pixels, respectively. Here, sizem (m = 6, 8, . . . , 16) means that region size is m × m pixels. The quantitative results are illustrated in Table I. Meanwhile, ROC curves of P and Q

are shown in Fig. 3(b) and (e), respectively. It is clear that the optimal region size of P and Q is 8 × 8 and 12 × 12 pixels, respectively. The different sizes between P and Q result in discrepant optimal region size values. More importantly, it is notable that size10 and size12 reach decent results for both sequences. 2) Comparison to Baseline Methods: To further illustrate the performance of NLCM method, the TopHat filtering method, ILCM method [2], as well as Wang’s method [11] are used as baseline methods to conduct a comparative study. The same raw IR images in Fig. 2(a) are used, and the detection results of all four methods are shown in Fig. 4. The true detected TRs are labeled in rectangle, while the false detected TRs are

1894

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 13, NO. 12, DECEMBER 2016

TABLE II Pd AND C OMPUTATION C OST C OMPARISON A MONG THE P ROPOSED M ETHOD AND O THER T HREE M ETHODS

The above experiments provide empirical evidence supporting the claim that the region size of 10 × 10 (or 12 × 12) pixels and K ranging from 2 to 3 are generally suitable for small target in IR images. Specifically, NLCM1 and NLCM2 can achieve not only a fast detection speed but also well performance of Pd . V. C ONCLUSION This letter presents a robust small target detection algorithm based on NLCM. First, DoG band-pass filter is employed to enhance target and suppress background clutter. Then, a segmentation operation is implemented to obtain IR regions of fixed size larger than general IR small target size. Finally, the salient map is obtained using the NLCM, and an adaptive threshold is adopted to extract the target region. The proposed method is evaluated for small target detection by an experimental study of two real sequences. The results show that the proposed approach exhibits encouraging performance in detection accuracy of complicated IR images. However, there are limitations of the proposed algorithm. For example, when two small targets locate in the same region, the method would not be adequate with only one target detected. We are going to deal with the limitations in future work. R EFERENCES

Fig. 5. Detection results comparison of R using the NLCM method, TopHat filtering method, ILCM method, and Wang’s method.

marked with circles. The detection results of TopHat, ILCM, and Wang’s methods are shown in Fig. 4(b)–(d), respectively. Note that Fig. 4(a) is identical with Fig. 2(d). The three baseline methods failed to detect targets in the first image because of the complicated background. In fact, the false alarms are clutters with high brightness, such as windmill aground [see Fig. 2(a)]. Despite that target in second image is extracted using all baseline methods, false alarms emerge at the same time. It is clear that the NLCM method outperforms baseline methods for target detection in the two representative images. Table II and Fig. 3(c) and (f) give comparisons between NLCM method with two parameter configurations [denoted as NLCM1 and NLCM2 and the corresponding region sizes and K values are configurations of the (12 × 12, 3) and (10 × 10, 2), respectively] and the baseline methods. All four methods have the same false-alarm rate (Fa = 0.005/sequence) in Table II. It is easy to find that the NLCM method (NLCM1 and NLCM2) has better detection performance for both sequences. In addition, both NLCM1 and NLCM2 can achieve a fast detection speed. Through the ROC curves in Fig. 3(c) and (f), we can see that both NLCM1 and NLCM2 can achieve better performance than baseline methods, when Fa is below 0.005. In order to further validate the generalization of NLCM algorithm, R is utilized to conduct an unbiased comparison between the four methods. The parameters used in the three baseline methods have been optimized, while the NLCM1 and NLCM2 are employed as the representatives of the proposed algorithm. The ROC curves are shown in Fig. 5. It is clear that NLCM2 outperforms all baseline methods, while NLCM1 only performs better than TopHat filtering and Wang’s methods.

[1] C. Gao, D. Meng, Y. Yang, Y. Wang, X. Zhou, and H. G. Alexander, “Infrared patch-image model for small target detection in a single image,” IEEE Trans. Image Process., vol. 22, no. 12, pp. 4996–5009, Dec. 2013. [2] J. Han, Y. Ma, B. Zhou, F. Fan, K. Liang, and Y. Fang, “A robust infrared small target detection algorithm based on human visual system,” IEEE Geosci. Remote Sens. Lett., vol. 11, no. 12, pp. 2168–2172, Dec. 2014. [3] X. Wang and T. Zhang, “Clutter-adaptive infrared small target detection in infrared maritime scenarios,” Opt. Eng., vol. 50, no. 6, art. no. 067001, 2011. [4] A. Toet and T. Wu, “Small maritime target detection through false color fusion,” Proc. SPIE, vol. 6945, art. no. 69450V, Apr. 2008. [5] B. Xiangzhi, Z. Fugen, X. Yongchun, and J. Ting, “Modified top-hat transformation based on contour structuring element to detect infrared small target,” in Proc. 3rd IEEE Conf. ICIEA, Jun. 2008, pp. 575–579. [6] J. Zhou, H. Lv, and F. Zhou, “Infrared small target enhancement by using sequential top-hat filters,” Proc. SPIE, vol. 9301, p. 93011L, Nov. 2014. [7] H. Deng, J. G. Liu, and Z. Chen, “Infrared small target detection based on modified local entropy and EMD,” Chin. Opt. Lett., vol. 8, no. 1, pp. 24–28, 2010. [8] Y. Cao, R. M. Liu, and J. Yang, “Small target detection using twodimensional least mean square (TDLMS) filter based on neighborhood analysis,” Int. J. Infr. Millim. Waves, vol. 29, no. 2, pp. 188–200, 2008. [9] C. Gao, T. Zhang, and Q. Li, “Small infrared target detection using sparse ring representation,” IEEE Aerosp. Electron. Syst. Mag., vol. 27, no. 3, pp. 21–30, Mar. 2012. [10] X. Shao, H. Fan, G. Lu, and J. Xu, “An improved infrared dim and small target detection algorithm based on the contrast mechanism of human visual system,” Infr. Phys. Technol., vol. 55, no. 5, pp. 403–408, 2012. [11] X. Wang, G. Lv, and L. Xu, “Infrared dim target detection based on visual attention,” Infr. Phys. Technol., vol. 55, no. 6, pp. 513–521, 2012. [12] C. L. P. Chen, H. Li, Y. Wei, T. Xia, and Y. Y. Tang, “A local contrast method for small infrared target detection,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 1, pp. 574–581, Jan. 2014. [13] H. Chen, H. Zhang, F. Cheng, D. Yuan, and M. Sun, “Infrared small target detection based on visual attention,” Proc. SPIE, vol. 9443, art. no. 94430L, Mar. 2015. [14] Y. Deng and M. Wang, “Infrared dim and small target detection based on the human visual attention mechanism,” in Proc. 2nd Int. Conf. Mechatronics Autom. Control, vol. 334. 2015, pp. 1153–1161. [15] C. Ding, M. Wang, and S. O. Automation, “Infrared small target detection algorithm based on human visual attention mechanism,” J. Huazhong Univ. Sci. Technol., 2015, pp. 173–177.