2010 International Conference on Pattern Recognition
Patchy Aurora Image Segmentation Based on ALBP and Block Threshold Xinbo Gao1, Yongjun Jian1
Rong Fu1,2
1.
2.
School of Computer Science Xi'an Polytechnic University Xi’an, China
[email protected]
School of Electronic Engineering Xidian University Xi'an China
[email protected],
[email protected]
Abstract—The proportion of aurora region to the field of view is an important index to measure the range and scale of aurora. A crucial step to obtain the index is to segment aurora region from the background. A simple and efficient aurora image segmentation algorithm is proposed, which is composed of feature representation based on adaptive local binary patterns (ALBP) and aurora region estimation through block threshold. First the ALBP features of sky image are extracted and the threshold is determined. The aurora image to be segmented is then equally divided into detection blocks from which ALBP features are also extracted. Aurora block is estimated through comparison its ALBP features with the threshold. Simple as it is, processing in huge data set is possible. The experiment illustrates the segmentation effect of the proposed method is satisfying from human visual aspect and segmentation accuracy.
diurnal patchy aurora consists of aurora rays being dimmed from bright to dark, blending into the background finally. It presents blurry shape and ambiguous edges. Thus it is hard to be segmented accurately by the same way of the arcs. Through the analysis of patchy aurora, the texture structure constructed by aurora rays is quite dissimilar to the texture structure of sky. Thus a segmentation algorithm based on texture structure invariance is proposed, which consists of ALBP features representation and block threshold segmentation. II.
The local binary pattern (LBP) [6] is one of the best texture methods available today and uniform local binary patterns are commonly used as features. In this method, if the gray value of the pixel in the neighborhood is greater than the gray value of the central pixel, the pixel in the neighborhood is set 1, otherwise is 0. Assume that gc is the gray value of the central pixel, gi is the gray value of the ith pixel in neighborhood, P means the number of pixels and R denotes the radius of the neighborhood, U is the number of the transitions between 1 and 0 in the pattern. If U is not greater than 2, the pattern is named uniform pattern [6].
Keywords-image segmentation; LBP; aurora
I.
INTRODUCTION
Aurora is a permanent feature of the Earth environment, which is produced by the collision of charged particles from Earth's magnetosphere and solar wind [1]. Therefore it is an important approach to monitor and investigate the physical processes in near-Earth space for geosciences. The proportion of aurora region to sky (PAS) in all-sky images is one of the significant indexes of describing the scale of aurora. In order to obtain the PAS of aurora image, a crucial step is to segment the aurora region from the background. Different categories of aurora present different morphologies which should be segmented by different approaches. Syrjasuo [2] identified three distinct categories of aurora appearance in 2004: arcs, patchy and omega-bands. Our goal is the segmentation of diurnal patchy aurora, since they are the domain forms of aurora at magnetic noon, which reflect the dynamics process of the interaction of solar wind and earth magnetosphere [1]. According to their characteristics, patchy auroras are divided into three subcategories [3]: drapery corona, radial corona and hot-spot as Fig. 1. Image segmentation has been very active in recent years, however only a few applications for aurora image segmentation are proposed. Syrjasuo considered clear aurora shapes for contented based retrieval [4][5]. The algorithm is just applied to a special part of aurora with salient shape, like arcs. Whereas our goal is the diurnal patchy aurora, its structure is much more complicated than arcs. Most of 1051-4651/10 $26.00 © 2010 IEEE DOI 10.1109/ICPR.2010.825
FEATURE REPRESENTATION
LBP ri ,u 2 P,R
⎧ P −1 ⎪ ∑ b , if U ≤ 2 = ⎨ i =0 i ⎪ P + 1, otherwise ⎩ (1)
where ⎧1, if gi ≥ gc bi = ⎨ ⎩0, if gi < gc
In (1) if U≤2 is met, the number of 1s is employed as the ULBP label for the central pixel, else instead of (P+1).
(a)
(b)
(c)
Figure 1. Three subcategories of patchy auroras: (a) drapery corona, (b) radial corona, and (c) hot-spot.
3368 3384 3380
67
89
42
53
85
100
97
19
22
0
Center pixel as threshold
1
0 1
A. Block threshold computation To estimate which block contains aurora rays, the system should be trained first by the images which do not or hardly contain aurora. In this step, ALBP is first extracted from the whole sky image to get the reference feature vector A. The sky image is then divided into same size blocks called detection block. ALBP descriptors of each block are computed pixel by pixel to extract the block feature vector B as Fig. 3(a). Through the similarity measurement between A and B, the threshold is obtained. To guarantee that every block contains adequate feature information, the block should be large enough. As a rule, each block contains at least 100 ALBP masks. The LBP mask is L×L, W ought to satisfy:
0 1
0
0
Figure 2. ALBP square neighborhood P=8, L=3.
The uniform patterns can effectively represent the textures composed of straight and low curvature edges, line endings, and corners. However, patchy aurora contains a lot of complicated shapes and most of them cannot satisfy the condition. Although these shapes contain a lot of texture information, they are assigned to nonuniform type and not considered as features. ULBP is insufficient to capture the textual information of aurora, it is unnecessary to utilize the whole possible patterns either. In fact, the occurrence frequencies of different patterns varied greatly and even some of them rarely occurred in a kind of texture images [6]. The proportions of these patterns are very small and dispensable for features description. Therefore in this paper the frequently occurred patterns are selected as the features. ALBP method is detailed as follow. 1) Step1. Main pattern set construction Assume there are M samples in the training set and N points of each sample image. When the number of neighborhood is P and LBP mask is L×L, each LBP mask has Q different patterns. The histograms of rotation invariant LBP in the training set are summarized and regulated to get the average histogram denoted as SumH which is sorted in descending order. The first G patterns, whose summarized probabilities are greater than or equal to a threshold (90% used in this paper), is selected to constructed the main pattern set named as SubLBP. 2) Step2. ALBP features extraction According to SubLBP, the main G patterns are selected from the histogram of rotation invariant LBP of the testing image. The probabilities and corresponding pattern labels of the G patterns are treated as the ALBP features of the image. In traditional versions of LBP, selecting neighborhood in circular form is to guarantee the algorithm invariant to rotation. Since orientation is an important physical property for aurora, selecting circular neighborhood is unnecessary. On the other hand, computing gray values which do not fall exactly in the center of pixels using interpolation in circular neighborhood takes much time, it is unsuitable for real-time processing. Therefore, in the proposed method, a square neighborhood is considered, which is used as the ALBP processing unit called ALBP mask. P is the number of pixel in the square neighborhood and L is the width of each side. The example of ALBP8,3 is illustrated in Fig. 2.
2
(W − L + 1) ≥ 100
(2)
Three resolutions are used in the experiments: ALBP8,3, ALBP16,5 and ALBP24,7. The maximum value of L is 7, therefore the corresponding W should be greater than or equal to 16 depending on (2). Besides the size of the block, the moving step of the block is another important factor affecting the threshold. The smaller the moving step is, the more accurate the threshold gets. Though the increasing overlapping part will decrease detection speed, the step length of 2 pixels is considered, because the training stage is an off-line process and the training set only consists of a limited number of samples. The feature vector of each block is measured against the reference feature vector A of the whole image to find the most different block. Assuming that there are N detection blocks in one image, the ALBP descriptor of the jth block is Bj. The Chi-square likelihood ratio for jth block is calculated as follows: G
2
K Train = X ( B j , A) =
∑ i =1
( Bij − Ai ) Bij + Ai
2
, j = 1, 2,..., N
(3)
G is the number of the elements in main pattern set. Bij is the ith pattern in the jth block. The smaller KTrain is, the more the jth block is similar to the whole sky image. Select the maximum value of KTrain to be the threshold T as (4). T = max( K Train ), j = 1, 2,..., N Moving step in training stage
ALBP mask
(4) Moving step in testing stage
Detection block
III.
AURORA IMAGE SEGMENTATION This phase is composed of two parts: training to obtain the block threshold from sky images and estimating the aurora block according to the threshold to segment aurora region from the field of view.
Figure 3. LBP mask and detection block: (a) image of sky, and (b) image containing aurora.
3381 3385 3369
(a)
(b)
performance of the proposed method. In the segmentation outcomes, the white area corresponds to aurora and black area is sky. Arcs aurora, consisting of one or more salient bands, is much brighter than sky; the luminance of the bands is even and changes a little. Thus this kind of aurora is suitable to be segmented by OTSU method. The segmentation results in Fig. 4 illustrate that the proposed method is as good as the OTSU method at segmenting the salient bands in arcs aurora. Patchy auroras have two kinds of histogram distributions illustrated in Fig. 5. The histogram consisting of two peaks in Fig. 5(a) corresponds to the patchy auroras which are composed of rays and patches. The luminance of the rays and patches of this type is different and maps to two peaks. On the contrary, the right histogram containing one peak in Fig. 5(b) corresponds to the patchy auroras only composed of rays whose luminance is close to the background and results in one peak. Therefore the experiments of patchy aurora include two groups: y Group 1: patchy auroras containing patches and rays y Group 2: patchy auroras only containing rays In Fig. 6, the texture part constructed by rays is an integral part for group 1. The rays are not detected by OTSU, however both of the rays and the patches are segmented by proposed method. As Fig. 7(a) patchy auroras of group 2 are only consisting of rays that are a little brighter than background. The segmentation results of this group by OTSU are opposite to the ideal result, in which, most of the white area actually does not belong to aurora but to sky. Compared with Fig. 7(b), the segmentation result of proposed method in Fig. 7(c) is similar to the actual aurora area. The reason of the error segmentation result caused by OTSU is attributed to its mechanism which divides the object from the environment depending on luminance variance between the foreground and background. The auroras only have one peak in the histograms as Fig. 5(b), which is insufficient to be utilized by OTSU. However, the threshold of the proposed method does not relay on the luminance but the texture structure, hence the segmentation results are not affected by the insufficient luminance feature.
(c)
Figure 4. The segmentation results: (a) original image, (b) OTSU, and (c) proposed method.
B. Aurora Block Estimation To improve the segmentation accuracy, the image to be segmented is divided into overlapping blocks. In order to get the most suitable moving step, both of the detection accuracy and the computational complexity should be considered, as this step is on-line processing. Based on the simulation results, if the overlapping step of the detection blocks is W/2, the proposed method presents satisfying performance in detection precision and time cost. Assume that D is the ALBP feature vector of detection block in the testing image. The similarity measure between D and A is represented by KTest. G
K Test = X ( D, A) = ∑ 2
i =1
( Di − Ai )
2
(5)
Di + Ai
KTest is then compared with the block threshold T to estimate which block contains aurora:
⎧aurora
KTest > T
⎩ sky
K Test ≤ T
Result = ⎨
(6)
In (6), if KTest is less than or equal to T, the block is labeled as sky block and set black, otherwise it is aurora block and set white. IV. EXPERIMENT SETTING AND IMPLEMENT The aurora data used in this paper were obtained from the all-sky imagers at Yellow River Station (YRS) in Ny-lesund, Svalbard. A. Comparison in vision The experiments are designed into three groups, one is composed of arcs aurora and the other two are patchy auroras with different histograms. All of them are segmented by proposed method. The results are also compared with the classical segmentation method OTSU [7] to evaluate the 4000
(a)
(b)
(c)
Figure 6. The segmentation results of group 1: (a) original images, (b) OTSU, and (c) proposed method.
5000
3500
4500 4000
3000
3500 2500
3000 2000
2500
1500
2000 1500
1000
1000 500
500
0
0 0
50
100
150
(a)
200
250
0
50
100
(b)
150
200
250
(a)
(b)
(c)
Figure 7. The segmentation results of group 2: (a) original images, (b) OTSU, and (c) proposed method.
Figure 5. The two kinds of histograms of patchy auroras: (a) patches and rays, and (b) only rays.
3382 3386 3370
Besides the error rate, the processing speed is also of importance, since all-sky imagers (ASI) capture millions of pictures annually. The comparison of time cost in the process of segmentation is showed in Table 2. ALBP8,3 and ALBP16.5 is a little slower than OTSU and is much fast than ALBP24,7. Considering the error rate and time cost, ALBP16,5 is also comparatively applicable to rapidly increasing ASI image database.
The experiments of two groups illustrate that the segmentation outcomes of the patchy auroras by proposed method is better than OTSU in vision aspect. B. The detection error rate A segmentation error rate algorithm is designed in this paper, which is also another scale to measure the performance of the algorithm: RError
=
( Eaurora + Esky ) N all
V.
(7)
In (7), RError represents the error rate of segmentation. The less the error rate occurs, the better the segmentation effect is. Nall means the number of the nonoverlapping blocks. Eaurora is the number of the detection blocks which should be labeled aurora but be labeled sky. Esky denotes the number of the detection windows which ought to be labeled sky but be labeled aurora. The experiments are applied to the 3 categories of patchy auroras respectively, which consist of 200 samples per group selected randomly. The sample images are segmented manually by experts previously to compare with the aromatically processing results by different algorithms. Table 1 illustrates that the error rate comparison between ALBP and OTSU method. ALBP is advanced from traditional LBP, thus the comparison between different LBP is necessary. Table 1 also lists the segmentation error rates of ULBP and ALBP in different resolutions. In Table 1, all the error rates of ALBP are much lower than OTSU method. ALBP24,7 is the best one and ALBP16,5 is only approximately 0.5% higher than it. The error rates of ULBP are also lower than OTSU but higher than ALBP on average. ULBP divides patterns into uniform and nonuniform. Even some patterns occur frequently and contain a lot of texture information; they are not treated as features because they are labeled nonuniform. ALBP selects the frequent occurrence pattern as features regardless uniform or not, which achieves more objective feature representation of image and lower error rates than ULBP. TABLE I. OTSU ULBP8,3 ULBP16,5 ULBP24,7 ALBP8,3 ALBP16,5 ALBP24,7 TABLE II.
THE SEGMENTATION ERROR RATES COMPARISION (%) Drapery 27.18 12.91 6.11 5.46 7.14 4.76 4.55
Hot-spot 19.55 10.47 5.95 4.15 7.34 4.24 2.91
1.22
ALBP8,3
1.58
ALBP16,5
ALBP24,7
3.01
11.37
The algorithm designed for patchy aurora segmentation provides a powerful aurora research tool. It is based on the different texture structure between the patchy aurora and sky. To evaluate the validity of the proposed method, the proposed method in three different resolutions are implemented and compared with the classical segmentation method OTSU. The experiments denote the proposed algorithm obtains lower error rate and is satisfying in time cost. Compared with ALBP24,7, ALBP16,5 is much faster and is more suitable for huge dataset. ACKNOWLEDGMENT We would like to thank the helpful comments and suggestions from the anonymous reviewers. This research was supported by the R&D Special Fund for Public Welfare Industry (meteorology) (GYHY200706043), supported by the Ph.D. Programs Foundation of Ministry of Education of China (No. 20090203110002) and the Natural Science Basic Research Plane in Shaanxi Province of China (2009JM8004). We also would like to thank YRS for providing labeled samples of aurora. REFERENCES [1]
[2]
[3]
Average 22.32 12.28 6.26 5.31 7.95 4.29 3.86
[4]
[5]
THE TIME CONSUMING PER IMAGE OF OTSU AND ALBP IN DIFFERENT RESOLUTIONS (S) OTSU
Time cost(s)
Radial 20.24 13.46 6.73 6.31 9.38 3.87 4.13
[6]
[7]
3383 3387 3371
CONCLUSIONS
H. G. Yang, and et al., Multiple wavelengths observation of dayside auroras in visible range-A preliminary result of the first wintering aurora observation in Chinese Arctic Station at Ny-Alesund. Chinese Journal of Polar Research, vol. 17(2) ,2005, pp. 107-114. M. T. Syrjasuo, E. F. Donovan, and et al., Automatic classification of auroral images in substorm studies. International Conference on Substorms (ICS8), University of Calgary, Alberta, Canada, 2007, pp. 309-313. Z. J. Hu., H. G. Yang, D. Huang and et al., Synoptic distribution of dayside aurora: Multiple-wavelength all-sky observation at Yellow River Station in Ny-Alesund, Svalbard. Journal of Atmospheric and Solar-Terrestrial Physics, vol. 71, 2009, pp. 794-804. M. T. Syrjasuo, E. F. Donovan and L. L. Cogger, Content-based retrieval of auroral images-thousands of irregular shapes. International Conference on Visualization, Imaging, and Image Processing, Marbella, Spain, 2004, pp. 224-228. M. T. Syrjasuo, and E. F. Donovan, Using relevance feedback in retrieving auroral images. International Conference on Computational Intelligence, Calgary Alberta, Canada, 2005, pp. 420-425. T. Ojala, M. Pietikinen, and T. Menp, Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24(7) , 2002, pp. 971-987. N. Otsu, A threshold selection method from gray-level histogram. IEEE Trans. on Systems, Man and Cybernetics, vol. 9(1) ,1979, pp. 62-66.
