Stokes-vector-based polarimetric imaging system for ... - OSA Publishing

Research Article

Vol. 55, No. 21 / July 20 2016 / Applied Optics

5513

Stokes-vector-based polarimetric imaging system for adaptive target/background contrast enhancement MINJIE WAN, GUOHUA GU,* WEIXIAN QIAN, KAN REN,

AND

QIAN CHEN

Jiangsu Key Laboratory of Spectral Imaging and Intelligent Sense (SIIS), Nanjing University of Science and Technology, Nanjing 210094, China *Corresponding author: [email protected] Received 18 April 2016; revised 16 June 2016; accepted 16 June 2016; posted 17 June 2016 (Doc. ID 263490); published 12 July 2016

A novel method to optimize the polarization state of a polarimetric imaging system is proposed to solve the problem of target/background contrast enhancement in an outdoor environment adaptively. First, the last three elements of the Stokes vector are selected to be the observed object’s polarization features, the discriminant projection of which is regarded as the detecting function of our imaging system. Then, the polarization state of the system, which can be seen as a physical classifier, is calculated by training samples with a support vector machine method. Finally, images processed by the system with the designed optimal polarization state become discriminative output directly. By this means, the target/background contrast is enhanced greatly, which results in a more accurate and convenient target discrimination. Experimental results demonstrate that the effectiveness and discriminative ability of the optimal polarization state are credible and stable. © 2016 Optical Society of America OCIS codes: (110.5405) Polarimetric imaging; (100.5010) Pattern recognition. http://dx.doi.org/10.1364/AO.55.005513

1. INTRODUCTION Military targets can be easily hidden in natural backgrounds by relying on similarities in color or intensity, which makes it difficult to discriminate a camouflaged target. In the past, contrast enhancement techniques in the signal processing domain were widely used for target discrimination. However, most of these methods, like the histogram equalization based method [1–3], the curvelet transform based method [4], the compressed domain based method [5], and the spectral curve fitting based method [6], are mainly based on image properties, rather than the surface properties of the target and the background themselves. Recently, techniques of polarimetric imaging for target/ background contrast enhancement have attracted much attention, and various systems have been set up [7–13], making it possible to reveal contrasts that are not distinct in standard intensity images. For military applications, these kinds of techniques can be used to detect camouflaged targets in a battlefield environment. For biomedical applications, polarimetric imaging is an efficient tool in the identification of cancerous cells and normal cells. Also, pathological diagnosis of skin is a new task associated with it. And, for remote sensing applications, it is employed to enhance image details. Moreover, polarimetric imaging plays an important role in the detection of oil spills on the sea. Since most object surfaces have physical attributes different from the background, current research on polarimetric imaging 1559-128X/16/215513-07 Journal © 2016 Optical Society of America

systems indicates that the polarization states of electromagnetic waves will be changed by the material characteristics of object surfaces during the process of reflection. It is thus reasonable and promising to use polarimetric information as a discriminant for distinguishing reflectors with different materials, conductivities, surface topographies, and so on. The Stokes formalism is widely used to describe polarization states of light according to the Poincaré sphere theory [14], and a coherent method was proposed to describe the full polarization state of light based on this theory [15]. Light can be represented by a four-component vector called a Stokes vector, S I ; Q; U ; V T . I denotes the total intensity of light, and the other quantities represent the amount of linear and circular polarization. Since light can be represented by a Stokes vector uniquely, it is reasonable to use elements of the Stokes vector to distinguish different objects due to surface differences. This paper’s aim is to calculate the optimal discriminative polarization state of a polarimetric imaging system in an outdoor environment by maximizing the Euclidean distance between two Stokes feature margins of the target and the background that cannot be separated in standard visible images directly due to similarities of color or intensity. The main basis of this novel method is that the detecting function of our polarimetric imaging system is the discriminant function in a support vector machine (SVM) classifier. We focus on determining whether a pixel belongs to a target or the background,

5514

Research Article


so that constructing the optimal discriminative polarization state by the method in this paper can be used to obtain the most separable image of target and background.

Common polarimetric imaging systems with active light sources are made up of a light source, a polarization state generator (PSG), a polarization state analyzer (PSA), and a detector. Since our task is conducted in the outdoor environment, which means that the light source is the sun, it is impractical to add a PSG to the system. As a result, the polarization state is only related to the PSA. In addition, the detector used in our designed system is a monochrome CCD. In this section, the theoretical analysis and design of the system are discussed in detail. A. Design of the Physical Classifier

For polarimetric imaging systems with a natural light source, the Stokes vector of the output light detected by an analyzer can be transformed by the Stokes vector of the reflected light multiplied by the Mueller matrix of the PSA: (1)

Since for S out I 0 ; Q 0 ; U 0 ; V 0 T only the first element, I 0 , can be detected by the CCD detector directly, the intensity image captured through the detector is only related to the first row of M PSA , which is shown as Eq. (2): 0I 1 BQ C I 0 V S reflect 1; v 1 ; v 2 ; v 3 @ A; U V

x

Q U V

(2)

where V 1; v 1 ; v 2 ; v 3 is the first row of M PSA and represents the polarization state of the PSA. For our polarimetric imaging system, there are two orthogonal states of polarization state analysis, corresponding to Mueller matrices of the PSA in the parallel and crossed states. The parallel and crossed polarization state analyses can be described by V ∥ 1; v 1 ; v 2 ; v 3 and V ⊥ 1; −v 1 ; −v 2 ; −v3 . Thus, intensity images with respect to the two orthogonal states can be given by I ∥ V ∥ S reflect I v1 Q v 2 U v 3 V : (3) I ⊥ V ⊥ S reflect I − v 1 Q − v2 U − v 3 V I∥ − I⊥ v Q v2 U v3 V : 1 I∥ I⊥ I

(6)

(7)

B. Acquisition of the Optimal Polarization State

In this paper, polarization state is designed as a physical linear classifier. For a binary classification problem, the discriminant function of the linear classifier is expressed as hx wT x b;

N X 1 Ψi ; Jw; w0 ; ψ ‖w‖2 α 2 i1

with the two constraints y i wT x i b ≥ 1 − Ψi ; Ψi ≥ 0

i 1; 2; …; N ;

(4)

(5)

T can be seen as a sum of polarization features multiplied by the polarization state of the PSA. By further observation, we find that T can coincidentally be regarded as a projection onto the polarization state vector, which indicates that the polarization state of the imaging system can be treated as a classifier. Now, we define the feature vector and the projection vector of the classifier as

(8)

where w is a normal vector of the hyperplane, x is the feature vector, and b is the intercept. A target/background contrast enhancement and discrimination problem can be converted to highlight differences between feature vectors of the target and the background as effectively as possible. This is equivalent to obtaining the most discriminative T with a maximum projection distance between the feature vector x and the projection vector wT . The final aim is to find an optimal physical classifier with the maximum discriminative ability. Considering that T can be regarded as the projection of feature vector x to projection vector wT , the SVM classifier satisfies our purpose. It separates two classes accurately with a maximum distance between two margins of training samples, as shown in Fig. 1. In SVM theory [16], if two classes of training samples are marked by a factor y i (for the target, y i 1, while for the background, y i −1), an optimal hyperplane is calculated by minimizing a target function Jw; w0 ; ψ, given in Eqs. (9) and (10):

Furthermore, Eq. (4) can be transformed as T PI v 1 Q v 2 U v 3 V :

:

T wT x

Hence, the degree of polarization (P) [10] can be calculated as P

!

Thus, it is reasonable to select T as a discriminant function with regard to the Mueller matrix of the PSA. The final discriminant function is expressed as

2. THEORETICAL ANALYSIS AND SYSTEM DESIGN

S out M PSA S reflect :

w

! v1 v2 ; v3

Fig. 1. Schematic diagram of SVM classifier.

(9)

(10)

Research Article where wT0 is the optimal projection vector, Ψi is a slack variable for nonseparability of data, α is a constant regulation parameter that controls permitted errors, and N denotes the total number of samples. By calculating the optimal projection vector wT0 , the optimal polarization state of the PSA (V 0 ) can be obtained. Then, the most separable image of the target and the background can be taken with the new polarization state. 3. EXPERIMENTAL RESULTS In this section, we briefly introduce the layout of our polarimetric imaging system and the experimental process of the presented method. Then, two groups of experimental results with different material samples, the target and the background of which are similar in color and intensity, are shown, and comparisons with other polarization states used in existing systems are made to demonstrate the accuracy and efficiency of our method. A. Test Platform and Experimental Process

Figure 2 presents a schematic diagram of the proposed polarimetric imaging system for adaptive target/background contrast enhancement in an outdoor environment. The system is composed of five parts: a source of natural light (the sun), the samples under test, the PSA, a monochrome CCD detector, and a PC. Among them, the PSA is a combination of a λ∕4 wave plate and a linear polarizer, which are used to analyze an unknown Stokes vector of the reflected light. The whole experiment consists of two main steps: training and recognition. For the training step, we presume that the azimuth angles of the linear polarizer and the λ∕4 wave plate are θ and φ, respectively. We first adjust θ and φ to acquire the Stokes vector of the reflected light. Since S out M PSA S reflect and only the first element I out of S out can be detected by the detector, we set six groups of different azimuth angles, which equal six groups of I out and M PSA . Assuming that V is the first row of M PSA , S reflect is calculated as S reflect V −1 I out via the least-squares method. Table 1 lists the six groups of azimuth angles we calculate in this paper. It should be noted that V −1 is a full-rank matrix with the angles in Table 1. Actually, we only need four groups of data to determine S reflect , but the more polarization states used, the more the noise can be reduced. The last three elements Q; U ; V of S reflect are obtained for each pixel in the training frame. We extract two classes of samples, which represent the target and the background from

Fig. 2. Layout of the proposed polarimetric imaging system.


5515

Table 1. Azimuth Angles of the Polarizing Devices Group Number θ φ

1

2

3

4

5

6

0 0

π∕2 0

0 π∕4

π∕4 π∕4

π∕2 π∕4

3π∕4 π∕4

the training frame. Samples of the target are called positive samples, while those of the background are called negative samples. By solving the optimal problem of Eqs. (9) and (10) using SVM theory, an optimal hyperplane is calculated. Then, the corresponding optimal polarization state V 0 of the PSA is obtained via the normal vector w0 of the optimal hyperplane. For the recognition step, the optimal polarization state V 0 of the PSA is employed within the polarimetric imaging system. The final T image is obtained by measuring I ∥ and I ⊥ in the two orthogonal states and the intensity component I of the reflected light. Note that the target/background contrast in this condition is enhanced most greatly via SVM theory. Finally, a simple segmentation method, like the maximum entropy method [17] or Otsu’s method [18], can be utilized to get the discriminant result. The flowchart is shown in Fig. 3.

Fig. 3. Flowchart of the presented method.

Fig. 4. Photographs of two groups of samples: (a) a piece of camouflage cloth, (b) a dark green leaf, (c) a light green painted board, and (d) a light green leaf.

5516


Research Article

B. Results and Comparisons

In our experiment, two groups of samples with different materials are selected to demonstrate the accuracy and effectiveness of the presented method. Figure 4 shows the two groups of samples we use in this experiment. The upper group is a dark green leaf with a camouflage cloth, which is called Group 1, and the lower one is a light green leaf with a green painted board, which is called Group 2. The target and the background have similar color and intensity, but their polarization properties are quite different due to surface differences. Positive and negative samples of Group 1 and Group 2 are extracted. The spatial distributions of the two classes in Q; U ; V feature space and the corresponding optimal hyperplanes calculated by SVM theory are shown in Fig. 5. They clearly indicate that the feature vector Q; U ; V T can represent surface differences between both groups. Thus, optimal hyperplanes calculated via SVM can separate two classes completely. The standard intensity image without any polarizing devices and two other projection vectors taken from existing polarimetric imaging pffiffiffi psystems ffiffiffi pare ffiffiffi selected for further comparison. w1 1∕ 3; 1∕ 3; 1∕ 3 is taken from Ref. [10], p whose ffiffiffi elements pffiffiffi are all the same and positive. w2 2∕3; 2∕ 3; −3∕ 3 is obtained from Ref. [11], whose third element is negative.

Fig. 5. Spatial distributions of positive and negative samples for (a) Group 1 and (c) Group 2; the corresponding optimal hyperplanes for (b) Group 1 and (d) Group 2.

Table 2. V θ° φ°

Polarization States of the Systems under Test

w1 p 1; 1∕ p 3; 1∕p 3; 1∕ 3 40.1322 22.5

w2

w 3 (Group1)

w 3 (Group2)

1; p 2∕3; 2∕ p3; −3∕ 3 90 17.6322

1; −0.8788; −0.2985; −0.3722 91.5449 350.6193

1; −0.8366; 0.5387; −0.0997 76.4711 343.6102

Fig. 6. Result images and the corresponding segmentation maps: (a)–(d) are the result images for the standard intensity image, w1 , w2 , and w3 for Group 1; (e)–(h) are the segmentation results corresponding to (a)–(d); (i)–(l) are the result images for the standard intensity image, w1 , w2 , and w3 for Group 2; (m)–(p) are the segmentation results corresponding to (i)–(l).

Research Article Table 3.


Discriminative Accuracy and Target/Background Contrast of Each Result Image Group 1

ξ C

5517

Group 2

I

w1

w2

w3

I

w1

w2

w3

0.3835 0.1276

0.8149 0.8112

0.8538 0.8189

0.9502 0.9094

0.4526 0.0531

0.7437 0.1293

0.8831 0.2784

0.9416 0.4486

w3 is the projection vector generated by our system. The polarization state of the PSA and the corresponding azimuth angles of linear polarizer (θ) and λ∕4 wave plate (ϕ) are listed in Table 2. In order to show experimental results and demonstrate advantages compared to the standard intensity image without polarizing devices and the other two projection vector techniques, an experiment that tests the two groups of typical samples is conducted. For each group, the experiment tests the samples under the same conditions (including incident angle and emergent angle). For convenient comparison and calculation, an elliptical region of interest is cut out, half of which is the leaf and the other half of which is the background. Intensity images I and T images (all of them are called result images) resulting from the different polarization states of the PSA are given in the left column of Fig. 6. Motivated by the polarization maps shown in Ref. [19], segmentation results with the maximum entropy method are given in the right column of Fig. 6. For Group 1, the red parts are pixels discriminated as leaf, while the green parts are pixels discriminated as background. For Group 2, the yellow parts are discriminated as leaf, while the blue parts are discriminated as background. Two essential conclusions can be summarized: (1) Compared with the standard intensity images, the contrast between the target and the background is enhanced in all three different polarization states, which proves that obtaining surface differences with polarization state is useful and effective. (2) For both groups, the classifying result of w3 is best compared to the other three conditions, which indicates that our SVM-trained polarization state of the PSA can create two classes of materials that are well separable when incident and emergent conditions are fixed. Furthermore, a quantitative analysis of discriminative accuracy ξ is acquired by calculating the number of incorrect classification pixels for leaf and background, as in Eq. (11): IN INB ; (11) ξ1− T TN where TN is the total number of pixels in the elliptical region, and INT and INB are pixel numbers with incorrect classifications in the upper and lower halves, respectively. The target/background contrast C [20] is calculated by the following expression: C

jM T − M B j ; MT MB

is far lower, which means it almost cannot discriminate between target and background in this situation. Also, the target/ background contrast related to w3 is maximal among all four conditions. Compared to the standard intensity image, the target/background contrast of our polarization result is about 10 orders of magnitude higher. Inspired by the contrast analysis given in Ref. [21], Fig. 7 shows gray scale histograms of the result images before segmentation. The left column is for Group 1, and the right one is for Group 2. The more separable the two classes are in the histograms, the higher the contrast is between the target and the background. From our direct observation, the overlapping area of the two curves corresponding to w3 is the smallest, and the

(12)

where M T and M B are the average pixel values of the target and background regions. Table 3 shows the discriminative accuracy and target/ background contrast of each result image in the left column of Fig. 6. For each group, the discriminative accuracy under the optimal polarization state corresponding to w3 is the highest (higher than 90%), while the case without polarizing devices

Fig. 7. Gray scale histograms of the result images: (a)–(d) are histograms of the result images corresponding to the standard intensity image, w1 , w2 , and w3 for Group 1, while (e)–(h) are those for Group 2.

5518

Table 4.

Research Article


Criteria of Classification Ability for Two Groups Group 1

JKE P BE

Group 2

I

w1

w2

w3

I

w1

w2

w3

4.6943 0.5989

22.0554 0.0443

22.0914 0.0327

24.9480 0.0009

3.7043 0.7643

7.9941 0.4012

8.9103 0.1932

11.6611 0.0715

two classes are the most separable visually. Thus, two criteria that reflect classification ability in the pattern recognition domain are being exploited to make a specific study of the data contained in Figs. 6 and 7. The Kullback–Leibler (K-L) distance (J K L ) and Bayesian probability of error (P BE ) introduced in Ref. [9] are exploited. The K-L distance is a measure of the relative entropy between two distributions. In Ref. [9], this is a nonsymmetric measurement of the difference between two probability distributions. In this paper, an adjustment is made to get a symmetric and nonnegative K-L distance, as shown in Eq. (13): Z Z pxjω1 pxjω2 dx pxjω2 ln dx; J K L pxjω1 ln pxjω2 pxjω1 (13) where ω1 denotes the class of target (leaf ), and ω2 denotes the class of background; pxjω1 and pxjω2 are the conditional probability distribution functions (PDFs) of the gray scale frequency for the target and the background, respectively. A higher value of J K L means that the two classes are more separable, i.e., the target/background contrast is enhanced and the target can be better distinguished from the background. Another criterion for the Bayesian probability of error is a classical measurement to reflect the minimal classification error probability for two different classes, which is defined as Eq. (14): Z (14) P BE minpω1 jx; pω2 jxpxdx; where pω1 jx and pω2 jx denote a posteriori probability for class ω1 and class ω2 , respectively, and px is a confidence factor. In contrast to J K L , a lower value of P BE corresponds to a better classification performance. In fact, P BE can be understood as the overlapping area of the histograms in Fig. 7. The two criteria discussed above are calculated using histograms in Fig. 7 for each group. According to Table 4, the K-L distance with respect to SVM-trained projection vector w3 is the largest, and the corresponding Bayesian probability of error is the smallest, which means the best separability can be obtained under the polarization state derived from the projectiontrained vector. In other words, effectiveness and accuracy of our polarization method are best demonstrated using the Bayesian metric. 4. CONCLUSION In this paper, a Stokes-vector-based polarimetric imaging system is designed for adaptive target/background contrast enhancement in an outdoor environment. The whole system is made up of five parts: a source of natural light (the sun), the sample, an analyzer, a detector, and a PC. The innovation

we have introduced is that the detecting function of the imaging system is regarded as a discriminant projection of the observed object’s polarization features (last three elements of the Stokes vector). The polarization state can be treated as a physical classifier obtained by applying this technique to training samples. Images acquired by the system with the desired optimal polarization state are obtained directly. Large numbers of experiments demonstrate that the discriminative ability of the optimal polarization state is stable with respect to discriminative accuracy, contrast, Kullback–Leibler distance, and Bayesian probability of error. In the future, we intend to continue our work by studying the effect of incident and emergent angles on discriminative ability. It should be noted that the optimal polarization of the PSA is not invariable since the polarization property of the target’s surface is quite different under variable directions. Consequently, one needs to calculate the optimal trained polarization state again when the incident and emergent angles change. This indicates to us that incident and emergent angles are factors worthy of further investigation. Funding. National Natural Science Foundation of China (NSFC) (61271332); Natural Science Foundation of Jiangsu Province (BK20130769). REFERENCES 1. J. A. Stark, “Adaptive image contrast enhancement using generalizations of histogram equalization,” IEEE Trans. Image Process. 9, 889–896 (2000). 2. Y. T. Kim, “Contrast enhancement using brightness preserving bi-histogram equalization,” IEEE Trans. Consum. Electron. 43, 1–8 (1997). 3. M. Abdullah-Al-Wadud, M. H. Kabir, M. A. A. Dewan, and O. Chae, “A dynamic histogram equalization for image contrast enhancement,” IEEE Trans. Consum. Electron. 53, 593–600 (2007). 4. J. L. Starck, F. Murtagh, E. J. Candes, and D. L. Donoho, “Gray and color image contrast enhancement by the curvelet transform,” IEEE Trans. Image Process. 12, 706–717 (2003). 5. J. Tang, E. Peli, and S. Acton, “Image enhancement using a contrast measure in the compressed domain,” IEEE Signal Process. Lett. 10, 289–292 (2003). 6. A. J. Brown, “Spectral curve fitting for automatic hyperspectral data analysis,” IEEE Trans. Geosci. Remote Sens. 44, 1601–1608 (2006). 7. J. S. Tyo, M. P. Rowe, E. N. Pugh, and N. Engheta, “Target detection in optically scattering media by polarization-difference imaging,” Appl. Opt. 35, 1855–1870 (1996). 8. S. G. Demos and R. R. Alfano, “Optical polarization imaging,” Appl. Opt. 36, 150–155 (1997). 9. J. S. Tyo, D. L. Goldstein, D. B. Chenalt, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45, 5453–5469 (2006). 10. S. Breugnot and P. Clemenceau, “Modeling and performances of a polarization active imager at λ = 806 nm,” Opt. Eng. 39, 2681–2688 (2000).

Research Article 11. C. S. L. Chun and F. A. Sadjadi, “Polarimetric laser radar target classification,” Opt. Lett. 30, 1806–1808 (2005). 12. P. Terrier, V. Devlaminck, and J. M. Charbois, “Segmentation of rough surfaces using a polarization imaging system,” J. Opt. Soc. Am. A 25, 423–430 (2008). 13. T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proc. IEEE 102, 1450–1469 (2014). 14. W. Wang and G. Li, “The analysis and study of Poincare sphere in Stokes space,” J. Appl. Opt. 3, Ł5–7 (2002). 15. A. J. Brown, “Equivalence relations and symmetries for laboratory, LIDAR, and planetary Müeller matrix scattering geometries,” J. Opt. Soc. Am. A 31, 2789–2794 (2014). 16. C. Cortes and V. Vapnik, “Support vector machine,” Mach. Learn. 20, 273–297 (1995).


5519

17. A. K. C. Wong and P. K. Sahoo, “A gray-level threshold selection method based on maximum entropy principle,” IEEE Trans. Syst. Man. Cybern. 19, 866–871 (1989). 18. N. Otsu, “A threshold selection method from gray-level histograms,” Automatica 11, 23–27 (1975). 19. A. J. Brown and Y. Xie, “Symmetry relations revealed in Mueller matrix hemispherical maps,” J. Quant. Spectrosc. Radiat. Transfer 113, 644– 651 (2012). 20. N. Vannier, F. Goudail, C. Plassart, M. Boffety, P. Feneyrou, L. Leviandier, F. Galland, and N. Bertaux, “Comparison of different active polarimetric imaging modes for target detection in outdoor environment,” Appl. Opt. 55, 2881–2891 (2016). 21. N. Vannier, F. Goudail, C. Plassart, M. Boffety, P. Feneyrou, L. Leviandier, F. Galland, and N. Bertaux, “Active polarimetric imager with near infrared laser illumination for adaptive contrast optimization,” Appl. Opt. 54, 7622–7631 (2015).