IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, SUBMITTED, SEPTEMBER 2016
1
Robust Matrix Decomposition for Image Segmentation under Heavy Noises and Uneven Background Intensities Garret Vo, Student Member, IEEE, Chiwoo Park, Member, IEEE
arXiv:1609.08078v1 [cs.CV] 26 Sep 2016
F
Abstract—This paper presents a robust matrix decomposition approach that automatically segments a binary image to foreground regions and background regions under high observation noise levels and uneven background intensities. The work is motivated by the need of identifying foreground objects in a noisy electron microscopic image, but the method can be applied for a general binary classification problem. The proposed method models an input image as a matrix of image pixel values, and the matrix is represented by a mixture of three component matrices of the same size, background, foreground and noise matrices. We propose a robust matrix decomposition approach to separate the input matrix into the three components through robust singular value decomposition. The proposed approach is more robust to high image noises and uneven background than the existing matrix-based approaches, which is numerically shown using simulated images and five electron microscope images with manually achieved ground truth data. Index Terms—Image De-mixing, Background Subtraction, Image Segmentation, Robust Regression, Regularized Singular Value Decomposition
1
I NTRODUCTION
T
HIS paper is concerned with a robust image segmentation problem that segments an input image into foreground regions and background regions under high observation noise levels and uneven background intensities. This is motivated by the need to analyze heavily noisy microscope images to detect objects of interest, e.g. nanomaterials or biological cells. For imaging small scale objects, electron microscopy uses a high energy electron beam, which is transmitted through a very thin layer of a sample containing interested materials. As the electrons in the beam zoom through the sample, the microscopic structure of materials in the sample can be captured [1], [2]. The electron microscopic image is typically contaminated with high level of noises, and the background intensity of the image is not uniform partially due to the spatial variation of electron beam radiations (like Fig. 1-(a)) and partially due to the transitions between different background materials (like Fig. 1-(b)).
Garret Vo is with the Department of Industrial and Manufacturing, Florida State University, Tallahassee, FL, 32310 USA e-mail:
[email protected]. Chiwoo Park is with the Department of Industrial and Manufacturing, Florida State University, Tallahassee, FL, 32310 USA e-mail:
[email protected].
(a) Example 1: Gradual back- (b) Example 2: Deep shadow ground change with heavy in background with mild noises noises
Fig. 1: Example Electron Microscopic Images A general image segmentation problem has been extensively studied in computer vision. The state-of-the-art methods are categorized into three studies, graph cut [3], [4], active contour [5], [6], [7], and morphological segmentation [8], [9], [10], [11]. The graph cut and active contour formulations are based on either expensive graph problems or nonlinear optimization problems, so some greedy solutions are typically sought. The greedy solutions are very sensitive to image noises, and a quality solution is hard to achieve without good initial solutions. Morphological image segmentation methods, including watershed and its variants, first find markers pointing to the locations of objects and then segments an input image to influence zones of the markers [8], [9], [10], [11]. The biggest challenge in the approaches is how to find good markers under heavy image noises and uneven background intensities. Several noise-robust methods were proposed for finding markers, noise-robust morphological erosion [11], gradient-weighted distance transform [12], and local density clustering [13]. There are many image segmentation methods specialized for detecting circular objects in electron microscope images. Two most recent methods are segmentation of partially overlapping nanoparticles using convex set (SCS) [11], and segmentation of partially overlapping nanoparticles using concave points (SCP) [14]. Both of the two methods come in a two step procedure, finding markers and subsequent segmentation boundary estimation. Many of the aforementioned methods, as global method, do not handle inhomogeneous background intensity issue very effectively. To address the issue, some local approaches
2
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, SUBMITTED, SEPTEMBER 2016
are proposed, including local binary fitting [15], local image fitting [16], the local correntropy-based K-means clustering [17], sliding band filter [18], and iterative voting method (IVM) [19]. On the other hand, the problem of separating foregrounds from uneven backgrounds and image noises has been studied in video-frame data analysis, using different matrix decomposition approaches [20], [21], [22], [23]. The approaches model an input image as a matrix of image pixel values and decompose the matrix into a foreground matrix and a background matrix. The approaches are applied to a video clip, where the foreground matrix contains time-varying image pixels and the background matrix contains time-stationary image pixels. The approaches are very computationally efficient and work well for high noise levels. However, the approaches cannot be directly applied to the image segmentation of a single still image, because they require multiple image frames as inputs to extract the time-stationary background of images. Aware of the advantages of the matrix decomposition approach, we generalize the idea of the matrix decomposition approach for video clips to the image segmentation for a single-frame image that contains high level noises and exhibits uneven background intensities. We assume that the background image exhibits smooth image intensity changes, while foreground objects and image noises generate outliers (intensity jumps) on the smooth background image pattern. For separating the background from those outliers, we propose a robust singular value decomposition approach. To further separate the outliers into foregrounds and noises, we propose a smooth-sparse singular value decomposition, assuming that typical foregrounds more sparsely locate and generate more smooth image intensity patterns than noises. The remainder of this paper is organized as follows. Section 2 presents our matrix decomposition formulation for image segmentation, which is solved by iteratively estimating background and foreground. The background estimation step is described in Section 2.1, and the foreground estimation step is described in Section 2.2. Section 4 presents the numerical performance of the proposed approach with comparison to the four state-of-the-art methods. Section 5 concludes this paper with discussion.
2
M ETHOD
Let Y denote a m × n matrix representing an input image, where the (i, j)th element of the matrix is the image intensity value of the input image at a pixel location (i, j). The input matrix is a mixture of three component matrices of the same size, a background matrix B , a foreground matrix F , and an noise matrix E ,
Y = B + F + E,
(1)
where the noise matrix is random with var(E) = σ 2 I . We assume that the image pixel values in B change smoothly, which implies that two spatially neighboring pixels have similar intensity values instead of having big intensity jumps, while F or E adds big intensity jumps on the background B . We also assume that F is a sparse matrix because foregrounds sparsely locate in a typical image, and the intensity jumps introduced by F are higher than those by E , i.e., the signal-to-noise ratio is at least greater than
(a) Y: input image
(b) B: background
(c) F: foreground
(d) E: noise
Fig. 2: Examples of Foreground, Background and Noise Image Patterns.
one. Figure 2 illustrates the foreground, background and noise examples that follow the assumption. We want to separate Y into B, F and E under these assumptions. For a good model fitness, B + F should have a small bias to Y . The choices of B and F can be optimized by minimizing Minimize ||Y − (B + F )||2F + Pλ (B) + Qµ (F )
(2)
where || · ||F is the Frobenius norm, Pλ (B) is a smoothness penalty on the choice of B and Qµ (F ) is a sparsity penalty on the choice of F ; we will describe the specific forms of the penalty terms later. When the low rank forms are assumed for B and F , the problem of estimating B and F is reduced to a low-rank-and-sparse matrix decomposition problem [24], [25]. The solution is sought by the alternating direction method of multipliers, which basically optimizes B and F iteratively. The major problem of the iterative approach is that estimating B given F is very sensitive to the outlying features contributed by F , because the L2 loss ||Y − (B + F )||2F is very sensitive to outliers; see our numerical example in Fig. 3-(c), 3-(d) and 3-(e). On the other hand, estimating F first is not straightforward with no good estimate of B . In robust statistics [26], the L2 loss is often replaced with the outlier-robust loss function, e.g. the Huber loss function or L1 loss function. Borrowing the idea, we propose a robust matrix decomposition approach to iteratively estimate B and F . The underling idea is summarized as follows. We first separate Y into backgrounds B and the remainders O = F + E, Y = B + O, where we regard B as the main signal to estimate, while O are regarded as outliers added on the main signal; note that
VO AND PARK: ROBUST MATRIX DECOMPOSITION FOR IMAGE SEGMENTATION
as we mentioned, the background intensities are smoothly changing, while the foreground and noises make sudden jumps on the smooth intensity patterns. To minimize the effect of the outliers O on estimating the main signal B , we replace the outliers-sensitive L2 loss function with a robust loss function. The new loss function is basically the weighted L2 loss that poses smaller weights on larger squared errors attributed to outlying features,
||W ◦ (Y − B)||2F , where ◦ is the element-wise matrix multiplication operator, and the weighting matrix W is defined to lower weights on the existence of significant outliers O as r c (W )ij = 1 if |(O)ij | ≤ c and otherwise, |(O)ij | The choice of the weighting matrix is well known as the Huber loss function in the robust statistics [27], and the choice of c is recommended 1.345 to achieve the robustness while maintaining efficiency for the no outliers case. Please note that when |(O)ij | ≤ c (where the outlying features are not significant), the loss is same as the L2 loss,
(W )2ij ((Y )ij − (B)ij )2 = ((Y )ij − (B)ij )2 , but when |(O)ij | > c, the loss is suppressed to the L1 loss, c (W )2ij ((Y )ij − (B)ij )2 = ((Y )ij − (B)ij )2 |(O)ij | (3) = c|(Y )ij − (B)ij |. The loss function makes the estimation of B limitedly affected by the outliers. Since the outlying matrix O is unknown, W is unknown. Therefore, estimating W and B ˆ is given, will be iteratively solved. When the estimated W the problem of estimating B is formulated as
ˆ ◦ (Y − B)||2F + Pλ (B). Minimize ||W
(4)
ˆ , we can estimate O ˆ = Y −B ˆ , and the With the estimated B ˆ can be computed using (3) with O ˆ . Within the second W ˆ can be further decomposed into F step of the iteration, O and E as follows, we solve for F , ˆ − F ||2F + Qµ (F ). Minimize ||O
(5)
In the subsequent subsections, we describe the details of formulations (4) and (5) with the solution approaches. 2.1
Robust Estimation of Background
Consider to approximate the background with a low rank version, p X B= uk v Tk , k=1
where p is the rank, uk is a m×1 unnormalized left singular vector, and v k is a n × 1 right singular vector with a unit norm; the term ‘unnormalized’ implies that the norm of a singular vector is not equal to one. Estimating the p pairs of the singular vectors produces the estimate of B . The estimation of the p pairs can be performed sequentially, following the classical power method for Singular Value Decomposition (SVD) [28]; the first pair of the singular
3
vectors is estimated to best approximate Y , and the next pair is then estimated to best approximate the residual matrix, i.e. Y −u1 v T1 . In general, the k th pair is estimated to Pk−1 approximate the residual Y − l=1 ul v Tl , and estimating each pair can be seen as a rank-one approximation of the residual matrix. In this section, we describes the rank-one approximation of an arbitrary residual matrix R. Consider the rank-one approximation of an arbitrary residual matrix R in the following form,
B 1 = uv T , where u is the m × 1 unnormalized left singular vector, and v is the n × 1 right singular vector with a unit norm. Since we assumed that the background is smooth, we want the rank-one approximation B 1 to be smooth, which implies Assumption 1. The gradients of B 1 along its rows are small, which are quantified by the norm of the gradients of its left singular vector Gu u; Let uj denote the j th element of u and Gu be the gradient operator matrix with the j th element of Gu u as 2uj − uj−1 − uj+1 . In addition, the gradients of B 1 along its columns are small, which are quantified by the norm of the gradients of its right singular vector Gv v ; Gv is the gradient operator matrix. Following [29, Theorem 1], we quantify the total gradients of B 1 by
Pλ (B 1 ) =λuT Ωu u||v||2 + λv T Ωv v||u||2 + λ2 uT Ωu uv T Ωv v,
(6)
where Ωu = GTu Gu and Ωv = GTv Gv . The value of Pλ (B 1 ) decreases as B 1 becomes smoother We solve the following weighted least square problem with smoothness regularization Pλ (B 1 ) to estimate B 1
ˆ ◦ (R − B 1 )||2F + Pλ (B 1 ) Minimize f (u, v) := ||W subject to B 1 = uv T .
(7)
The formulation is equivalent to the robust singular value decomposition proposed by Zhang et. al. [30]. Suggested by the paper, the solution of the optimization is achieved by the following power iteration. To describe the power iteration, we define some notations. The vec(A) denotes the vectorization of a m × n matrix A, which is the mn × 1 column vector obtained by stacking all columns of A. When a is a m × 1 column vector, the diag(a) denotes the m × m diagonal matrix with the ith diagonal element equivalent to the ith element of a. Using the notations, we define
y = vec(R), y ∗ = vec(RT ) ˆ = diag(vec(W ˆ ∗ = diag(vec(W ˆ ◦W ˆ )), W ˆ T ◦W ˆ T )) W U = I n ⊗ u, and V = I m ⊗ v, where ⊗ is the Kronecker product. With the notations, the first order necessary conditions of the problem (7) are
∂f (u, v) ˆ ∗ (Vu − y ∗ ) + 2Ωu|v u = 0 = VT W ∂u ˆ ∗ V + 2Ωu|v )−1 V T W ˆ ∗ y ∗ , and ⇒ u = (V T W ∂f (u, v) ˆ = U T W(Uv − y) + 2Ωv|u v = 0 ∂v ˆ + 2Ωv|u )−1 U T Wy. ˆ ⇒ v = (U T WU
(8)
(9)
4
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, SUBMITTED, SEPTEMBER 2016
where Ωu|v = v T (I + λΩv )v(I + λΩv ) − v T vI and Ωv|u = uT (I + λΩu )u(I + λΩu ) − uT uI . The power iteration updates u using (8) while fixing v = v old and then updates v using (9) while fixing u = uold . Once both of u and v are updated, the weight matrix is updated by
(a) input image
(b) background ground truth
ˆ = R − uv T . W ˆ continue until The iterative updates of the u, v and W ||uv T − uold (v old )T ||F < . (c) background estimated with the L2 loss (λ=1)
2.1.1 Choice of Tuning Parameters The proposed algorithm has a tuning parameter λ that determines the smoothness of the background estimation. We propose using the cross validation approach to choose λ. Consider a range of λ values in Λ. For each λ ∈ Λ, we can compute a cross validation error as follows. We randomly partition the pixel location (i, j) of Y into training set R and testing set T . Fix (W )ij = 0 for (i, j) ∈ T to remove influence of testing pixels on the training stage. The resulting estimates of u and v with λ are denoted by u ˆλ and v ˆλ . The cross validation error of the choice λ is X CV (λ) = ((R)ij − (ˆ uλ v ˆTλ )ij )2 .
(e) background estimated with the L2 loss (λ=100)
(d) background estimated with the L2 loss (λ=10)
(f) background estimated with our robust method
(i,j)∈T
We can choose λ ∈ Λ that minimizes the cross validation error. 2.1.2 Illustration To show the effectiveness of the proposed robust algorithm against the background estimation of the conventional lowrank and sparse matrix decomposition, we designed a simulated study. We simulated a 900×900 image, Y = B+F +E with (B)ij = log((1 + i/9) ∗ (1 + j/9)),
Fig. 3: Comparison of the background estimations with the L2 loss and the robust loss function. For the L2 loss, the smoothness parameter λ varied over {1, 10, 100}.
We represent the foreground as a low-rank form
(F ij ) = − δ((1 + i/9 − 5)2 + (1 + j/9 − 5)2 ≤ 0.25) 2
2
− δ((1 + i/9 − 2) + (1 + j/9 − 2) ≤ 0.25) and
F =
q X
αk β Tk ,
k=1
(E ij ) ∼ N (0, 0.52 ), where δ is the Kronecker delta function, i = 1, ..., 900 and j = 1, ..., 900. We applied the L2-loss-based matrix decomposition approach (2) with the simulated image as inputs, while λ varied over {1, 10, 100}. We also applied our proposed robust background estimation with the cross validation choice of λ. Figure 3 shows the outcomes of the two approaches along with the ground truth B . The approach (2) is significantly affected by foreground features even with very large smoothing parameter values (further increasing λ induces too much bias), while the proposed approach estimates the background very closely to the ground truth.
where q is the rank, αk is a m × 1 unnormalized left singular vector, and β k is a n × 1 right singular vector with a unit norm. Estimating the foreground is achieved by estimating the q pairs of (αk , β k ). Similar to the background estimation, the estimation of the q pairs can be performed by applying the rank-one approximation q times sequentially from the first pair to the end pair, updating the residual to be approximated. The remainder of this section discusses the rank-one approximation. Since the foreground F is smooth and sparse, we define the regularization term Qµ (F ) as a combination of the L2smoothness and L1-sparsity terms,
Qµ (F ) =µ2 αT Ωα α||β||2 + µ2 β T Ωβ β||α||2 2.2
Foreground Estimation
ˆ achieved by Section With the estimate of background B Section 2.1, we can have the estimate of the outliers, ˆ = Y − B. ˆ O We split the outlying features into foregrounds F and noises E by minimizing
ˆ − F ||2F + Qµ (F ). ||O
(10)
+ µ22 αT Ωα αβ T Ωβ β + µ1 ||α||1 ||β||1 . where the first three terms correspond to the L2-smoothness in the same form as Pλ (·), and the L1-sparsity poses the sparsity of F . This double penalization can be found in the elastic net [31]. It is well known that the elastic net produces sparsity and grouping effect, which implies that the estimated values of the highly correlated variables have small differences [31]. This is desirable because each of
VO AND PARK: ROBUST MATRIX DECOMPOSITION FOR IMAGE SEGMENTATION
two groups of pixels, foreground and noises, should have small within-group variations in intensities. The α and β are iteratively estimated using the power iteration. Let o = vec(O), o∗ = vec(O T ), A = I n ⊗ α and B = I m ⊗ β . When β is fixed, the problem (10) becomes an elastic net regression problem for α, Minimize ||o∗ − Bα||22 + αT Ωα|β α + µα|β ||α||1 ,
(11)
where Ωα|β = (µ2 ||β||2 + µ22 β T Ωβ β)Ωα + µ2 β T Ωβ βI m , and µα|β = µ1 ||β||1 . The elastic problem can be converted into a lasso regression problem, which can be effectively solved by a lasso solver [32]. On the other hand, when α is fixed, the problem (10) becomes another elastic regression problem for β , Minimize ||o −
Aβ||22
T
+ β Ωβ|α β + µβ|α ||β||1 ,
(12)
5
that the average intensity of the foreground is one, and the noise variance of (0.9)2 is comparable to the average intensity. We applied our approach to Y to achieve the ˆ and foreground estimation Fˆ . background estimation B The estimated values were compared to the ground truth to get the mean square errors. Fig. 4 shows the estimated foregrounds and backgrounds for the three different noise cases. Table 1 summarizes the estimation accuracy in terms of the mean square errors. We can see that the performance of our approach does not significantly vary depending on different noise levels.
4
A PPLICATIONS TO R EAL M ICROSCOPIC I MAGES
This section shows how our proposed method works with real microscopic images, and compare its performance with where Ωβ|α = (µ2 ||α||2 + µ22 αT Ωα α)Ωβ + µ2 αT Ωα αI n , two state-of-the-art methods in cell and nanoparticles segand µβ|α = µ1 ||α||1 . We iteratively solve (11) and (12) to mentation: segmentation of partially overlapping nanoparoptimize α and β until convergence is reached. ticles using convex set [11, SCS] and segmentation of parAs a more computationally efficient solution approach, tially overlapping nanoparticles using concave points [14, the naive elastic-net type solution can be pursued as sug- SCP]. In addition, we also compare our proposed method’s gested in the literature [32, Section 2.2]. It basically gets the performance against two state-of-the-art methods in low ridge regression solution for an elastic net problem while rank and sparse matrix decomposition: Randomized lowfixing the L1 penalty parameter zero, which is followed by rank and sparse matrix decomposition [24, GoDec], and applying the soft-thresholding to the ridge output. When simultaneous of low-rank and sparse structure in matrix µ1 = 0, the original problem (10) becomes [25, Simul]. For the numerical comparison, we implemented T 2 T T 2 2 Simul [25] by ourselves. We used the implementation made ˆ Minimize ||O − αβ ||F + µ2 α Ωα α||β|| + µ2 β Ωβ β||α|| by the corresponding authors for SCS [11], SCP [14], and + µ22 αT Ωα αβ T Ωβ β. GoDec [24]. We chose five different electron microscopic images of The solution can be achieved by the same procedure as we nanoparticles for evaluation and comparison purposes. The used for the background estimation in Section 2.2 with W = ˆ images chosen are different in their background patterns, 1 fixed. Let α ˆ and β denote the solutions. The solutions can density of foreground objects and noise levels. Figure 5 be thresholded as shows the five images. The first image has a deep slope ˆ = (β ˆ − τβ )+ sgn(β), ˆ ˆ 1,2 = (α α ˆ − τα )+ sgn(α), ˆ and β 1,2 in background, and the other four images have gradual (13) changes in background. The second and third images have where z+ denotes the positive part of z , τα and τβ are the low signal-to-low ratio, and the fourth and fifth images have thresholds, and sgn(z) denotes the sign of z . The proposed foreground objects locate very densely. algorithm has two tuning parameters, smoothness parameter µ2 and the two threshold parameters, τα and τβ . The 4.1 Comparison to the Matrix Decomposition Apchoice of the smoothness parameter can be made by the proaches cross validation in the same manner as the background estimation step. For the choice of the sparsity parameters, In this section, we compare the foreground and background we propose to use the generalized model description length estimations of our approach to those of the two existing [33, gMDL], which has shown better performance than the matrix decomposition approaches, GoDec [24], and Simul [25]. Figs. 6 through 10 show the estimation results from AIC and the BIC. the three compared method for the five test images. Our approach outperforms the GoDec and the Simul for all of 3 S IMULATED S TUDY the five test images. The background estimation of the GoDec and Simul are This section shows how our proposed method works for simulated images. For the simulation study, we simulated significantly affected by the existence of foreground objects, and the effects increase as the density and sizes of fore900 × 900 images, Y = B + F + E with ground objects increase. For example, the effects are quite (B)ij = log((1 + i/9) ∗ (1 + j/9)), mild for the first three test images but very significant for the (F ij ) = − δ((1 + i/9 − 5)2 + (1 + j/9 − 5)2 ≤ 0.25) last two test images that have densely located foreground objects of large sizes. Our approach’s background estimates − δ((1 + i/9 − 2)2 + (1 + j/9 − 2)2 ≤ 0.25) and are quite robust to the existence of foreground objects. This 2 (E ij ) ∼ N (0, σ ), is because our approach is the robust loss function in the where σ 2 is chosen (0.3)2 for a low noise case, (0.6)2 for a background estimation as we discussed in Section 2.1, while medium noise case, and (0.9)2 for a high noise case. Note the other competing methods use the L2 loss function.
6
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, SUBMITTED, SEPTEMBER 2016
(i) original
(ii) background
(i) original
(ii) background
(iii) foreground
(iv) noise
(iii) foreground
(iv) noise
(a) Low Noise Case
(b) Medium Noise Case
(i) original
(ii) background
(iii) foreground
(iv) noise
(c) High Noise Case
Fig. 4: The result of our approach for simulated cases TABLE 1: Mean Square Errors (MSE) of the Background and Foreground Estimation for the Simulation Study Noise Level Background MSE Foreground MSE Low 0.0086 0.0009 Medium 0.0082 0.0014 High 0.0080 0.0015
VO AND PARK: ROBUST MATRIX DECOMPOSITION FOR IMAGE SEGMENTATION
(a) image 1
7
(b) image 2
(d) image 4
(c) image 3
(e) image 5
Fig. 5: Electron Microscopic Images of Nanoparticles. Our method Simul [25] GoDec [24] FPR FNR FPR FNR FPR FNR image 1 0.0151 0.0633 0.1776 0.1397 0.0847 0.1750 image 2 0.0435 0.1870 0.2176 0.3166 0.0550 0.5361 image 3 0.0450 0.2761 0.2313 0.4217 0.0573 0.6706 image 4 0.0039 0.2224 0.0396 0.1094 0.0101 0.3996 image 5 0.0083 0.0580 0.0448 0.4857 0.7494 0.0672 TABLE 2: Comparison of Performance on Low-Rank and Sparse Matrix Decomposition. The foreground estimation of the GoDec and Simul are quite noisy (containing many tiny blobs spread over images), while the foreground estimation of our approach is much less noisy. The difference between the existing approaches and our approach gets much more significant as the noise level increases. For image 2 and 3, the foreground estimations of the two existing approaches contain many salt-and-pepper types of noises, while our approach does not contain those. Our approach doubly penalizes the foreground estimation with the L1 and L2 penalties, while the two existing approaches use the L1 thresholding. The L2 penalization added on our formulation provides additional filtering and smoothing noises. Table 2 provides the quantitative comparison of the foreground estimation accuracy. For the comparison, we manually created the ground-truth foreground image for each of the test images as shown in the first column of Fig. 11. Based on the ground-truth data, we computed the
true positive (TP) by counting the total number of pixels belong to both the ground-truth objects and the estimated foregrounds, and the false negative (FN) by counting the number of pixels belong to the ground-truth but not the estimated foregrounds. Similarly, we computed the false positive (FP) by counting the number of pixels belong to the estimated foregrounds but not in the ground-truth, and the true negative (TN) by counting the number of pixels not belong to the ground-truth and the estimated foregrounds. Based on the four numbers, we computed the false positive rate (FPR) and false negative rate (FNR) for each method based on the following equations,
FP , and FP + TN FN FNR = . FN + TP
FPR =
8
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, SUBMITTED, SEPTEMBER 2016
(a) original
(b) background (ours)
(e) background (GoDec)
(h) background (Simul)
(c) foreground (ours)
(f) foreground (GoDec)
(i) foreground (Simul)
(d) noise (ours)
(g) noise (GoDec)
(j) noise (Simul)
Fig. 6: Comparison for image 1 (a) original
(b) background (ours)
(e) background (GoDec)
(h) background (Simul)
(c) foreground (ours)
(f) foreground (GoDec)
(i) foreground (Simul)
(d) noise (ours)
(g) noise (GoDec)
(j) noise (Simul)
Fig. 7: Comparison for image 2
VO AND PARK: ROBUST MATRIX DECOMPOSITION FOR IMAGE SEGMENTATION
(a) original
9
(b) background (ours)
(e) background (GoDec)
(h) background (Simul)
(c) foreground (ours)
(f) foreground (GoDec)
(i) foreground (Simul)
(d) noise (ours)
(g) noise (GoDec)
(j) noise (Simul)
Fig. 8: Comparison for image 3 (a) original
(b) background (ours)
(e) background (GoDec)
(h) background (Simul)
(c) foreground (ours)
(f) foreground (GoDec)
(i) foreground (Simul)
(d) noise (ours)
(g) noise (GoDec)
(j) noise (Simul)
Fig. 9: Comparison for image 4
10
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, SUBMITTED, SEPTEMBER 2016
(a) original
(b) background (ours)
(e) background (GoDec)
(h) background (Simul)
(c) foreground (ours)
(f) foreground (GoDec)
(i) foreground (Simul)
(d) noise (ours)
(g) noise (GoDec)
(j) noise (Simul)
Fig. 10: Comparison for image 5 4.2
Comparison to the Particle Segmentation Methods
The foreground estimation results from our method and the two state-of-the-arts in bio-cell and particle segmentation are presented in Fig. 11. In the figure, the first column contains the groundtruth foreground boundaries overlaid over the five test images, while the other columns contain the boundaries of the foreground detections of the compared methods overlaid over the five test images. The particle segmentation methods did not work well when the background contain deep shading such as the first test image, missing most of the particles in the shaded area. The particle segmentation methods also did not work with the third test image for which the signal-to-noise ratio is pretty low. Our proposed approach worked well for all of the test images. This is well reflected in our quantitative analysis of the foreground estimation results. Table 3 summarizes the FPR and FNR performance of the compared method. For image 1, image 3 and image 5, the proposed approach outperformed the existing approaches significantly while it was comparable to the existing ones for the other two images.
5
C ONCLUSION
We presented a new approach that solves an image segmentation problem when image noises are severe and
background is not uniform. The approach is motivated by the great success of the background subtraction approach in video frame data analysis. The background subtraction approach models an input image matrix as a mixture of additive effects, foreground and background, and applies the de-mixing approach to decompose the mixture into individual components of foreground and background. In the de-mixing, the background is typically regarded as timenonstationary, so the de-mixing procedure uses multiple image frames to find non-stationary components of images, which provides the background estimation. However, the same approach cannot be used for image segmentation, because only a still image is available as an input for image segmentation. The new approach we proposed in this paper generalizes the de-mixing idea for image segmentation when the image background show simple patterns, containing only intensity gradation. The approach iteratively estimates the background and the foreground, filtering out noises. For the background estimation, the robust singular value decomposition problem was formulated, and the power iterations-type solution approach was proposed. For the foreground estimation, the regularized singular value decomposition with the elastic-net regularization was proposed to minimize the corruption of the foreground estimation with heavy noises. The proposed approach is expected useful for a general image segmentation problem when an input image has
VO AND PARK: ROBUST MATRIX DECOMPOSITION FOR IMAGE SEGMENTATION
11
(a) Groundtruth
(b) Our method
(c) SCS [11]
(d) SCP [14]
(e) Groundtruth
(f) Our method
(g) SCS [11]
(h) SCP [14]
(i) Groundtruth
(j) Our method
(k) SCS [11]
(l) SCP [14]
(m) Groundtruth
(n) Our method
(o) SCS [11]
(p) SCP [14]
(q) Groundtruth
(r) Our method
(s) SCS [11]
(t) SCP [14]
Fig. 11: Foreground boundaries overlaid on the five test images, groundtruth, SBF [18], and SCS [11], from the left to the right. TABLE 3: Comparison to the Particle Segmentation Methods Our method SCS [11] SCP [14] FPR FNR FPR FNR FPR FNR image 1 0.0151 0.0633 0.0045 0.4931 0.0512 0.5900 image 2 0.0435 0.1870 0.0673 0.1677 0.0336 0.2984 image 3 0.0450 0.2761 0.0312 0.5050 0.0357 0.5050 image 4 0.0039 0.2224 0.0036 0.1858 0.0015 0.3127 image 5 0.0083 0.0580 0.0241 0.1386 0.0276 0.1576
12
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, SUBMITTED, SEPTEMBER 2016
uneven background and heavy image noises. The applicable examples include the analysis of noisy electron microscope images for identifying objects of interest. The proposed approach has shown great performance for simulated images and noisy electron microscope images containing particles, which outperformed the state-of-the-art particle image segmentation approaches and other matrix decomposition approaches.
ACKNOWLEDGMENT The authors would like to acknowledge support for this project. This work is partially supported by NSF 1334012, AFOSR FA9550-13-1-0075, AFOSR FA9550-16-1-0110, and FSU PG 036656.
R EFERENCES [1]
[2] [3] [4] [5] [6] [7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
H. S. Kushwaha, S. Tanwar, K. Rathore, and S. Srivastava, “Denoising filters for TEM (transmission electron microscopy) image of nanomaterials,” in 2012 Second International Conference on Advanced Computing & Communication Technologies. IEEE, 2012, pp. 276–281. L. Reimer, Transmission Electron Microscopy: Physics of Image Formation and Microanalysis. Springer, 2013, vol. 36. J. Shi and J. Malik, “Normalized cuts and image segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888–905, 2000. P. F. Felzenszwalb and D. P. Huttenlocher, “Efficient graph-based image segmentation,” International Journal of Computer Vision, vol. 59, no. 2, pp. 167–181, 2004. T. F. Chan and L. A. Vese, “Active contours without edges,” IEEE Transactions on Image Processing, vol. 10, no. 2, pp. 266–277, 2001. W. Yu, F. Franchetti, Y.-J. Chang, and T. Chen, “Fast and robust active contours for image segmentation,” in 2010 IEEE International Conference on Image Processing. IEEE, 2010, pp. 641–644. S. Niu, Q. Chen, L. de Sisternes, Z. Ji, Z. Zhou, and D. L. Rubin, “Robust noise region-based active contour model via local similarity factor for image segmentation,” Pattern Recognition, vol. 61, pp. 104–119, 2017. N. Malpica, C. Ortiz de Solorzano, J. J. Vaquero, A. Santos, I. Vallcorba, J. M. Garcia-Sagredo, and F. d. Pozo, “Applying watershed algorithms to the segmentation of clustered nuclei,” 1997. F. B. Tek, A. G. Dempster, and I. Kale, “Blood cell segmentation using minimum area watershed and circle radon transformations,” in Mathematical Morphology: 40 years on. Springer, 2005, pp. 441– 454. P. U. Adiga and B. Chaudhuri, “An efficient method based on watershed and rule-based merging for segmentation of 3-D histopathological images,” Pattern Recognition, vol. 34, no. 7, pp. 1449– 1458, 2001. C. Park, J. Z. Huang, J. X. Ji, and Y. Ding, “Segmentation, inference and classification of partially overlapping nanoparticles,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 3, pp. 669–681, 2013. G. Lin, U. Adiga, K. Olson, J. F. Guzowski, C. A. Barnes, and B. Roysam, “A hybrid 3d watershed algorithm incorporating gradient cues and object models for automatic segmentation of nuclei in confocal image stacks,” Cytometry Part A, vol. 56, no. 1, pp. 23–36, 2003. H. Yang and N. Ahuja, “Automatic segmentation of granular objects in images: Combining local density clustering and gradientbarrier watershed,” Pattern Recognition, vol. 47, no. 6, pp. 2266– 2279, 2014. S. Zafari, T. Eerola, J. Sampo, H. K¨alvi¨ainen, and H. Haario, “Segmentation of partially overlapping nanoparticles using concave points,” in International Symposium on Visual Computing. Springer, 2015, pp. 187–197. C. Li, C.-Y. Kao, J. C. Gore, and Z. Ding, “Minimization of regionscalable fitting energy for image segmentation,” IEEE Transactions on Image Processing, vol. 17, no. 10, pp. 1940–1949, 2008.
[16] K. Zhang, H. Song, and L. Zhang, “Active contours driven by local image fitting energy,” Pattern Recognition, vol. 43, no. 4, pp. 1199–1206, 2010. [17] L. Wang and C. Pan, “Robust level set image segmentation via a local correntropy-based K-means clustering,” Pattern Recognition, vol. 47, no. 5, pp. 1917–1925, 2014. [18] P. Quelhas, M. Marcuzzo, A. M. Mendonc¸a, and A. Campilho, “Cell nuclei and cytoplasm joint segmentation using the sliding band filter,” IEEE Transactions on Medical Imaging, vol. 29, no. 8, pp. 1463–1473, 2010. [19] B. Parvin, Q. Yang, J. Han, H. Chang, B. Rydberg, and M. H. Barcellos-Hoff, “Iterative voting for inference of structural saliency and characterization of subcellular events,” IEEE Transactions on Image Processing, vol. 16, no. 3, pp. 615–623, 2007. [20] Y. Zhang, B. Du, L. Zhang, and S. Wang, “A low-rank and sparse matrix decomposition-based mahalanobis distance method for hyperspectral anomaly detection,” IEEE Transactions on Geoscience and Remote Sensing, vol. 54, no. 3, pp. 1376–1389, 2016. [21] H. Ji, S. Huang, Z. Shen, and Y. Xu, “Robust video restoration by joint sparse and low rank matrix approximation,” SIAM Journal on Imaging Sciences, vol. 4, no. 4, pp. 1122–1142, 2011. [22] R. Otazo, E. Candes, and D. K. Sodickson, “Low-rank plus sparse matrix decomposition for accelerated dynamic MRI with separation of background and dynamic components,” Magnetic Resonance in Medicine, vol. 73, no. 3, pp. 1125–1136, 2015. [23] X. Cui, J. Huang, S. Zhang, and D. N. Metaxas, “Background subtraction using low rank and group sparsity constraints,” in Computer Vision–ECCV 2012. Springer, 2012, pp. 612–625. [24] T. Zhou and D. Tao, “Godec: Randomized low-rank & sparse matrix decomposition in noisy case,” in Proceedings of the 28th International Conference on Machine Learning (ICML-11), 2011, pp. 33–40. [25] Q. Yan, J. Ye, and X. Shen, “Simultaneous pursuit of sparseness and rank structures for matrix decomposition,” Journal of Machine Learning Research, vol. 16, pp. 47–75, 2015. [26] P. J. Rousseeuw and A. M. Leroy, Robust Regression and Outlier Detection. John Wiley & Sons, 2005, vol. 589. [27] P. J. Huber, Robust Statistics. Springer, 2011. [28] G. H. Golub and C. F. Van Loan, Matrix Computations. Johns Hopkins University Press, 2012, vol. 3. [29] J. Z. Huang, H. Shen, A. Buja et al., “Functional principal components analysis via penalized rank one approximation,” Electronic Journal of Statistics, vol. 2, pp. 678–695, 2008. [30] L. Zhang, H. Shen, and J. Z. Huang, “Robust regularized singular value decomposition with application to mortality data,” The Annals of Applied Statistics, vol. 7, no. 3, pp. 1540–1561, 2013. [31] D.-X. Zhou, “On grouping effect of elastic net,” Statistics & Probability Letters, vol. 83, no. 9, pp. 2108–2112, 2013. [32] H. Zou and T. Hastie, “Regularization and variable selection via the elastic net,” Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 67, no. 2, pp. 301–320, 2005. [33] M. H. Hansen and B. Yu, “Minimum description length model selection criteria for generalized linear models,” Lecture NotesMonograph Series, pp. 145–163, 2003.
Garret Vo (St.M’ 14) received his B.S. in physics and M.S. in mechanical engineering at Montana State University. He is currently pursuing his Ph.D. degree in the Department of Industrial and Manufacturing Engineering at Florida State University. His research interests are computer vision, text mining, and high dimensional data analysis. He is a student member of IEEE.
VO AND PARK: ROBUST MATRIX DECOMPOSITION FOR IMAGE SEGMENTATION
Chiwoo Park (St. M’ 08, M’12) received his B.S. in industrial engineering at Seoul National University and Ph.D. degree in industrial engineering at Texas A&M University in 2011. He is currently Assistant Professor in the Department of Industrial and Manufacturing Engineering at Florida State University and a principal investigator at High Performance Materials Institute. His research includes data analytics for science and engineering problems, especially image and functional data analytics in nanoscience and manufacturing engineering. His work is being supported by the National Science Foundation and the Air Force Office of Scientific Research. He received the best student paper award at IEEE Conferences on Automation Science and Engineering in 2008, the Ralph E. Powe Junior Faculty Award from the Oak Ridge Associated Universities in 2013, and the IIE best application paper in 2014. He is an active member of IIE, IEEE and INFORMS.
13
