Comparison of Methods for Splitting of Touching and ... - Springer Link

Comparison of Methods for Splitting of Touching and Overlapping Macrophages in Fluorescent Micrographs Christian Held1 , Jens Wenzel2 , Roland Lang2 , Ralf Palmisano3, and Thomas Wittenberg1

3

1 Fraunhofer-Institute for Integrated Circuits IIS, Department for Biomedical Image Processing, Erlangen, Germany 2 University of Erlangen-Nuremberg, Institute for Immunology and Hygiene, Erlangen, Germany University of Erlangen-Nuremberg, Department of Biology, Erlangen, Germany

Abstract. The segmentation of complex touching and overlapping cells in fluorescent micrographs poses a challenge for automated image analysis systems. In order to improve performance for complex image data, multi-channel approaches exist that additionally incorporate information from the cell nuclei. The most frequently method used for fluorescent micrograph segmentation is the seeded watershed transform. But methods based on level sets and graph cuts can also be used as alternatives to the watershed transform based splitting of cells. In this work we investigate if segmentation results obtained by one of the named methods are superior in terms of segmentation performance. Therefore, a hybrid watershed transform based method, a very efficient fast marching cell splitting method and a cell splitting method based on graph cuts are described and investigated. For performance comparison the parameters of each method are automatically optimized toward a manual ground truth including cross validation techniques. Our evaluations show that for the presented dataset of bone marrow-derived macrophages the hybrid watershed transform based method can compete with both, the fast marching level set and the graph cut based method. Keywords: Image processing, segmentation, fluorescence microscopy, watershed, fast marching, graph cut, macrophages.

1

Introduction

Experiments in microbiology and virology often require the evaluation of a huge number of fluorescent micrographs. Manual assessment of such immense image data is prone to inter as well as intra observer errors. Hence, automated image analysis methods are required to assist the human expert. For the automated analysis of fluorescent micrographs the segmentation of fluorescently strained cells usually is the most challenging part. Especially if cells show varying intensity patterns and complex morphology as e.g. macrophages. A. Campilho and M. Kamel (Eds.): ICIAR 2012, Part II, LNCS 7325, pp. 456–464, 2012. c Springer-Verlag Berlin Heidelberg 2012

Splitting of Fluorescent Micrographs

457

For such complex micrographs a robust segmentation usually is only possible if the depicted cells rarely touch and do not overlap each other. If clumped cells or strong overlaps have to be considered, automatically computed segmentation results tend to perform an erroneous splitting of the cells. For human examiners manual evaluation of such datasets is also prone to errors as some of the overlaps cannot be dissolved without ambiguities. In order to assist the human observers with the analysis of such complex cell structures an additional nuclei staining can be applied. Using information on the cell nuclei position and extension many of the ambiguities can be dissolved and most of the errors performed by manual segmentation are prevented. This procedure can be imitated by automated image segmentation methods. Hence, an automatic segmentation of the cell nuclei is performed in a first step. Information on position and extension of the nuclei is then used to improve segmentation of the cells.

2

State of the Art

Lindblad et al. [1] propose such a multi-channel segmentation scheme for the segmentation of Chinese hamster Ovary (CHO) cells. After segmentation of the cell nuclei this information is used for segmentation of the CHO cells by using a seeded watershed transform based on intensity information. Bengtsson et al. [2] also describe a seeded watershed transform based on intensity for the robust segmentation of cells including information on the cell nuclei. As an alternative to the watershed transform the level set method has also been applied to the multi-channel segmentation of fluorescent nuclei. An example for a level set based scheme that uses information on the cell nuclei for segmentation of the cells is provided by Yan et al. [3]. Their work shows that using independent level sets no good segmentation of the cells is obtained. In order to enable segmentation using level sets they include an interaction model in the energy functional. A further level set based method is described in Yu et al. [5], who include topology preserving constraints to prevent the level set method from splitting or merging cells. In Yu et al. [4], these constraints are replaced by an interaction model based on generalized Voronoi diagram. In previous work the fast marching level set based cell splitting method showed good results [9].

3

Materials and Methods

As methods based on level sets as well as graph cuts and the watershed transform have been proposed in literature to the problem of multi-channel segmentation in fluorescent micrographs a comparison of these methods is performed. For comparison of the three methods each method uses an identical preprocessing and threshold selection method based on Difference of Gaussian filters and kmeans clustering. Then cells are split using one of the splitting methods is applied to obtain a final segmentation of the cells.

458

3.1

C. Held et al.

Materials

To carry out the described experiments, bone marrow-derived macrophages from C57BL/6 mice were stimulated with LPS. For visualization of the cell surface a CD11b/AP C staining was applied. In order to provide information on the cell nuclei an additional DAPI staining is used. Figure 1 shows an example for a typical representative of this dataset. In total 20 images with a resolution of 1388 × 1040 pixels were captured using a Zeiss Axiovert microscope with a 20× objective. In order to obtain a high quality ground truth, 558 cells were manually annotated by an experienced biologist. For assessment of the intra observer error a subset of 235 cells was manually delineated for a second time by the same expert user.

(a)

(b)

(c)

(d)

Fig. 1. Example of CD11b/AP C stained macrophages (a). For assessment of the cell nuclei a DAPI staining is used (b). Manually obtained ground truth (c). Figure (d) shows an overlay of Figures (a) and (b).

3.2

Preprocessing and Threshold Selection

For a fair comparison of all 3 methods, identical preprocessing and threshold selection methods are applied for each cell splitting method. For preprocessing a Difference of Gaussian (DoG) filter is applied for both: smoothing of the image and removal of background and illumination artifacts. The input image is convolved with two different Gaussian smoothing filters that are computed using standard deviations σnoise and σbg . The DoG filtered image is obtained by subtracting the two resulting images. As result noise pixels are eliminated and low frequency illumination artifacts are removed. Next a k-means clustering is used to obtain a threshold for separation of macrophages from the background. Using this threshold selection method a clustering of the image intensities is performed. Pixels belonging to the darkest cluster are considered as background, remaining pixels as foreground. This method is used as the threshold level can easily be adjusted by changing the number of clusters k. 3.3

Cell Splitting Using the Seeded Hybrid Watershed Transform

The seeded watershed transform (SWT) is the most commonly used method for splitting of fluorescently stained cells. The obtained segmentation performance


459

depends on the type and quality of input data the SWT is applied to. For the macrophage dataset the SWT cannot be applied directly to the CD11b/AP C stained images because cells have to be split at location of minimum or maximum intensities. In literature the distance transform is most commonly applied to the distance transformed binary images obtained after thresholding. Alternatively, the gradient magnitude image can be used. A method combining both approaches has been proposed by Malpica et al. [6] and is known as hybrid watershed transf orm. Denoting the gradient magnitude image as ∇I(x, y) and the distance transformed binary image as ID (x, y) the input of the hybrid watershed transform IH (x, y) can be obtained by: IH (x, y) = (1 − α)ID (x, y) + α∇I(x, y),

(1)

where α ∈ [0, 1] is a weighting factor allowing balancing between the impact of the gradient magnitude based and the distance transform based components. In order to reduce over segmentation we use a Gaussian smoothing filter with σws for smoothing of IH (x, y). This smoothing technique allows reduction of over segmentation artifacts. 3.4

Cell Splitting Using Fast Marching Level Sets

The fast marching level set method (FM) [7] can be considered as an alternative to the SWT. We use the FM level sets because of its fast runtime and its topology preserving characteristics. Performance of the level sets depends on choice of the speed function F (x, y). In order to prevent the FM method from running into the image background, information on the binary image IB (x, y) is used, with IB (x, y) = 0 if (x, y) is a background pixel and IB (x, y) = 1 otherwise. In order to increase the probability that touching cells are split at location of strong gradients, F (x, y) depends on the gradient magnitude ∇I(x, y) of the fluorescently stained image. Implementation of the ∇ operator is performed using a differential of Gaussian filter with standard deviation σls . The weight of the gradient magnitude is adjustable by an exponential weighting factor αls . As the FM method tends to leaking, a curvature term κ(x, y) is incorporated as proposed by Nilsson et al. [8] with a slight modification [9]. Modification is required because in our application the image contains multiple fronts starting at different cell nuclei. That is why the motion of each front is tracked in an additional label image IL (x, y). For estimation of the curvature κ(x, y), N equally spaced points Δ(x, y)i , i = 1, ..., N located on a discrete circle with radius rls are analyzed. The curvature is then estimated by counting the number of points that share the label of the current pixel: κ(x, y) =

1 1 #{i : IL ((x, y) = IL ((x, y) + Δ(x, y)i )} − . N 2

(2)

460

C. Held et al.

In order to in- or decrease the effect of the curvature term, a weighting factor 0 ≤ λls ≤ 2 is included for definition of the speed function: F (x, y) =

3.5

1 + λls κ(x, y) IB (x, y). (∇I(x, y))αls

(3)

Cell Splitting Using Graph Cuts

In contrast to the FM method computing a local optimum for the splitting of the cells, the graph cut algorithm (GC) is able to globally optimize an energy function [13,14]. That is why the GC is considered as an alternative to both the FM and the SWT method. For splitting of the cells we use a multilabel graph cut as each cell nucleus is represented by a different label. The multilabel GC minimizes an energy function based on a smoothness energy and a data term: E(A) = Edata + Esmooth .

(4)

For definition of the smoothness term Esmooth , cells are split at the location of the strongest gradient magnitude. Introducing an exponential weighting factor αgc determining weights of the gradient magnitude, the smoothness energy term can be set to ∇I(x, y)αgc . Implementation of the ∇ operator is performed using a differential of Gaussian filter with standard deviation σgc . One of the major problems concerning GCs is the shrinking bias [13] causing that smaller contours are preferred to larger contours by the GC energy function. As a result initialization of each cell with the contour of its nucleus may lead to erroneous segmentations. In order to reduce this bias an additional distance based weight map ID (x, y) is included. Assuming that a binary segmentation, partitioning the image into fore- and background is available from the previous object segmentation stage and denoted as IB (x, y), the distance based weight map can be computed by calculating the distance of each pixel to the closest nucleus. The distance of pixels assigned to the image background as well as the distance inside the cell nuclei is set to 0. This image is then inverted and normalized to [1,q ] the range [1, qgc3 ] and denoted as ID gc (x, y) (see Figure 2). Assuming that Ap and Aq denote the label of pixels p and q the GC smoothness energy term is obtained: [1,qgc ]

Esmooth = (1 − ID

(x, y))(1 − ∇I(x, y))αgc δ(Ap , Aq ),

(5)

where δ(Ap , Aq ) is 0 if Ap = Aq and 1 otherwise. For the data based energy term, information of the image background is combined with information of the cell nuclei. Assuming that each cell contains only one nucleus, an object containing N nuclei is split into N cells. By assigning a unique label to each nucleus and an additional label to the image background N + 1 labels are required for the data term. For pixels identified as background seeds the regional term is set to ∞ for all labels except the background label. For pixels representing a nucleus the regional term is set to ∞ for all labels except its unique label. For pixels that are not part of the background and do


(a)

(b)

(c)

461

(d)

Fig. 2. Illustration of the GC cost function. Touching cells (a) are split using information on the cell nuclei with a GC method. The energy function consists of a gradient magnitude (b) and a distance transform based term (c). The segmentation result obtained is shown in (d).

not represent a nucleus the label regional term is set to ∞ for the background label only. All remaining combinations of target label and label assigned by the graph cut are plausible. Hence, the corresponding regional term is set to 0. An example for application of the described graph cut based cell splitting scheme is shown in Figure 2(d). 3.6

Performance Measurement

For the comparison of manually delineated ground truth with results obtained from automatic segmentation the Jaccard similarity measure is used. Denoting the set of pixels representing a single ground truth region of interest as Sgt and the pixels representing a segmentation result region of interest as Sres . Then, the Jaccard similarity OJ is defined as: |Sgt Sres | . (6) OJ = |Sgt Sres | For comparison of several ground truth objects with multiple segmentation results, the best matching ground truth object is searched for each segmented cell. i i Such a pair of optimal cells is denoted as (Sgt , Sres ) with i ∈ {1, ..., N } where N denotes the number of ground truth ROIs. As both the human observer as well as the algorithms use identical information on the cell nuclei no erroneously detected cells or missed cells appear and hence do not have to be considered. That is why the Jaccard similarity for a complete dataset is obtained by: OJN =

3.7

N 1 i (S , S i ). N i=1 gt res

(7)

Parameter Optimization

For parameter optimization in this work genetic algorithms are used as these have shown to optimize high dimensional parameter spaces efficiently. Specifically, the

462

C. Held et al.

GAlib is used [10]. To avoid learning by heart, a threefold cross validation process is included for performance comparison. Hence, the macrophage dataset is split into three equally sized sets, whereof two are used for training of the algorithm. Performance evaluation is carried out on the remaining subset. This procedure is repeated for each constellation of training and testing data. This makes sure that training and testing data are disjunctive.

4

Results

In this work three methods for segmentation and splitting of fluorescent micrographs are described and compared. To obtain comparable results a genetic algorithm is used for optimization of all parameters. Global convergence of the genetic algorithm is assumed if after identification of a new performance maximum no new optimum is determined for 200 new parameter combinations. Results of this optimization are the compared to the manually annotated ground truth. Thereby, the SWT achieves OJN = 0.653, the FM level sets OJN = 0.665 and the GC based method OJN = 0.657. For evaluation of intra observer variance a second manual annotation of 20% of the dataset is performed which results in OJN = 0.822. A more detailed analysis of the distribution of segmentation performance for the dataset is provided in Figure 3. Segmentation examples are shown in Figure 4.

i i Fig. 3. Comparison of detailed performance measurement (Sgt , Sres ) for the seeded watershed transform (SWT), the fast marching (FM) and the graph cut (GC) based cell splitting method. Additionally, the intra observer variance is shown for a subset of the data.


(a) Macrophages

(b) Annotated nuclei

(c) Ground truth

(d) SWT

(e) FM

(f) GC

463

Fig. 4. Visual comparison of segmentation results. Errors are indicated by white arrows

5

Discussion and Outlook

Quantitative comparison of the results obtained by application of the different cell splitting schemes shows that results obtained by the different methods are comparable. A detailed analysis of the example images shows that none of the cell splitting methods is able to completely eliminate the leaking out effect for all images in the dataset. Hence, for a further improvement of the segmentation quality accuracy of the threshold selection must be improved and the k-means clustering based method must be replaced by more accurate methods that are more robust to blur. In terms computational complexits the watershed transform and fast marching level set method is faster than the graph cut based cell splitting scheme. Using priority queues runtime of the watershed and fast marching level set algorithm is O(N logN ), where N denotes the number of pixels. Runtime of the graph cut algorithm is O(N 2 M ) with N denoting the number of vertices (or pixels) and M denoting the number of edges in the graph.

References 1. Lindblad, J., W¨ ahlby, C., Bengtsson, E., Zaltsman, A.: Image analysis for automatic segmentation of cytoplasms and classification of Rac1 activation. Cytometry Part A 57(1), 22–33 (2004) 2. Bengtsson, W.C.: Robust cell image segmentation methods. Pattern Recognition and Image Analysis 14(2), 157–167 (2004)

464

C. Held et al.

3. Yan, P., Zhou, X., Shah, M., Wong, S.T.C.: Automatic segmentation of highthroughput RNAi fluorescent cellular images. IEEE Transactions on Information Technology in Biomedicine: A Publication of the IEEE Engineering in Medicine and Biology Society 12(1), 109–117 (2008) 4. Yu, W., Lee, K.H., Hariharan, S., Bu, W., Ahmed, S.: Evolving generalized Voronoi diagrams for accurate cellular image segmentation. Cytometry Part A 77(4), 379–386 (2010) 5. Yu, W., Lee, H.K., Hariharan, S., Bu, W., Ahmed, S.: Quantitative neurite outgrowth measurement based on image segmentation with topological dependence. Cytometry Part A 75(4), 289–297 (2009) 6. Malpica, N.: Applying watershed algorithms to the segmentation of clustered nuclei. Cytometry A 28(4), 289–297 (1997) 7. Sethian, J.A.: Level set methods and fast marching methods. Cambridge University Press, Cambridge (1999) 8. Nilsson, B., Heyden, A.: A fast algorithm for level set-like active contours. Pattern Recognition Letters 24(9-10), 1331–1337 (2003) 9. Held, C., Wenzel, J., Webel, R., Marschall, M., Lang, R., Palmisano, R., Wittenberg, T.: Using Multimodal Information for the Segmentation of Fluorescent Micrographs with Application to Virology and Microbiology. In: Conf. Proc. IEEE Eng. Med. Biol. Soc., pp. 6487–6490 (2011) 10. Wall, M.: GAlib: A C++ library of genetic algorithm components (1996) 11. Smith, T.F., Waterman, M.S.: Identification of Common Molecular Subsequences. J. Mol. Biol. 147, 195–197 (1981) 12. May, P., Ehrlich, H.-C., Steinke, T.: ZIB Structure Prediction Pipeline: Composing a Complex Biological Workflow Through Web Services. In: Nagel, W.E., Walter, W.V., Lehner, W. (eds.) Euro-Par 2006. LNCS, vol. 4128, pp. 1148–1158. Springer, Heidelberg (2006) 13. Boykov, Y., Kolmogorov, V.: Computing geodesics and minimal surfaces via graph cuts (2003) 14. Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(9), 1124–1137 (2004)