6. REFERENCES. [1] R. Klein, B.E.K. Klein, and S.E. Moss, âThe relation of sys- ... [7] Kang Li, Xiaodong Wu, Danny Z. Chen, and Milan Sonka,. âOptimal surface ...
RETINAL VESSEL WIDTH MEASUREMENTS BASED ON A GRAPH-THEORETIC METHOD X. Xu1 , M. Niemeijer2,3 , Q. Song2, M. K. Garvin2 , J. M. Reinhardt1, M. D. Abr`amoff1,2,3,4 1
Dept. of Biomedical Engineering, and Dept. of Electrical and Computer Engineering, and 3 Dept. of Ophthalmology and Visual Sciences, University of Iowa, Iowa City, USA 4 Dept. of Veterans’ Affairs Center of Excellence for Prevention of Visual Loss and Blindness, Iowa City, USA 2
ABSTRACT A reliable and accurate method to measure the width of retinal blood vessel in fundus photography is proposed in this paper. Our approach is based on a graph-theoretic algorithm. The two boundaries of the same blood vessel are segmented simultaneously by converting the two-boundary segmentation problem into a two-slice, threedimension surface segmentation problem, which is further converted into the problem of computing a minimum closed set in a nodeweighted graph. An initial segmentation is generated from a vessel probability image. Two datasets from the REVIEW database were used to assess the performance of proposed method. This algorithm is robust and is able to produce accurate measurements. Index Terms— retinal photography, graph-based segmentation, vessel width measurement. 1. INTRODUCTION Accurate measurement of retinal blood vessel width (actually, the blood column width) in fundus photography is important as the AVRatio (Arterial/Venous width Ratio) is an independent risk factor for stroke, myocardial infarct, as well as eye diseases [1]. Automated determination of the AV ratio depends on an accurate estimate of retinal vessel width. In addition to AV-ratio analysis, automated segmentation of both vessel boundaries would also allow the quantification of the geometry of retinal blood vessels, also affected by cardiovascular disease, diabetes, and other systemic and eye diseases [2]. Accurate determination of local pathologic vascular changes such as tortuosity and vessel beading will also benefit from blood vessel width measurement [3]. However, an accurate measurement is complicated, because of the large variation in fundus images. Especially for small vessel branches, an accurate measurement can be complicated because of noise and the low vessel contrast. Numerous retinal vessel segmentation approaches have been published previously, but these mostly emphasize segmentation and localization of the vessels, and lack an accurate determination of vessel width. Niemeijer et al. proposed a pixel feature based retinal vessel detection method using a Gaussian derivative filter bank and k-nearest-neighbor classification [4]. Gang et al. proposed a retinal vessel detection using second-order Gaussian filter with adaptive filter width and adaptive threshold [5]. Al-Diri et al. proposed an algorithm for segmentation and measurement of retinal blood vessels by growing a “Ribbon of Twins” active contour model, the extraction of segment profiles (ESP) algorithm, which uses two pairs of contours to capture each vessel edge [6].
The proposed method segments the vessel boundary and measures the vessel width based on a graph-theoretic method. Graphbased method has been widely used in medical image processing recently [7] [8]. An initial segmentation is derived from the vesselness image [4]. A two-slice, three-dimensional graph is constructed based on the initial segmentation. It is further converted to the problem of solving the minimum closed set in a node-weighted graph. 2. MATERIALS We used the REVIEW database to estimate performance of our approach [9]. REVIEW is a publicly available database that contains a number of retinal profiles with manual measurements from three observers (http://reviewdb.lincoln.ac.uk), and we chose the largest two datasets, HRIS and VDIS in our validation. The High Resolution Image Set (HRIS) dataset consists of four images where 90 segments containing 2368 vessel profiles are manually marked. The images were subsequently down-sampled by a lateral factor of four, so that the vessel widths are known within ±0.25 of a pixel, discounting human error. The Vascular Disease Image Set (VDIS) consists of eight digitally captured images, six of which illustrate different types of diabetic retinopathy. A set of 79 segments containing 2249 profiles are manually marked. This set of images is more noisy and suffers from pathologies. Each profile in the REVIEW database consists of 15 numbers: series number, image number, segment number and four measurements for each of the three observers. For each observer, the four measurements indicate two points on each edge (x1 , y1 , x2 , and 2 y1 +y2 y2 ). The vessel centerline is defined as [ x1 +x , 2 ] and the ves2 p sel width as (x1 − x2 )2 + (y1 − y2 )2 for each observer [9]. A reference standard, denoted by RS, is then created by averaging the manual results of the three observers. 3. METHODOLOGY 3.1. Method Summary We convert the simultaneous two-boundary segmentation problem to an optimal surface problem on a two-slice three-dimensional graph. Before the graph can be constructed, a pre-processing step is necessary to obtain the initial segmentation. Image pre-processing is described in section 3.2. The construction of the graph is explained in section 3.3. The weights of the profile nodes in the graph are obtained using steerable first order derivatives of Gaussian as described in section 3.4.
3.2. Pre-processing An initial vessel segmentation is needed to build the graph. We use a vesselness map as proposed by Niemeijer et al. as our initial segmentation [4]. A region of a vesselness image is shown in Fig 1 (b). A binary vessel segmentation is generated by thresholding the gray scale image. We use a low threshold to better maintain the continuity of blood vessels. The vessel regions with an area smaller than 20 pixels are discarded as noise. A sequential thinning approach is then applied to the binary vessel segmentation to find the vessel centerlines, followed by a pruning step to erase small branches mostly caused by noise [10].
(a)
node step was 0.1 pixels. We used the “multi-column model” proposed by Li et al. [7]. Along each normal profile Col(x, y) every node V (x, y, z) (z > 0) has a directed arc to the node V (x, y, z − 1). Along the x-direction, meaning along the same boundary, for each node, a directed arc is constructed from V (x, y, z) ∈ Col(x, y) to V (x + 1, y, max(0, z − ∆x )) ∈ Col(x + 1, y). Similarly, arcs from V (x, y, z) ∈ Col(x, y) to V (x − 1, y, max(0, z − ∆x )) ∈ Col(x − 1, y, z) is constructed. ∆x is the maximum difference allowed between two adjacent normal profiles within one boundary. Along the y-direction, meaning between the two slices, arcs from V (x, y, z) ∈ Col(x, y) to V (x, y + 1, max(0, z − ∆y )) ∈ Col(x, y + 1, z) and arcs from V (x, y, z) ∈ Col(x, y) to V (x, y − 1, max(0, z − ∆y )) ∈ Col(x, y − 1, z) is constructed. ∆y is the maximum difference allowed between two corresponding normal profiles between the two boundaries. The base nodes are all connected [7]. A surface is regarded feasible if it satisfies the smoothness constraint defined by ∆x and ∆y . An optimal surface is defined as the surface with the minimum cost among all feasible surfaces defined in the 3-D volume.
(b) 3.4. Cost Function
(c)
(d)
Fig. 1. Image pre-processing and resulting cost image. (a) Part of a color fundus image. (b) The vessel probability image of (a). (c) The green channel image of (a). (d) The final cost image. The bifurcation points and crossing points are excluded from the vessel centerline image, because these points need special treatment. Bifurcation points and crossing points are detected by a sequence of 3 × 3 kernels [10]. By deleting the bifurcation points and crossing points, the vessel trees are cut into vessel segments. Each vessel segment is labeled and a three-dimension graph is constructed based on the vessel segment. 3.3. Graph-based vessel boundary segmentation For each labeled vessel segment, the vessel growing direction for every centerline pixel is calculated. We use adjacent centerline pixels on both sides of the target centerline pixel and apply principal component analysis on the resulting vector [11]. The first principal component corresponds to the growing direction of the pixel. The number of adjacent centerline pixels is related to the image resolution. A high resolution image needs a larger neighborhood. An end point that does not have enough neighboring centerline pixels to define the vessel growing direction is determined to have the direction of the nearest centerline pixel that has a definition of a vessel growing direction. A counter-clockwise 90◦ of the vessel growing direction is regarded as the normal direction for this point. Using the centerline pixels as the base nodes, profiles on the positive direction of the normals are built as one slice and profiles on the negative direction are built as another slice, as illustrated in Figure 2 and Figure 3. The
The cost image is generated from the orientation sensitive first order derivative of Gaussian of the green channel. The green channel of the color image usually shows the highest contrast between the blood vessels and background [12]. In order to find the gradient of vessels specified at different locations with different orientations, a steerable first order derivative of Gaussian filter is used in this study [13]. The separable first order derivative of Gaussian along the x-axis and along the y-axis are given in equations 1 and 2. The first order derivative of Gaussian along any angle θ is defined in equation 3. ◦
G01 = − ◦
x x2 exp(− 2 ), 2 σx 2σx
(1)
y2 y exp(− 2 ), 2 σy 2σy
(2)
G90 =− 1
◦
◦
Gθ1 = cos(θ)G01 + sin(θ)G90 1 ,
(3)
where σx and σy are the scale of the first order derivative kernel along the x-axis and y-axis. Because equations 1 and 2 are separable ◦ ◦ kernels, the original image is first convolved with G01 and G90 to 1 get the first order derivative image along the x-axis and y-axis. Then within each normal profile (as illustrated in figure 2), the weights of the profile nodes are calculated according to equation 3. Figure 1 (d) is an example of the final cost image. 3.5. Graph search and boundary determination The value of ∆x is set to one to maintain the smoothness within one boundary and ∆y is set to two so that a difference is allowed between widths from the centerline to the left edge and from the centerline to the right edge, in case the centerline is not exactly at the center of the two edges. After the node-weighted directed graph is constructed, the optimal surface is determined [7]. To determine the vessel width, the coordinate difference between the two corresponding nodes on the optimal surface from the two slices are calculated. To show the boundary on the image, the nearest integer coordinate of every node on the optimal surface is calculated and shown on the image.
(a)
(b)
(c)
(d)
Fig. 2. An illustration of how to calculate the normal profiles. (a) A small segment of vessel. (b) The vessel probability image of (a). (c) The vessel centerline image of (a). (d) The vessel growing directions are calculated and node normal profile perpendicular to the vessel growing direction are constructed. slice 1 (y=0)
slice 2 (y=1)
z
rameter σx and σy to be the same under the current study. Figure 5 shows the estimated average vessel width with regard to σ. The performance is relatively insensitive to the choice of scale parameter σ with σ < 6. When σ >= 6, the width measurement began to fail on some of the vessel segments (estimated width=0) and result in a smaller average vessel width. The final choice of the scaling parameters was σ = σx = σy = 3 for both datasets.
x
vessel growing direction base nodes base nodes
Fig. 3. An illustration of how to convert the two boundaries into a two-slice three-dimension graph. Red normal profiles are built into the red slice and green normal profiles are built into the green slice. The black nodes represent the base nodes. Red nodes represent the graph for one boundary. Green nodes represent the graph for the other boundary. Smoothness constraints are applied differentially to control the boundary smoothness within one boundary and between the two boundaries. (a) 4. RESULTS An example of the quality of the obtained vessel boundary detection is shown in Fig 4. It shows a good performance on even the smallest vessels, which have low contrast. The vessel width measurement results for REVIEW dataset are given in Table 1 and Table 2. In order to compare our measurement with the observers’ results, we determine the vessel centerline as explained in Section 3.2, and search the observers’ measurement profile for a corresponding centerline. If a corresponding centerline is found, the measuremen is labeled a “success measurement”. The percentage of “success measurement” is called success rate. The first three rows are the manual results from the observers. The last row shows the result of proposed method. The first column is the success rate. The columns 2-3 are the mean and standard deviation of all measurements labeled as “success measurement”. Column 45 are the signed mean and standard deviation of the point-by-point differences. The HRIS dataset has an image resolution of 3584 × 2438 pixels. The VDIS dataset has a resolution of 1360 × 1024 pixels. The performance of vessel width estimation by our method on both datasets were comparable to human observers. We chose scale pa-
(b) Fig. 4. Enlarged parts of result images. (a) An enlarged part of result image from HRIS dataset. (b) An enlarged part of result image from VDIS dataset. The detected vessel boundaries are shown in green.
Table 2. Vessel Width Measurement Accuracy of VDIS (success rate in percentage, mean µ and standard deviation σ in pixels) Method Success Measurement Difference Name Rate % µ σ µ σ Observer 1 100.0 8.50 2.54 -0.35 0.54 Observer 2 100.0 8.91 2.69 0.06 0.62 Observer 3 100.0 9.15 2.67 0.30 0.67 Proposed Method 96.0 8.59 2.44 -0.29 1.13
5. DISCUSSION AND CONCLUSION
average vessel width (pixels)
Table 1. Vessel Width Measurement Accuracy of HRIS (success rate in percentage, mean µ and standard deviation σ in pixels) Method Success Measurement Difference Name Rate % µ σ µ σ Observer 1 100.0 4.12 1.25 -0.23 0.29 Observer 2 100.0 4.35 1.35 0.00 0.26 Observer 3 100.0 4.58 1.26 0.23 0.29 Proposed Method 100.0 4.54 1.23 0.18 0.47
10
◦ HRIS * VDIS
9 8 7 6 5 4 3 0
2
4 6 8 scale parameter σ (pixels)
10
Fig. 5. A plot of average vessel width. The x-axis was scale parameter σ. The blue crosses was the average vessel width of VDIS dataset. The red crosses was the average vessel width of the HRIS dataset. Abramoff, “Retinal arterial but not venous tortuosity correlates with facioscapulohumeral muscular dystrophy severity,” Journal AAPOS, vol. 14, no. 1, pp. 240–243, 2010. [3] Vinayak Joshi, Joseph M. Reinhardt, and Michael D. Abramoff, “Automated measurement of retinal blood vessel tortuosity,” SPIE Medical Imaging, vol. 7624, 2010.
We introduced a novel method to find the retinal blood vessel boundaries using a graph-based approach. The results of our evaluation show a performance that is comparable to human observers. Our method allows us to find the accurate retinal vessel boundaries of fine vessels, which is usually difficult because of low image contrast and noise. The simultaneous two-boundary detection approach is able to detect both boundaries even if one of the boundary is noisy. This is also a robust algorithm.The final measurement result is relatively insensitive to the choice of the two scale parameters within a reasonable choice of σ . The proposed algorithm is fast. The running time for a retinal image of size 2160 × 1440 is about 120 seconds for image pre-processing and 60 seconds to solve the graph. The proposed algorithm still has some limitations. The crossing points and branching points are not treated now, because the vessel normal directions at those points are not well defined. A possible solution to the direction problem is the distance transform of the vessel centerline image. For the bifurcation points and crossing points, the direction with the maximum gradient can be considered as the normal profile direction. Another limitation is that the measurement method relies largely on the initial segmentation. False positives or disconnected vessels may result in incorrect vessel boundary detection and vessel width measurements. A low threshold value and spur pruning is one possible way to reduce the problem. But the trade-off is vessel discontinuity and the loss of fine vessel information. In summary, we proposed a retinal vessel boundary detection and vessel width measurement algorithm based on a graph-theoretic method. This method is tested on a publicly available dataset of expert annotated vessel widths. Our method showed a performance that is comparable to human experts.
[10] M. Sonka, V. Hlavac, and R. Boyle, Image Processing, Analysis, and Machine Vision, PWS, New York, 1998.
6. REFERENCES
[11] Jonathon Shlens, “A tutorial on principal component analysis,” http://www.snl.salk.edu/ shlens/pub/notes/pca.pdf, 2005.
[1] R. Klein, B.E.K. Klein, and S.E. Moss, “The relation of systemic hypertension to changes in the retinal vasculature: The Beaver Dam eye study,” Trans. Amer. Ophthalmological Soc, vol. 95, pp. 329–350, 1997. [2] Susannah Q. Longmuir, Katherine D. Mathews, Reid A. Longmuir, Vinayak Joshi, Richard J. Olson, and Michael D.
[4] M. Niemeijer, J. Staal, B. van Ginneken, M. Loog, and M.D. Abramoff, “Comparative study of retinal vessel segmentation methods on a new publicly available database,” SPIE Medical Imaging, vol. 5370, pp. 648–656, 2004. [5] Luo Gang, Opas Chutatape, and Shankar M. Krishnan, “Detection and measurement of retinal vessel in fundus images using amplitude modified second-order Gaussian filter,” IEEE Transactions on Biomedical Engineering, vol. 49, no. 2, pp. 168–172, 2002. [6] Bashir Al-Diri, Andrew Hunter, and David Steel, “An active contour model for segmenting and measuring retinal vessels,” IEEE Transactions on Biomedical Engineering, vol. 29, no. 9, pp. 1488–1497, 2009. [7] Kang Li, Xiaodong Wu, Danny Z. Chen, and Milan Sonka, “Optimal surface segmentation in volumetric images-A graph theoretic approach,” IEEE Trans Pattern Anal Mach Intell, vol. 28, no. 1, pp. 119–134, 2006. [8] Qi Song, Yinxiao Liu, Yunlong Liu, Punam K. Saha, Milan Sonka, and Xiaodong Wu, “Graph search with appearance and shape information for 3-D prostate and bladder segmentation,” MICCAI, vol. 6363, pp. 172–180, 2010. [9] “Review database,” http://reviewdb.lincoln.ac.uk.
[12] Sangyeol Lee, Michael D. Abramoff, and Joseph M.Reinhardt, “Retinal atlas statistics from color fundus images,” SPIE Medical Imaging, vol. 7623, no. 762310, 2010. [13] William T. Freeman and Edward H. Adelson, “The design and use of steerable filters,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 9, pp. 891–906, 1991.
