An Automated Image Processing System for the Detection of Photoreceptor Cells in Adaptive Optics Retinal Images Anfisa Lazareva 1, Panos Liatsis 1, Franziska G. Rauscher 2 1
School of Engineering and Mathematical Sciences, City University London, London, UK 2 Department of Ophthalmology, Leipzig University Hospital, Leipzig, Germany
[email protected]
Abstract - This paper presents an automated image processing framework for facilitating the accurate detection of photoreceptor cells. The performance of the proposed method was evaluated in terms of cone density calculated on synthetic and high-resolution retinal images. The validation study on the synthetic data showed an average accuracy of 98.8% for the proposed method in comparison with 93.9% obtained by the Li and Roorda algorithm. The cone density calculated on the high-resolution retinal images demonstrated satisfactory agreement with the histological data as well as previously published data on photoreceptor packing density at a given location. Keywords - Image enhancement; Adaptive Optics retinal imaging; Photoreceptor cells; Object detection
I.
INTRODUCTION
The rapid progress in Adaptive Optics (AO) imaging, in the last decades, changed the entire approach underpinning the investigations of retinal tissues. Capable of imaging the retina in-vivo at cellular level, AO systems revealed new insights into retinal structures, function, and the origins of various retinal pathologies. Advances in image processing techniques contribute to a better observation of retinal microstructures and therefore more accurate detection of pathological conditions. Quantitative analysis of highresolution retinal images assists eye professionals in the diagnosis and follow-up of patients with degenerative retinal diseases. Based on the measurements of photoreceptor loss, it is possible to interpret retinal disorders and the associated visual deterioration [1-4]. Therefore, it is of vital importance to develop automated diagnostic tools capable of detecting photoreceptor cells and thereby assisting the diagnosis of eye pathologies during the early stages of their development. Initial attempts in automatic photoreceptor detection algorithms were made by Li and Roorda [5] and Xue and Choi [6]. In both methods, the photoreceptors cell location is identified as the regional maxima of the image. A manual input is required for such parameters as the intensity threshold and inter-cell spacing. This can adversely affect the inter-rater reliability of cone density calculations and thus objective judgment on the patient’s diagnosis. Loquin et al. [7] implemented an interactive tool that allows physicians to adjust the parameters of the algorithm in order to achieve
optimal cone detection. This approach as well as manual cone counting procedures [8-10] are generally impractical solutions in regular clinical examination where a large number of images needs to be processed. In this paper, an automated framework is presented for processing AO high-resolution retinal images. The proposed image processing framework consists of several stages: illumination compensation, noise suppression, image registration, image enhancement and cone detection. Details regarding the formulation of the implemented model and the performed validation test-cases are discussed in Section II. II. METHODS Due to various photoelectric noise sources in AO imaging system, high-resolution retinal images undergo a degradation process, which results in low signal-to-noise ratio and poor contrast [11]. Moreover, the visibility of retinal features is altered by inhomogeneous illumination, caused by imperfections in the imaging optics. In order to improve visualization of retinal images and facilitate next steps of the image processing framework, wavelet-based illumination correction was performed on each frame of the retinal datasets [12]. A wavelet-Fourier filter [13] was used as modification to this model. It was applied to the reconstructed retinal images to eliminate block artefacts, and thereby to preserve the integrity of the retinal features. In order to further increase the quality of high-resolution retinal images, thus allowing for better distinction of photoreceptor cells, noise suppression was performed on the retinal datasets. Since the scale of the retinal features is known, a Gaussian band-pass filter was chosen as the method to suppress the high frequency noise, above photoreceptor cone stop frequency, and enhance the image contrast, within the start-stop frequency [11]. Based on the scale of the retinal features, corresponding frequencies were calculated and used as a stop and the start-stop frequency of the band-pass filter. In order to compensate for the eye motions inflicted between the frames during image acquisition, a transformation model was calculated for each pair of retinal frames using image registration. The translation vector was found using Phase-only correlation [14] which served as an
initial estimate for the transformation model. Then, tracking of brightest cones was utilized in order to calculate the rotation parameter and correct for the residual displacement. The Procrustes algorithm [15] was employed to refine the transformation model. The final image was calculated by averaging the frames in order to compensate for the photon noise. Human photoreceptor mosaic is characterized by a hexagonal pattern of organisation. Due to light scattering during image acquisition and optical limitations of the AO system, the observed structures can be approximated by circular structures of different diameters. Based on this assumption, we employed the blob/dot enhancement method proposed in [16]. This method is based on multiscale analysis of the Hessian matrix. For circular structures, the enhancement filter λdot is computed as follows: 2 2 dot (1 , 2 ) , if 1 0 and 2 0 1 0, otherwise,
(a)
(b)
(c)
(d)
(e)
(f)
(1)
where λ1 and λ2 are the eigenvalues of the Hessian matrix. In order to enhance objects of all scales and at the same time compensate for the residual noise, the original image is firstly filtered with the second-order derivatives of the Gaussian function. For each object size in the scale range [d0, d1], the corresponding Gaussian kernels i are calculated as:
1
d0 d , 2 r 1 , 3 r 2 2 ,.. N r N 1 1 , 4 4 1
where r
d1 d0
(2)
1 /( N 1)
and N is a number of Gaussian kernels.
After applying the dot enhancement filter (Fig. 1b), the contrast of photoreceptor cells was further improved by convolving the image with the a Laplacian of Gaussian (LoG) operator [17]. The filter has the shape of a circular center region with positive weights, surrounded by another circular region with negative weights. Therefore, applying this kernel over an image containing circular objects increases the contrast between the regions of interest (Fig.1c). After this process, photoreceptor cells can be easily segmented by thresholding the image (Fig. 1e). The cone coordinates are calculated as the maximum intensity of the original image within the region of interest. The photoreceptor cells located at the boundaries of the image are excluded from the counting procedure (Fig. 1f). The performance of the proposed Hessian-LoG filter for cone counting was initially validated on the synthetic data with predefined cell count. To this purpose, we employed the algorithm for artificial retinal image simulation as described in [18]. Following this approach, two sets of ten artificial retinal images were created, imitating the retina at 0.59 mm and 1.1 mm eccentricity. The obtained cone counts were also compared against those calculated with the use of the wellknown Li and Roorda algorithm.
Figure 1. Steps of Hessian-Log filtering for cone counting; (a) synthetic retinal image; (b) image (a) processed with a blob enhancement filter; (c) image (b) convolved with the LoG operator; (d) image (c) with negative values set to 0; (e) binary image with the boundary cones excluded; (f) result of cone counting overlaid on the image (a).
In order to validate the proposed image processing framework on real retinal data, four datasets with highresolution retinal images were acquired with the commercial AO-assisted flood illumination system (rtx1, Imagine Eyes, Orsay, France). The obtained datasets were post-processed using the proposed image processing framework and the calculated cone packing densities were compared against available histological data [19] as well as results published by Song et al. [20]. The results of the aforementioned validation test-cases are presented in the next section. III.
RESULTS
A. Synthetic Data Table I summarizes the averaged results for the tested sets of synthetic images, in terms of the packing density, obtained by the proposed cone counting method and the Li and Roorda algorithm. The density of human photoreceptor mosaic peaks in the fovea and declines rapidly as moving away from the fovea [21]. Therefore, in the images
representing the section of retina further away from the fovea (set B), the cone contrast is generally better and thus the accuracy of cone identification is higher. An average accuracy of 99.4% was shown by the proposed method and 97.9% respectively by the Li and Roorda algorithm. In the images imitating retina closer to the fovea (set A), the photoreceptor cells are densely populated and the image sharpness is significantly poorer. This results in a lower detection rate in both cone counting methods: 98.1% for the proposed method and 89.8% for the Li and Roorda algorithm. TABLE I.
B. Adaptive Optics Retinal Images Four image datasets acquired at different retinal eccentricities at four meridians of the retina (Temporal, Nasal, Superior and Inferior) were post-processed with the proposed image processing framework. Fig. 3 illustrates the section of high-resolution retinal image before and after enhancement. As it can be seen from the images and calculated image metrics ‒ sharpness [22], contrast [23] and variance ‒ the image quality has been significantly increased and the visibility of photoreceptor cells improved (Table II).
RESULTS OF THE PROPOSED C ONE COUNTING ALGORITHM AND THE ALGORITHM OF L I AND ROORDA Current method, set A
Li and Roorda, set A
Current method, set B
Li and Roorda, set B
Average accuracy
98.1%
89.8%
99.4%
97.9%
False positives
0.06 %
2.3%
0.2%
1.7%
False negatives
1.9%
10.2%
0.6%
2.1%
Fig. 2 demonstrates the results of two cone counting methods in set A and B. For better visibility, the detected cone locations are overlaid on the image with disks, which is created as the initial step in the process of artificial retinal image simulation [18]. As it can be noticed from the images and Table I, the Li and Roorda algorithm gives a higher number of false positives and false negatives than the proposed method. This happens due to the use of the dilation operator which tends to merge closely located cones. Current method
Figure 3. Original high-resolution retinal image (left); post-processed retinal image (right). TABLE II.
QUALITY ASSESSMENT OF ORIGINAL AND PROCESSED RETINAL IMAGES
Image contrast
Original retinal image 0.0017
Post-processed retinal image 0.0033
Image variance Image sharpness
0.0013 0.0214
0.0025 0.0445
Li and Roorda
(a)
(b)
(c)
(d)
Figure 2. Results of cone counting obtained with the proposed method and the Li and Roorda algorithm; (a) and (b) detected cone coordinates overlaid on the initial image with disks from set B; (c) and (d) detected cone coordinates overlaid on the initial image with disks from set A.
After processing four retinal datasets, cone packing density was calculated in the retinal area of 1 mm2 on the resulting retinal images using the proposed Hessian-LoG filter. Fig. 4 shows an example of cone counting on two images acquired at 0.9 mm Temporal and Nasal eccentricities. The obtained results showed a satisfactory agreement with photoreceptor packing density at a given location published in [20] as well as the histological data of Curcio et al. [19] (Table III).
Figure 4. Result of cone counting on the AO high-resolution retinal images with the proposed method at different retinal locations: (left) at 0.9 mm Nasal, (right) at 0.9 mm Temporal eccentricity.
TABLE III.
VALIDATION OF C ALCULATED CONE PACKING DENSITY Song et al. (× 103 cells/mm2)
Curcio et al. (× 103 cells/mm2)
Current method (× 103 cells/mm2)
0.9 mm2 Nasal
24.2 ± 1.2
~21 874
21 327
0.9 mm2 Temporal
24.1 ± 1.5
~20 300
19 616
2
18.5 ± 1.2
~17 187
18 892
2
19.4 ± 1.6
~19 270
20 085
0.9 mm Superior 0.9 mm Inferior
[3]
[4]
[5]
[6]
IV. CONCLUSION AND DISCUSSION In this paper, we presented an automated image processing framework for cone density calculation in AO high-resolution retinal images. Following the stages of illumination compensation, noise suppression and image registration, the Hessian-LoG filter was employed in order to enhance the photoreceptor cells and facilitate their detection. Benefiting from both, geometric and intensity information, the proposed cone-counting approach appeared to be more accurate than the existing Li and Roorda algorithm. The validation tests performed on the synthetic data showed an average accuracy of 98.8% for the proposed method in comparison with 93.9% obtained by the Li and Roorda algorithm. Preliminary results on the real retinal datasets correlate well with the histological data and results available in the literature. A slight deviation in Nasal and Temporal directions can be explained by the presence of large blood vessels that obscure the detection of cones. The results presented here provide a basis for characterizing the density and spacing distribution of photoreceptor cells in-vivo. Quantitative analysis of the final images obtained with the proposed image processing framework can be used for future comparison with data related to pathological retinas, as well as for understanding the effect of age and retinal pathology on cone packing density and other microstructures. Therefore, the current study can be viewed as an initiation of a much broader research work which is required in order to thoroughly examine the performance of the proposed model for different pathological cases and derive correlations between spatial arrangement of cone mosaic and visual function of the eye. ACKNOWLEDGMENT We would like to thank Imagine Eyes, Orsay, France, especially Dr. Laurent Vabre, Dr. Nicolas Chateau and Loic Le Kernevez, for their assistance and provision of retinal images for this study.
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15] [16]
[17]
[18] [19]
[20]
REFERENCES [1]
[2]
A. Boretsky, F. Khan, G. Burnett, D. X. Hammer, R. D. Ferguson, F. van Kuijk, and M. Motamedi, “In vivo imaging of photoreceptor disruption associated with age-related macular degeneration: A pilot study.,” Lasers Surg. Med., vol. 44, no. 8, pp. 603–10, Oct. 2012. J. Tam, K. P. Dhamdhere, P. Tiruveedhula, S. Manzanera, S. Barez, M. A. Bearse, A. J. Adams, and A. Roorda, “Disruption of the retinal parafoveal capillary network in type 2 diabetes before the onset of
[21]
[22]
[23]
diabetic retinopathy.,” Invest. Ophthalmol. Vis. Sci., vol. 52, no. 12, pp. 9257–66, Nov. 2011. E. Lieth, T. W. Gardner, A. J. Barber, and D. A. Antonetti, “Retinal neurodegeneration: early pathology in diabetes.,” Clin. Experiment. Ophthalmol., vol. 28, no. 1, pp. 3–8, Feb. 2000. M. Parravano, M. Lombardo, G. Lombardo, B. Boccassini, S. Lioi, and M. Varano, “In vivo investigation of the retinal microscopy in patients with type 1 diabetes mellitus,” Invest. Ophthalmol. Vis. Sci., vol. 53, no. 6, p. 5657, Mar. 2012. K. Y. Li and A. Roorda, “Automated identification of cone photoreceptors in adaptive optics retinal images.,” J. Opt. Soc. Am. A. Opt. Image Sci. Vis., vol. 24, no. 5, pp. 1358–63, May 2007. B. Xue, S. S. Choi, N. Doble, and J. S. Werner, “Photoreceptor counting and montaging of en-face retinal images from an adaptive optics fundus camera.,” J. Opt. Soc. Am. A. Opt. Image Sci. Vis., vol. 24, no. 5, pp. 1364–72, May 2007. K. Loquin, I. Bloch, K. Nakashima, F. Rossant, and M. Paques, “Photoreceptor detection in in-vivo adaptive optics images of the retina: Towards a simple interactive tool for the physicians,” in Proceedings - International Symposium on Biomedical Imaging, pp. 191–194. J. Carroll, M. Neitz, H. Hofer, J. Neitz, and D. R. Williams, “Functional photoreceptor loss revealed with adaptive optics: an alternate cause of color blindness.,” Proc. Natl. Acad. Sci. U. S. A., vol. 101, no. 22, pp. 8461–6, Jun. 2004. J. I. Wolfing, M. Chung, J. Carroll, A. Roorda, and D. R. Williams, “High-resolution retinal imaging of cone-rod dystrophy.,” Ophthalmology, vol. 113, no. 6, p. 1019.e1, Jun. 2006. N. M. Putnam, H. J. Hofer, N. Doble, L. Chen, J. Carroll, and D. R. Williams, “The locus of fixation and the foveal cone mosaic.,” J. Vis., vol. 5, no. 7, pp. 632–9, Jan. 2005. C. Rao, T. Yu, and B. Hua, “Topics in Adaptive Optics. AO-Based High Resolution Image Post-Processing,” in Topics in Adaptive Optics, R. K. Tyson, Ed. InTech, 2012, pp. 69–94. G. Ramaswamy and N. Devaney, “Pre-processing, registration and selection of adaptive optics corrected retinal images.,” National University of Ireland, Galway, 2013. B. Münch, P. Trtik, F. Marone, and M. Stampanoni, “Stripe and ring artifact removal with combined wavelet-Fourier filtering,” Opt. Express, vol. 17, no. 10, p. 8567, May 2009. C. D. Kuglin and D. C. Hines, “The phase correlation image alignment method,” in Proceedings of IEEE Cybernet Society, 1975, pp. 163–165. S. Chris and B. Toby, Fundamentals of Digital Image Processing: A Practical Approach with Examples in Matlab. Wiley-Blackwell, 2011. Q. Li, S. Sone, and K. Doi, “Selective enhancement filters for nodules, vessels, and airway walls in two- and three-dimensional CT scans.,” Med. Phys., vol. 30, no. 8, pp. 2040–51, Aug. 2003. S. Zhenghao, M. Zhao, Y. Wang, H. Lifeng, K. Suzuki, M. Zhang, and C. Jin, “Hessian-LoG: a novel dot enhancement filter,” ICIC Express Lett., vol. 6, no. 8, pp. 1987–1992, 2012. L. Mariotti and N. Devaney, “Performance analysis of cone detection algorithms,” J. Opt. Soc. Am. A, vol. 32, no. 4, p. 497, Mar. 2015. C. A. Curcio, K. R. Sloan, R. E. Kalina, and A. E. Hendrickson, “Human photoreceptor topography.,” J. Comp. Neurol., vol. 292, no. 4, pp. 497–523, Feb. 1990. H. Song, T. Y. P. Chui, Z. Zhong, A. E. Elsner, and S. A. Burns, “Variation of cone photoreceptor packing density with retinal eccentricity and age.,” Invest. Ophthalmol. Vis. Sci., vol. 52, no. 10, pp. 7376–84, Sep. 2011. J. Carroll, S. S. Choi, and D. R. Williams, “In vivo imaging of the photoreceptor mosaic of a rod monochromat.,” Vision Res., vol. 48, no. 26, pp. 2564–8, Nov. 2008. D. Kanjar and V. Masilamani, “A new no-reference image quality measure for blurred images in spatial domain,” J. Image Graph., vol. 1, no. 1, 2013. E. Peli, “Contrast in complex images,” J. Opt. Soc. Am. A, vol. 7, no. 10, p. 2032, Oct. 1990.
