12
Comparison of boundary detection techniques to improve image analysis in medical thermography U I Bajwa*a, R Vardascab, F Ringb and P Plassmannb a Department of Computer Engineering, Center for Advanced Studies in Engineering, 19-Attaturk Avenue, G-5/1 Islamabad, Pakistan b
Medical Imaging Research Unit, Department of Computing and Mathematical Sciences, Faculty of Advanced Technology, University of Glamorgan, Pontypridd, Rhondda Cynon Taff CF37 1DL, UK
Abstract: In digital imaging, poor contrast between the target and its background can affect the extraction of the object of interest and increase the time used in its analysis. Medical thermal imaging requires the correct interpretation of the thermal values obtained from the region of interest. In this investigation, a subjective and objective comparison of currently available outlining techniques is performed to determine the optimal method. Results indicate that probability-based operators produce the best outcome especially after pre-processing with a noise removal filter. The findings of this study suggest that probability-based edge detection techniques in combination with homomorphic filtering and limited post-processing provide initial estimate delineations of areas. These delineations are of sufficient quality for subsequent automatic or semiautomatic post-processing so that a maximum of the original information inside the regions is preserved without loss or distortion of data. Keywords:
1
boundary, comparison, edge detection, infrared imaging, segmentation
INTRODUCTION
Medical thermal imaging is a technique based on the analysis of skin surface temperatures, which under certain conditions can reflect underlying normal or abnormal conditions in human physiology.1 The technique is, for example, helpful for analysing circulatory and neurological conditions. Thermal imaging is non-invasive, accurate and non-invasive.2 A medical infrared image is produced by converting the passive temperature radiation emitted by diverse regions of the body via an appropriate sensor array which creates a surface temperature image.3 The generally accepted capture protocol for thermal imaging of the human body defines examination The MS was accepted for publication on 25 August 2009. * Corresponding author: Usama Ijaz Bajwa, Department of Computer Engineering, Center for Advanced Studies in Engineering, 19-Attaturk Avenue, G-5/1, Islamabad, Pakistan; email:
[email protected]
The Imaging Science Journal Vol 58
room temperatures between 20 and 24uC, dependent on examination type.4 This temperature range is close to that of peripheral parts of the body, especially hands and feet, and it is therefore often difficult to identify their outline correctly in a thermal image. Accurate edge definition, however, is vital for a number of statistical image analysis procedures5 and a precise approach is needed in order to produce repeatable results. This study is an investigation into a range of established boundary extraction techniques to determine their practical performance in identifying objects with poor background contrast. Edge detection solutions using artificial intelligence methods such as genetic algorithms or neural networks have been suggested.6,7 These methods are computationally expensive and partly depend on heuristic parameters. If those parameters are not properly defined, errors will emerge. Other popular approaches use edge maps, which rely on the correct IMAG mp201 # RPS 2010
DOI: 10.1179/136821909X12520525092846
BOUNDARY DETECTION TECHNIQUES
identification image points both inside and outside the object of interest for adequate performance and their application can be complex.5 Using (non-thermal) digital images, two studies comparing traditional, well documented edge detectors have shown that probabilistic methods (Canny and Shen–Castan) based on average risk and signal to noise ratio (SNR) measures8,9 perform better than those mentioned above. An important property of infrared images when compared to normal digital images is the relatively high level of noise. Dependent on the thermal imaging sensor used, noise can be up to 5% of the dynamic signal range, e.g. in a thermal image with a measurement range between 16 and 36uC, this could be 1uC. It is therefore important to include noise reduction techniques into any study aimed at improving boundary detection in thermal images, and to evaluate if their inclusion aids subsequent segmentation. In the context of thermal images homomorphic filters are thought to be useful as they allow noise to be modelled as an additive term to the original image data, reducing image luminance and improving reflectance,10,11 which is an approximation of the physical processes inside a thermal camera sensor.
2
MATERIALS AND METHODS
A four-phase methodology is used in this study: image capture, image selection, algorithm application and comparison of results. All images of the human body used in this study were taken according to a capture protocol developed by the medical imaging research unit at the University of Glamorgan.2 The most important parameters of the protocol are:
N N N N N N N
examination room temperature between 20 and 24uC humidity of less than 50% subject acclimatisation period of 15 min minimal loose, comfortable clothes no jewellery, oils or ointments on the skin no food, coffee and tea intake 2 h before examination no strenuous physical activity (or physiotherapy) on the day of examination.
Images were taken using a FLIR A40 thermal camera with a resolution of 320 6 240 pixels, a measurement accuracy (bias, offset) of ¡2uC and a precision IMAG mp201 # RPS 2010
1
Homomorphic enhancement
13
filtering
approach
for
image
(repeatability) of ¡0.1uC. The camera was connected to a PC workstation using the CTHERM software package developed at the authors’ research unit. Thirty-five images with different views of the human body and poor thermal contrast between body and background were selected for this study. A homomorphic filter was used to study the effect of noise filtering in different boundary discovery techniques. It refines features and smoothes illumination variance in the image, homogenizing the brightness across the image while at the same time boosting the contrast. Image illumination and reflectance are not separable, but can be combined multiplicatively. This combination is made additively by taking the logarithm of the image intensity; it allows for linear separation of these components in the frequency domain. The variations in illumination are used as multiplicative noise, which can be reduced when filtered in the logarithmic domain. Normally, this filter is very effective in images with large variation in lighting as is the case in infrared images where the radiation emitted by the observed body is registered. The implementation of this technique covers five stages and receives five parameters; the parameters are: boost (the ratio that high frequency values are boosted as a percentage of low frequency values) where the value used was 2, cutoff (maximum frequency of the filter) where the value used was 0.25, order (order of the modified Butterworth style filter that is used) where the value used was 2, lower histogram threshold (percentage of the lower end of the filtered image’s histogram to be truncated, this eliminates extreme values in the image from distorting the final result) where the value used was 0 and higher histogram threshold (percentage of upper end of histogram to truncate) where the value used was 5. The five stages of the homomorphic filter are shown in Fig. 1. The first stage consists of calculating the natural log of image values, and the second stage deals with transforming these values to frequency domain where the illumination and reflectance can be handled separately by applying filter in third phase. In the fourth stage, the inverse discrete Fourier transform is taken to bring back the values in spatial domain. The last stage is to reverse the effect of first stage by exponentiation of the data. The Imaging Science Journal Vol 58
14
U I BAJWA ET AL.
A total of 11 algorithms from the following five classes of edge detection techniques were selected for processing of the images:
N
N
template-based detectors or gradient operators perform by calculating the first derivative of intensity. They are simple to implement and detect both edges and their orientation. Noisy image data affect their performance. Although this can be minimized by applying a Gaussian filter, this also removes much of the high frequency information present in edges. The four techniques selected from this class are the Roberts, Sobel, Prewitt and Kirsch detectors. The Roberts edge detector was among the first edge detectors introduced and it is probably the simplest. It is still widely used due to its simplicity and speed (mostly in hardware-based implementations). Its disadvantages are that it is asymmetric and that it cannot detect edges orientated at 45u or multiples thereof. The Sobel edge detector is the most promising of this class of operators as it implements noise removal and edge detection. It is, however, also the most complex to implement computationally. The Prewitt edge detector is simpler to implement than the Sobel and works well for the images, which are corrupted with Poisson type noise, but for other types of noises, it is deficient. The Kirsch edge detector has similar properties to that of the Sobel, but tends to perform slightly better on noisy images than the Sobel one. Both Kirsch and Sobel are superior to simple derivative operators as they apply rotated versions of masks/templates and thus find edges at different orientations11,12 second order difference operators have fixed characteristics for all edge orientations. Laplacian, Laplacian of Gauss and Marr–Hildreth were the selected operators in this class. They find the correct place of edges and also test a wider area around the pixel than the above gradient-based detectors. The disadvantages of these operators are their sensitivity to noise, possible multiple detection of the same edges, malfunctioning at corners/ curves and problems in places where the grey-level function varies. Edge orientation detection is affected due to the properties of the Laplacian approach, which looks for the alteration of the variation of the grey-level value for finding the correct zero-crossing. The Laplacian of Gauss approach uses the Laplacian method combined with a Gaussian smoothing filter. It can be
The Imaging Science Journal Vol 58
N
N
N
approximated by a discrete mask, which depends on the size of the Gaussian and the size of the kernel. The Marr–Hildreth method is an improvement on the Laplacian of Gauss technique. It locates the original edge from double edges by finding the zero crossings between the double edges but it is highly susceptible to noise11,12 the Canny and Shen–Castan algorithms were selected as examples of probabilistic operators: they have good localisation capabilities and response even in the presence of noise. Both compute probability values for determining an error rate. Their major disadvantages are poor detection of zero crossings and the complexity of computations11,12 as an example of segmentation-based operators, the Watershed algorithm was the selected. It filters the object’s boundaries and effectively removes image noise. This method first finds a gradient based on a threshold value, fills it to obtain edges, then searches for discontinuities in the image and finally tries to connect objects or border. It treats image foreground and background asymmetrically11,13 snakes were the selected as examples of active contour operators. Snakes are non-parametric and are based on internal and external energy terms: the internal term holds the curve together and prevents it from collapsing; the external attracts the curve to edges. This method needs an approximate contour outline as an input, which it then tries to improve. Snakes are able to reduce a second order problem to just one dimension and optimize locally. They are, however, relatively slow.11,14 These methods are categorized in two classes edge-based models and region-based models.15 In this specific study, with its emphasis on the comparison of different techniques, the edge-based class was used.
The 11 methods outlined above and the homomorphic filter were implemented in MatlabTM (MathWorks, Natick, MA, USA) and the 35 poor-contrast images were processed twice: in a first run, no noise preprocessing was used. In the second run, all images were subjected to homomorphic filtering before edge detection. In both processes, the parameters of the methods were the same, as described above. The general approach followed was that of using the respective default values for each method wherever possible and to use specific values only when needed. In particular, the parameters were set as outlined below: IMAG mp201 # RPS 2010
BOUNDARY DETECTION TECHNIQUES
2 Original image
N
N
N
N N
for gradient-based operators, thinning was applied automatically, and the threshold was also automatic. For the Sobel operator, the direction used was in multiples of 45u and on all others, it was horizontal and vertical the second order difference operators had as parameters the value 0.2 for the shape of the Laplacian. An automatic threshold and a standard deviation equal to 2 for the Laplacian of Gauss were used. Gaussian kernel size was 11, kernel width was 3 and standard deviation value was 1 (0 as mean of Gaussian for the Marr-Hildreth method) for the Canny detector, the low and high thresholds were obtained automatically according to the respective image values. Standard deviation was 6. For the Shen–Castan operator, the value of the smoothing factor was 1 for all images except the ones showing lower arms where it was 0.01. The value of the high threshold was varied across images in order to produce optimum results: values used were: arms 0.22, hands 3, lateral legs 9, feet 10, total lateral body 11, upper back 12, lower legs and thighs 13 and for anterior and dorsal total body views 19 the segmentation-based watershed method calculates its threshold value automatically based on the image data using eight connected neighbourhoods for snakes method, the parameters used were 0 as the energy contributed by the distance between control points, 0.1 as the energy contributed by the curvature of the snake and 1 pixel as a move unit. All parameters for this method were chosen to reduce the (long) computing time.
For the assessment of the algorithms’ performance, two approaches were followed, one subjective and IMAG mp201 # RPS 2010
15
3 Optimal outlining
one objective. The subjective approach used a ranking system completed independently by five image processing professionals. Objective assessment was performed using three different methods. The first method compared the length of the extracted boundaries with the respective ideal outlines by counting the amount of border pixels. The second method computes the SNR of the ideal outline image and the calculated outline image, i.e. the thermal images with the respective outlines overlaid in white colour which also represents the strongest thermal signal. The third method operates similarly but uses the root mean square error (RMSE).16
3
RESULTS
Figure 2 shows a typical input image with poor edgeto-background contrast. Figure 3 shows the corresponding optimal outline drawn manually. Figures 4 and 5 shows examples of images resulting from the application of the various edge detection methods with (Fig. 5) and without (Fig. 4) homomorphic filtering. Table 1 presents the averaged grading results from the five image processing professionals: 1 corresponds to the ‘best’ and 11 to what was considered the ‘worst’. According to this subjective manual grading scale, the best output was produced by the probabilisticbased methods followed by second order and active contour-based methods. Figure 6 summarizes the result of the ‘outline length’ benchmarking method obtained from the average of the 35 selected thermograms. The best edge detection algorithm when not using noise filtering is The Imaging Science Journal Vol 58
16
U I BAJWA ET AL.
4 Results of the application of the outlining methods without any pre-processing
5
Results of the application of the outlining methods with pre-processing of homomorphic filtering
The Imaging Science Journal Vol 58
IMAG mp201 # RPS 2010
BOUNDARY DETECTION TECHNIQUES
17
6 Average difference (%) between various outline lengths and reference length
Shen–Castan, followed very closely by the classical gradient-based methods (Roberts, Sobel, Prewit and Kirsch). When using pre-process noise filtering, the best results are produced by the gradient-based, probabilistic-based and classical operators. With the exception of the watershed method, homomorphic filtering enhances the results for all algorithms. The significance of the values has been tested using a Student’s t-test resulting in a p,0.05. Figure 7 shows the average results obtained from using the RMSE method to compare processed image outlines with the optimal outline. In this case, the Watershed method, followed by the Canny filter, is the most appropriate algorithm. Figure 8 presents the results obtained from using the SNR method. Here the edge detection methods that indicate better results were the Watershed and the Marr–Hildreth ones. 4
DISCUSSION
Since the subjective performance evaluation method relies on human judgement alone, it is limited by the number of characteristics that a human eye can Table 1 Averaged grading results of five specialists Outlining method
Filtered
Non-filtered
Roberts Sobel Prewitt Kirsch Laplacian Laplacian of Gauss Marr–Hildreth Canny Shen–Castan Watershed Snakes
8.8 8.2 8.4 7.1 6.6 5.1 3.6 2.8 1.8 8.4 5.2
10 8.9 9.1 7.9 6.1 3.7 3.4 2.3 1.3 7.3 5.9
IMAG mp201 # RPS 2010
reliably distinguish.8 For this study, a combination of subjective and objective validation was therefore employed. The number of pixels forming the outline was used as an objective comparison method as it is simple to compute and provides a single figure for grading results. It could be argued that the difference between the areas enclosed by the outlines would be a more suitable measure since it is these areas that are used for subsequent clinical analysis and this approach will thus be studied in the future work. The results of this work support previous studies that used other types of digital images.8,9 It demonstrates that traditional techniques which are usually computationally inexpensive and thus fast and simple to implement can produce adequate if not superior results to more complex recent approaches such as those based on artificial intelligence6,7 or edge maps.5 There are other methods available and their performance against those used in this experiment will always be open to question. The current methods were selected based on simplicity of implementation and on their more common usage in the digital world. Some authors have suggested a method based on gradient of the grey-level profile of the image.17 This method provides a mean of assessing the sharpness of the edges in printed images, and may produce some good results, although it is rarely used. The results of the application of active contour methods to object boundaries discovery is highly dependent on the initial estimate and that dependence questions the reliability of this method; even in some very simple experiments,18 it had proved to be inefficient. The application of the results of the present work is not just confined to medical application purposes, and will be applicable on other applications where thermal imaging is used such as forensic imaging. The Imaging Science Journal Vol 58
18
U I BAJWA ET AL.
7 Root mean square error average of each method in difference with optimal outline
Further object or image region outlining methods such as region-based active contours15 and future emerging techniques should be implemented and are suggested for future studies.
5
CONCLUSIONS
From this study, it can be concluded: 1. Probability-based edge detection techniques are the most suitable methods to outline body views in medical thermal images.
2. Homomorphic filtering enhances boundary detection by reducing noise and ‘clearing up’ previously undetectable constructive features that assist edge detection algorithms. 3. Some post-processing such as such as thinning, artifact removal, etc. is needed to improve the results. The above results provide a means for the production of initial estimates of outlines, landmarks or control points for the delineation of areas of interest within thermal images. These initial estimates allow the production of automatic or semiautomatic systems
8 Signal to noise ratio average of each method in difference with optimal outline The Imaging Science Journal Vol 58
IMAG mp201 # RPS 2010
BOUNDARY DETECTION TECHNIQUES
for the standardisation of thermal image analysis, thus reducing errors while simultaneously improving speed. An example of such an application could be a morphing/warping tool of regions of interest in a source image towards a standard reference position with minimal introduction of processing errors. Such an application could be used to overlay thermal images of several subjects in a unified way in order to facilitate comparison or averaging (mean and standard deviation) and for example, be used to produce a normative reference image database from normal subjects. Despite the 40–50 year availability of infrared thermal imaging in medicine, such an ‘Atlas of Temperature Normals’ has not been produced yet. This study is part of a research programme to develop such a reference Atlas.19
ACKNOWLEDGEMENT The authors thank Professor Olivier Laligant and Professor Frederic Truchetet from Laboratoire Le2i at the Universite de Bourgogne in France for their help in implementing the Shen–Castan algorithm.
REFERENCES 1 Hobbins, W. In Biomedical Thermology, 1982, pp. 361– 375 (Alan R. Liss, Inc., New York). 2 Ring, E. F. J. and Ammer, K. The technique of infra red imaging in medicine. Thermol. Int., 2000, 10, 7–14. 3 Jones, B. and Plassmann, P. Digital infrared thermal imaging of human skin. Eng. Med. Biol. Mag., 2002, 21, 41–48. 4 Ring, E. F. J., Ammer, K., Jung, A., Murawski, P., ˙ uber, J., Zwolenik, S., Plassmann, P., Wie˛cek, B., Z Jones, C. D. and Jones, B. Standardization of infrared imaging, Proc. 26th Annual Int. Conf. of the IEEE Engineering in Medicine and Biology Society: IEMBS ’04, San Francisco, CA, USA, September 2004, IEEE, pp. 1183–1183. 5 Zhou, Q., Li, Z. and Aggarwal, J. K. Boundary extraction in thermal images by edge map, Proc. 2004 ACM Symposium on Applied computing, Nicosia, Cyprus, March 2004, ACM, pp. 254–258. 6 Suzuki, K., Horiba, I. and Sugie, N. Edge detection from noisy images using a neural edge detector, Proc. 10th IEEE Workshop on Neural networks for signal
IMAG mp201 # RPS 2010
7
8
9
10
11
12
13
14
15
16
17
18
19
19
processing: NNSP 2000, Sydney, NSW, Australia, December 2000, IEEE, pp. 487–496. Ghosh, P. and Mitchell, M. Segmentation of medical images using a genetic algorithm, Proc. 8th Annual Conf. on Genetic and evolutionary computation: GECCO 2006, Seattle, WA, USA, July 2006, ACM, pp. 1171–1178. Sharifi, M., Fathy, M. and Mahmoudi, M. T. A classified and comparative study of edge detection algorithms, Proc. Int. Conf. on Information technology: coding and computing: ITCC’02, Las Vegas, NV, USA, April 2002, IEEE Computer Society, pp. 117–120. Roushdy, M. Comparative study of edge detection algorithms applying on the grayscale noisy image using morphological filter. GVIP J., 2006, 6, 17–23. Arsenault, H. H. and Levesque, M. Combined homomorphic and local-statistics processing for restoration of images degraded by signal-dependent noise. Appl. Opt., 1984, 23, 845–850. Gonzalez, R. C., Woods, R. E. and Eddins, S. L. Digital Image Processing Using Matlab, 2nd edition, 2004 (Pearson Education, Upper Saddle River, NJ). Gonzalez, R. C. and Woods, R. E. Digital Image Processing, 2nd edition, 2002 (Addison-Wesley, Reading, MA). Karantzalos, K. and Argialas, D. Improving edge detection and watershed segmentation with anisotropic diffusion and morphological levellings. Int. J. Remote Sens., 2006, 27, 5427–5434. Kass, M., Witkin, A. and Terzopoulos, D. Snakes: active contour models. Int. J. Comput. Vision, 1988, 1, 321–331. Li, C., Kao, C., Gore, J. C. and Ding, Z. Implicit active contours driven by local binary fitting energy, Proc. IEEE Conf. on Computer vision and pattern recognition: CVPR ’07, Minneapolis, MN, USA, June 2007, IEEE, pp. 1–7. Kinape, R. and Amorim, M. A study of the most important image quality measures, Proc. 25th Annual Int. Conf. of IEEE/EMBS, Cancun, Mexico, September 2003, IEEE, Vol. 1, pp. 934–936. Fox, I. J., Claypole, T. C. and Bohan, M. F. J. Print sharpness evaluation using image analysis and a new peak area algorithm. Imag. Sci. J., 2003, 51, 199–211. Zhu, G. P., Zeng, Q. S. and Wang, C. H. Robust shape based active contour for circle detection. Imag. Sci. J., 2008, 56, 175–178. Ring, E. F. J., Ammer, K., Wiecek, B. and Plassmann, P. Technical challenges for the construction of a medical IR digital image database. Proc. SPIE, 2005, 5964, 191– 198.
The Imaging Science Journal Vol 58
