Three-Dimensional Nonlinear Invisible Boundary ... - IEEE Xplore

3020

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 10, OCTOBER 2006

Three-Dimensional Nonlinear Invisible Boundary Detection Maria Petrou, Senior Member, IEEE, Vassili A. Kovalev, and Jürgen R. Reichenbach

Abstract—The human vision system can discriminate regions which differ up to the second-order statistics only. We present an algorithm designed to reveal “hidden” boundaries in gray level images, by computing gradients in higher order statistics of the data. We demonstrate it by applying it to the identification of possible “hidden” boundaries of glioblastomas as manifest themselves in three-dimensional (3-D) MRI scans, using a model driven approach. We also demonstrate the method using a nonmodel driven approach where we have no prior information about the location of possible boundaries. In this case, we use 3-D MRI data concerning schizophrenic patients and normal controls. Index Terms—Boundary detection, image filtering, invisible boundary, nonlinear edge detection, three-dimensional (3-D) volume data.

I. INTRODUCTION

L

INEAR edge detectors are based on smoothing followed by differentiation where either the first or the second-order derivative of the signal is computed [9]. In either case, the edges detected identify the places where the mean value of the signal changes significantly. Edge detectors designed to detect boundaries when the spatial statistics of the data change are known as texture segmentation algorithms (e.g., [8] and [16]). We are not interested here in texture in the commonly understood sense of the word, i.e., in spatial patterns. We are interested in detecting boundaries between regions which do not form coherent spatial patterns, but, rather, look random, their only difference being the underlying probability density function from which the observed values are drawn. In particular, we are interested in detecting boundaries where the higher order moments of the probability density function, from which the observed values are drawn, change. The motivation of our work stems from a medical application, and the fact that the boundaries of some malignant tumours are diffuse and invisible to the naked eye. A commonly accepted model for an idealised solid tumour consists of a spherical body made up from several concentric shells: Manuscript received January 19, 2005; revised March 7, 2006. This work was supported in part by EPSRC under a portfolio Grant on Integrated Electronics, in part by the European Cooperation in the Field of Scientific and Technical Research (COST) Action B21 on the Physiological modeling of MR image formation. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Nicolas Rougon. M. Petrou is with the Electrical and Electronics Engineering Department, Imperial College, London SW7 2AZ, U.K. (e-mail: [email protected]). V. A. Kovalev is with the Centre for Vision, Speech, and Signal Processing, School of Electronics and Physical Sciences, University of Surrey, Guildford, GU2 7XH, U.K. (e-mail: [email protected]). J. R. Reichenbach is with the Medical Physics Group of the Institute of Diagnostic and Interventional Radiology, Friedrich-Schiller University Jena, D-07743 Jena, Germany (e-mail: [email protected]). Digital Object Identifier 10.1109/TIP.2006.877516

1) inner core composed of apoptotic cells and necrotic tissue, 2) a quiescent region composed of cells which are alive but nonproliferative, and 3) a proliferative rim with access to a nutrient concentration sufficient to maintain active cellular division, thus increasing the overall tumour size by proliferation. Therefore, we believe that there should be a boundary between tumorous and healthy tissue which may or may not be seen by conventional standard MRI methods, but which nevertheless exists on a cellular level. Ordinary mean value-based edge detectors [9] are only appropriate for detecting the boundary of the main body of the tumour which is clearly distinct from its surroundings in MRI images, due to its smooth and distinct gray value. The true tumour boundary lies outside that region, and it is neither a texture boundary nor a gray level boundary. From psychophysical experiments it is known that the human eye can distinguish borders between regions that differ in the first or second-order statistics only [4], [5]. Fig. 1 shows examples of pairs of regions which differ in the first, second and third-order gray value statistics, separated by a horizontal edge. The boundary in the third panel, where the probability density functions differ in their third-order statistics only (having the same mean and standard deviation), is hidden. These examples demonstrate that the human vision system cannot discriminate easily between regions that differ in their third and higher order statistics. For all the above reasons, we choose to investigate nonlinear edge detection algorithms [11], [2] that can identify third- or fourth-order gray level statistical differences. II. REQUIREMENTS FOR USING THIRD- AND FOURTH-ORDER STATISTICS To create simulated data that differ in their third- or higher order moments we need a probability density function that depends on at least three parameters. Such a function is the Pearson distribution. The probability density function of Pearson type III is given by

(1) , and , and where is the Gamma function of parameter . In Pearson type (second moment), III, the mean (first moment), variance (third moment), and kurtosis (fourth moment) skewness are given by the following formulae in terms of the parameters of the distribution [1]:

1057-7149/$20.00 © 2006 IEEE

(2)

PETROU et al.: THREE-DIMENSIONAL NONLINEAR INVISIBLE BOUNDARY DETECTION

Fig. 1. Pairs of regions which differ in their first-, second-, and third-order statistics. The graphs show the normalised histograms of the two regions of each image. As it can be seen, differences in the first and second-order statistics are visible but the difference in the third-order statistics is hardly discernible. In all three cases, the signal strength is the same. Top:   = = 1:0; middle:   = = 1:0; bottom:

= = 1:0, with referring to the third moment of the j th probability density function.

j 0 j

j

0

j

j 0 j

Sets of random numbers were created according to this probability density function, using a lookup table of the values of the with 25 600 entries corresponding distribution of

(3) was computed by implementing the relevant routine from [12]. Fig. 2 shows histograms of such numbers with the corresponding theoretical curve superimposed. A study was performed on the accuracy with which the third and fourth-order statistics could be computed from such data. The results of such experiments for three Pearson distributions which correspond to typical, minimum and maximum values of the third moment encountered in the real data we use in Section V-A are reported in Table I. Each experiment was repeated 100 times with different seeds for the random number generator, so that the mean and standard deviation of the relative erand can be estimated were extracted. It rors with which can be seen that for typical values of the third-order moment

3021

a minimum of 700 points are needed for the errors to stabilise around 10%. It is very difficult to have such a number of points in a local filter. The errors are much higher for fixed number of points when the third moment is weak and lower when it is strong. Filters of minimum size 9 9 9 should be used. However, even with 100 points, the third moment may be computed with an accuracy of about 20%, so we shall investigate this moment as a possible boundary indicator. The fourth moment appears to be very unreliable for use with local windows. In fact our experiments showed that we needed at least 2000 points to stabilise the error of its estimation to around 20%. There are two points worth discussing here. First, the use of such filters for local boundary detection may preferably be performed in conjunction with three-dimensional (3-D) data. For fixed resolution imagery, in two dimensions, one must use much larger filters in order to achieve the same statistical reliability as a smaller filter in 3-D. For example, one must use a 33 33 rectangular image patch in order to achieve the same accuracy as with a 10 10 10 3-D patch. This makes the usefulness of such filters particularly relevant to the analysis of 3-D data. Second, one may try to detect hidden edges by looking for changes in the full distribution of gray values. This is possible, and we are also performing experiments doing it. However, calculating the third moment, instead of using the whole probability density function, has the advantage that it makes explicit a characteristic of the distribution that is only implicit in it when the whole distribution is used. As such, the third moment is expected to be a more sensitive measure of change than the whole distribution. III. THREE-DIMENSIONAL NONLINEAR EDGE DETECTORS A nonlinear edge detector consists of a sliding window perpendicular to the direction of the hypothesised boundary [2]. In each half of the window, some statistic of the data is computed. The difference of the values computed in the two halves of the window is assigned to the central point of the window as the filter response at that point. The edge along a particular sliding direction is marked at the point where this difference has the maximum absolute value. The filters that are of most interest to us are those that estimate a third-order statistic. We list some of them here. A large number of these filters assume that the data have been subject to some nonlinear transformation [11] and try to detect high-order gradients taking into consideration the assumed transformation. If such a transformation is known a priori, the right filter from the list that follows may be picked up for use. Otherwise, one may try several of the filters and look for some sort of consensus result. In all the formulae that follow, is the value of a particular is the sample , is the mean value of all the samples, and total number of samples over which the statistic is computed. 1) Third moment

(4)

3022

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 10, OCTOBER 2006

Fig. 2. Histograms of random numbers drawn from a Pearson distribution superimposed on the theoretical frequency curves used to draw them.

2) Third

moment

where

is some parameter, and

(5) where

is some constant, and

is given by

(12)

is given by 6) Skewness (6) (13)

3) Third geometric moment

7)

skewness

(7) where

is given by

(14) 8) Geometric skewness

(15)

(8) 9) Harmonic skewness

4) Third harmonic moment

(16)

(9) 10) Contraharmonic where

skewness

is given by (17) (10)

5) Third contraharmonic

moment

(11)

For all of the above filters, one may consider their trimmed versions, by computing them using only the data in the range , where is the standard deviation of all samples. However, for (5), (7), (9), (12), and (14)–(17), one has the option to define the trimmed versions by using the corresponding standard deviation of the transformed data, defined in the Appendix. We also used the full distribution of gray values to characterise the data inside each half of the sliding window. In order

PETROU et al.: THREE-DIMENSIONAL NONLINEAR INVISIBLE BOUNDARY DETECTION

3023

TABLE I RESULTS OF AVERAGE AND STANDARD DEVIATION OF THE DISTRIBUTION OF PERCENTAGE ERRORS IN THE COMPUTATION OF THE THIRD AND FOURTH MOMENTS OF SETS OF POINTS DRAWN FROM A PEARSON DISTRIBUTION WITH (FROM LEFT TO RIGHT): TYPICAL VALUES ENCOUNTERED IN THE REAL DATA WE USE IN SECTION V-A,  = 106:7, = 7:5, AND p = 1:778. VALUES WHICH CORRESPOND TO THE MINIMUM SKEWNESS OBSERVED IN THE REAL DATA  = 62:86, = 1:75, AND p = 32:65 AND VALUES WHICH CORRESPOND TO THE MAXIMUM SKEWNESS OBSERVED IN THE REAL DATA  = 113:1, = 14:5, AND p = 0:476. THE CORRECT VALUES OF AND ARE: 1.5 AND 3.375, 0.35 AND 0.188, 2.9 AND 12.62, RESPECTIVELY.  AND  ARE THE MEAN AND STANDARD DEVIATION, RESPECTIVELY, OF THE RELATIVE ABSOLUTE ERROR OF ESTIMATING (i = 1; 2). “NUM PTS” IS THE TOTAL NUMBER OF POINTS USED FOR EACH EXPERIMENT. THE NUMBERS PRESENTED ARE STATISTICS COMPUTED OVER 100 REPETITIONS OF EACH EXPERIMENT USING DIFFERENT SEEDS FOR THE RANDOM NUMBER GENERATOR

to compare two full distributions, we use the norm, as well as the Kullback–Leibler divergence, defined as

the two halves of the window, along each scanning ray. If is the true position of the boundary along a scanning ray and is the position estimated by the boundary detector, then the root-mean-square error of the detector is

(18) and are the normalised gray level histograms where in the two halves of the scanning window. So, in all, 26 filters were used for assessing the higher order gradients in the data. In experiments not reported here, 25 more filters measuring fourth-order statistics, defined in an analogous way to the 25 filters listed above, were also investigated. However, they were not found to produce reliable results and so they were discarded from further investigation. IV. EXPERIMENTATION WITH SIMULATED DATA An artificial volume of data was created with size (60 60 60). Inside this volume, a sphere of radius 10 was placed, with its centre at position (30, 30, 30). The gray values of the voxels of the sphere and the background were drawn from two Pearson distributions with different parameters, so that they had the same mean and standard deviation, but different higher order moments. The boundary of the sphere had to be detected using scanning rays starting at the centre of the sphere and ending at distance 19 from the centre. For each experiment 72 rays were used placed with at orientations with polar angle in the range steps of 30 , and with azimuth angle in the range with steps also of 30 . Each half of the dipole filter used was 5 5 5 voxels in size. This implies that we were computing the third-order statistic with 125 points, which according to the results in Table I had an expected typical accuracy of about 20%. Such a choice was necessitated by the resolution of the data we use and constitutes a compromise between accuracy of the statistical computation and spatial resolution. The boundary detector marks the position of the boundary at the place of maximum difference in the values of the function computed inside

(19) where is the number of rays used. Fig. 3 shows some typical output signals along ray for each one of the third-order filters applied to the , , , simulated data, with and , where refers to the background and to the sphere. Fig. 4 shows the sum of the responses of all filters shown in 3. The vertical line in both figures signifies the true location of the boundary. Obviously, skewness is the optimal filter for these data as they have not been subjected to any nonlinear transformation. However, if we did not know that, we could have applied all filters. Fig. 3 demonstrates that some of them, even being suboptimal, would have responded correctly, and summing all outputs to reduce noise as shown in Fig. 4, would have produced a peak at the right place, even though not very pronounced due to the inclusion of several inappropriate filters. Obviously, if some prior information were available concerning filter compatibilities and performance, one might use a weighted sum of the filter responses to reduce or eliminate the effect of the inappropriate filters. For example, skewness, [see (13)], and trimmed contraappear to be incompatharmonic, [see (11)], with variance ible (see later discussion) and if one uses one of these filters, one may not include the other. In other words, filters may be grouped in sets of filters with compatible behaviour, and only filters from the same set may be used together. The response of the filters to various signal strengths was studied by considering various combinations of sphere and background distributions given in Table II.

3024

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 10, OCTOBER 2006

Fig. 3. Typical output signals along ray ( = 0;  = 0) for each of the third-order statistics computed for the case when the two distributions differ in the thirdand fourth-order statistic (with = 1, = 2).

TABLE II PARAMETERS OF THE SIMULATED EXPERIMENTS

In all cases, the signal strength was calculated as

(20)

Figs. 5–7 show the root-mean-square error of the performance of all filters when applied to the simulated data described earlier. The horizontal black line is the value of the we obtained when the location of the boundary was chosen at random from a uniform distribution in the range [1, 19]. Filters which perform worse than pure chance are the filters which were trimmed by using the standard deviation computed after applying the nonlinear transformation to the data. For example, the trimmed contraharmonic filter (see [11]), with transformed is one of them. input values in the range The same filter with original values in the range is one of the most reliable filters. This may be understood as follows: Trimming a filter is a process which allows us to deal with outliers. After the nonlinear transformation has been applied to the data, the values which happen to be away from the new (transformed) mean are not necessarily outliers. In fact

PETROU et al.: THREE-DIMENSIONAL NONLINEAR INVISIBLE BOUNDARY DETECTION

3025

Fig. 4. Sum of the responses of all the third-order filters in Fig. 3.

Fig. 5. Root-mean-square error of the untrimmed filters as a function of the signal strength (SS).

Fig. 6. Root-mean-square error of the trimmed (before the transformation) filters as a function of the signal strength (SS).

in the case of the contraharmonic filter, the outliers become inliers, and vice versa. Trimming, therefore, according to the variance of the transformed values, does not reduce the effect of outliers, but it rather reduces the number of inliers and destroys the true signal. This, however, is only because we constructed the simulated data so that they differ in the skewness. It is obvious that if we had constructed the simulated data so that they

differed in the contraharmonic skewness, the best filter would have been the one that computes the contraharmonic skewness. of the optimal filter for this Fig. 8 presents the type of data (namely the skewness filter), the sum of all filters, as well as the error of the filter that compares the full normalised histograms constructed with 32 bins each, either by using the norm, or the Kullback–Leibler divergence. We observe that

3026

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 10, OCTOBER 2006

Fig. 7. Root-mean-square error of the trimmed (after the transformation) filters and the sum of the all the filters, the sum of the best eight and the best nine filters as a function of the signal strength (SS).

Fig. 8. Root-mean-square error of the filter that compares the full histograms using the L norm or the Kullback–Leibler divergence, the optimal filter for this type of data (i.e., the filter that computes the skewness) and the sum of the all filters, as a function of the signal strength (SS).

for high signal values, direct comparison of the histograms performs well. However, for maximum sensitivity to low signal values, one must use the optimal filter, which for the case of these data is the one that looks for changes in skewness. The encouraging thing is that if we do not know which the optimal filter is for our data, and we simply apply all filters along the scanning line and sum their responses, the performance of the optimal filter does not deteriorate much, as shown by the curve that corresponds to the sum of all filters. V. EXPERIMENTS WITH REAL DATA If we knew the nonlinear transformation which the real data had possibly been subjected to, we would have been able to choose the most appropriate filter for estimating changes in the third moment. In the absence of any such information, we choose to use the skewness filter, as it is the simplest to compute. There are two modes of operation that the methodology described in Sections III and IV may be used for a model driven approach, where prior information is available for the existence

of a boundary, and a “blind” approach, where we have no prior knowledge about the existence of such boundaries and we are simply trying to assess whether any exist. We present here experiments for both cases. All images used are 8-bit images. The images used in Section V-B were 256 256 38 in size, while the images used in Section V-A were of different sizes for different patients, varying from 512 512 20 up to 512 512 50. A. Prior Knowledge is Available: Invisible Tumour Boundary Detection For this experiment we use T1-weighted and T2-weighted 3-D MRI data concerning six patients with glioblastomas multiforme, before they undergo any surgery. In all cases radiologists marked the visible boundary of the tumour, and an outer boundary inside which the true tumour boundary was suspected to be. Fig. 9 shows for two patients the normalised histograms of the gray values of the voxels around the visible part of the tumour, fitted with Pearson distributions. It is evident from all cases that these histograms are not symmetric or Gaussian-like.

PETROU et al.: THREE-DIMENSIONAL NONLINEAR INVISIBLE BOUNDARY DETECTION

3027

Fig. 9. Histograms of the gray values in (left) T1-weighted and (right) T2-weighted data around the tumour of two different patients, with Pearsons distributions fitted to them. The value of the skewness in each case is (a) 1.48, (b) 0.90, (c) 1.49, and (d) 0.88.

They clearly indicate a probability density function which exhibits nonzero high-order moments. It is, therefore, plausible, although not certain, that a hidden boundary may manifest itself as a third moment gradient. Fig. 10 shows some slices of these data with superimposed the points identified as possible tumour boundaries. It must be emphasised that there is no ground truth against which these results may be compared. In addition, these boundaries are forced, in the sense that we scan the data along rays emanating from outside the visible tumour boundary until we reach the outer boundary marked by the radiologist. We then force the system to choose the position where the third-order gradient is locally maximal between the two boundaries of the tumour. Thus, we identify a possible boundary point in all cases. The only way to check whether what we find corresponds to a physical boundary is to check for consistency between the boundary identified in the T1-weighted image and that in the T2-weighted image. For only two of the patients these two data sets had been collected at the same session, and the examples shown in Fig. 10 concern these two patients. Fig. 11 shows some typical profiles of the filtering process along four of the rays for these two patients. One can clearly see that in spite of their noisy nature, the two curves in each panel show remarkable correlation of the positions of their peaks. This is an indication that these peaks probably occur at places where some physical boundaries exist. In each panel, we show the correlation coefficient of the two curves. We can see that this is low even for cases where we can clearly see the presence of very similar structure in terms of local peaks. This is because the correlation coefficient takes into consideration the

actual values of each curve, while we are actually interested in the local peaks of the values. If we identify the local peaks in each sequence, we may define two measures of coincidence of the local peaks: The fraction of peaks in T1 which coincide with sample location in T2, and the fraction of peaks in T2 a peak which coincide with a peak sample location in T1. We call these measures Coincidence_T1_T2 and Coincidence_T2_T1. The histograms of the values of these two measures over all rays we used for these two patients are shown in Fig. 12. We can see that we have quite high values of coincidence of the peaks in the paired sequences. B. Prior Knowledge is Not Available: Structural Differences in the Brains of Schizophrenics and Normal Controls In this case, we have no prior reason to believe that a boundary is present in the data. Instead, we are searching for third-order invisible gradients. If we do not have prior information about the direction of the possible edge, we have to scan the data in three orthogonal directions to infer the three components of the gradient vector at each location. To ensure that all voxels used to compute the gradient vector associated with a voxel are within the same limiting distance from it, we use for these experiments a spherical scanning window. The scheme we use is schematically shown in Fig. 13. At each location, we split the window in three different ways into two halves using the three orthogonal planes defined by the co-ordinate axes, and we compute the statistic we are interested in each half. We compute the difference of the values of the statistic in two adjacent hemispheres in order to identify the corresponding component of

3028

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 10, OCTOBER 2006

The method was applied to data concerning 21 schizophrenic patients and 19 normal controls. These data have been used for previous studies reported elsewhere [13], [6]. In all cases, the images were preprocessed so that only the brain parenchyma was extracted for further analysis [14]. Fig. 14 shows some typical results concerning the two groups of subjects when the T2-weighted data were used. It is evident from these figures that the regions over which high third-order gradients are observed are much more extensive in normals than in schizophrenics. Fig. 15 shows the statistical significance of the number of voxels with such high third-order gradients as a feature characterising the two groups of subjects. The –test in this case was performed on the fraction of the total brain volume occupied by voxels with skewness gradients above the threshold. On average, this percentage was 2.85% in schizophrenics and 3.73% in the normals. The value of the test . There was no statistically significant was 3.34 with difference in the observed strengths of the skewness gradients between the two groups. The mean value of the skewness gradient in the schizophrenics was 1.69 and in the controls 1.68. No difference between the two groups was identified when the proton density data were used for a similar analysis. VI. DISCUSSION AND CONCLUSIONS

Fig. 10. Examples of detected tumour boundaries (indicated by the white dots) in the T1-weighted (left) and T2-weighted (right) scans of two different patients. The closed curves indicate the limits within which the radiologist considered that the true boundary of the tumour is. Lines 1 and 2 in the top two panels show the rays to which the profiles shown in Fig. 11(a) and (c) correspond, respectively.

the gradient vector. At the end, the magnitude of the gradient vector is computed for each voxel. As we are interested in gradients not visible to the human eye, we compute also gradients in the mean and the standard deviation at each location, and flag only the voxels where the mean and standard deviation gradient is below a certain threshold, while the third-order gradient is above another threshold. Finally, we count the number of voxels which satisfy these conditions, and produce a single number per subject. The radius of the spherical window we chose to use was five voxels. This implies that we perform the calculations using approximately 500 points, which according to Table I implies a typical accuracy of about 12%. The low threshold used to discard gradients in the first- and second-order moments was 3% of the maximum value, while the threshold we used to identify voxels with significant third-order gradients was 10% the maximum value. The choice of these thresholds was based on our experience from other studies in dealing with such data, where the low threshold is roughly chosen to discard approximately the weakest 2–3% of the values, and the second threshold is chosen 2–3 times higher to allow a margin of fuzziness between the two thresholds. We also experimented with windows of other sizes. The results did not change noticeably for window radii 4–7.

We demonstrated here that the availability of 3-D data allows the use of high-order gray level statistics to make explicit gradients in the data invisible to the human eye. First of all, we showed that the gray level values in MRI data are not Gaussianly distributed and they clearly show the presence of third-order moments. Gradients in these moments are not easy to compute because of the high number of samples required for their reliable estimation. We showed that for 10% accuracy in their estimation, one requires local windows of size at least 9 9 9. If the resolution of the data is about 1 mm in each direction, such . This is now within a window refers to volumes of about 1 the limits of clinical practice. There are even recent reports in the literature of data with resolution of 0.125 mm [17]–[19]. In order to understand the significance of such gradients in the data, we have to think in terms of the significance of the recorded gray values. In MRI, image intensity is a multiparameter function of the spin density , relaxation times , , , diffusion coefficient , and so on. Therefore, image con, which is defined in terms of differences in image trast intensity, may be expressed as (21) where the functional form depends on the exact data acquisition protocol. If the data acquisition parameters are chosen—as in the present case—so that the effect is dominant, then (22) and relaxation times of water or lipid protons Now, vary between different tissues and pathologies, making them major factors in determining contrast within the MR image. Since many disease states are characterised by a change of the

PETROU et al.: THREE-DIMENSIONAL NONLINEAR INVISIBLE BOUNDARY DETECTION

3029

Fig. 11. Typical response profiles along scanning rays of the difference of skewness filter when T2-weighted (dashed lines) and T1-weighted (continuous line) modalities acquired at the same time are used. (a)–(d) one patient and (e)–(h) another patient. The numbers in each panel are the correlation coefficients for the pairs of plotted curves.

Fig. 12. Histograms of the fraction of times (left) peaks in T1 coincide with a peak in T2 and (right) peaks in T2 coincide with a peak in T1 along scanning rays.

tissue value, T2-weighted imaging is a sensitive method for disease detection. One may argue that the use of a sliding window inside which statistics of the data are computed may be affected by the presence of mixed tissue inside the window. This is a classical problem in image processing. However, it is expected that it will affect all statistics computed from the data, not just the skewness. Further, in the case of schizophrenics, for whom it is known that they have smoother sulci surfaces than normals, one would expect to find lower gradients than in the normals where the sulci surfaces are more ragged and, thus, more likely

for one half of the window to contain bimodal distribution than the other, something which may cause excessive skewness gradients. However, we do not observe excessive skewness gradients in the normals in the region of the sulci (gray matter) but in the region of the white matter. Perhaps this is an indication that the over smoothness observed in the sulci regions of the schizophrenics is a much more widespread property of the structure of the schizophrenic brain than originally thought, dominating the structure of the tissue itself. Interestingly, in both examples of pathology presented here, the higher frequency of high values of skewness in the

3030

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 10, OCTOBER 2006

Fig. 13. Schematic representation of the procedure for calculating skewness gradients distinct from gradients perceivable by the human eye.

T2-weighted data was associated with the healthy tissue. In fact in the case of glioblastomas, when we considered all cases of skewness gradient we detected, the higher skewness value was associated with the interior of the tumour with a –value and . The two sets of compared distributions had standard deviations 1.220 and 1.133, justifying the use of the –statistic. The opposite was observed in the cases of skewness gradients detected in the T1-weighted data of the glioblastomas: The skewness was higher outside the detected boundary, with a –value 15.16495 and a value again . The two sets of compared distributions had standard deviations 1.449 and 1.390 this time, again justifying the use of the –statistic. Certainly, more work needs to be done to validate the detection of “invisible boundaries” in medical applications. With respect to brain tumours one interesting approach in the future may be to combine spectroscopic imaging with conventional MRI and to correlate the different borders seen with each method using the proposed statistical approach. Changes in the skewness of the distribution of gray values in the case of glioblastomas have been reported in [3] where the authors reported that although the mean of the intensity distribution was insensitive to tumour grade, measurements of skewness of the pixel distribution of blood recirculation images demonstrated a clear distinction between grade III and grade IV tumours. With respect to our investigation on schizophrenic patients, there have been reports on in vivo diffusion brain imaging studies which revealed microstructural abnormalities with lower diffusion anisotropy [15], [7]. Since conditions that increase self diffusion, such as oedema, may also alter the longitudinal and transverse relaxation time of protons, it is

possible that such changes could explain the observed diffusion anisotropy diminution seen in schizophrenia [10]. Pfefferbaum et al. [10] measured transverse relaxation time ( ) and proton density (PD) maps for gray matter and white matter in ten control men and ten men with schizophrenia and found that schizophrenics had significantly longer mean white matter (84.0 versus 81.9 ms, ) and gray matter T2 (95.1 versus 92.2 ms, ) whereas their mean white and gray matter PD values were not significantly different from those of controls. These findings are in accordance to our results that the T2-weighted data were more skewed in the controls and that no difference between the two groups was identified when the proton density weighted data were used. We further wish to remark that the nonzero third-order gradients were observed at the white matter of the brain parenchyma. These regions, which appear to be different in normals and schizophrenics, are different from the regions identified as structurally distinct between these two groups in previous studies which were mainly associated with the gray matter [4]. However, one should be careful in excluding the presence of third-order gradients in the gray matter: Gray matter contains several visible to the human eye gradients. Our analysis deliberately excluded the inclusion of those regions. In addition, the use of a filter of radius 5 automatically excluded from the analysis a layer of tissue of the same thickness all around the external boundary of the brain parenchyma. Finally, a couple of points on the methodology have to be re-emphasised. In Section IV, the study with the simulated data showed that the best filter was the one computing the skewness of the data. This was by no means unexpected: the data had been constructed to contain a gradient in skewness. In a real

PETROU et al.: THREE-DIMENSIONAL NONLINEAR INVISIBLE BOUNDARY DETECTION

3031

Fig. 14. Some slices of schizophrenic and normal subjects with the regions of skewness gradients highlighted.

use of both T1-weighted and T2-weighted data simultaneously, using coincidence measures of their peaks along the same scanning line, as well as information about peaks in neighbouring lines and perhaps some criteria of lateral continuity. APPENDIX The variances that may be used with the corresponding thirdorder filters to produce trimmed versions from them are (23)

(24) (25)

Fig. 15. Statistical significance of the extent of the skewness gradients in the schizophrenics and normals.

application, we do not know whether the data have been subject to some prior nonlinear transformation, and which. The most important point demonstrated in Section IV was that the sum of all filters had a comparative performance with that of the optimal filter, namely the skewness filter. So, in the absence of any knowledge about the real data, it is recommended that all filters are used simultaneously, and the sum of their responses is used as the response along the scanning ray. The responses along the scanning rays shown in Fig. 11 show clear correlation between the T1-weighted and T2-weighted profiles. However, when we pick the largest value in each profile, this correlation may be lost, and phenomenologically the two modalities may appear to identify totally different boundaries. This point is demonstrated in panel (a) of this figure. So, a more sophisticated approach in handling the profiles along the scanning rays has to be developed, perhaps one that makes

(26)

ACKNOWLEDGMENT The authors would like to thank Dr. S. Pallotta of the Medical School of the University of Florence, Italy, for supplying the glioblastoma data used in the reported experiments, as well Dr. J. Suckling of the Maudsley Hospital in London for supplying the schizophrenia data. REFERENCES [1] M. Abramowitz and I. A. Stegun, Handbook of Mathematical Functions with Formulas, Graphs and Mathematical Tables. New York: Dover, 1965.

3032

[2] J. Graham and C. J. Taylor, “Boundary cue operators for model based image processing,” in Proc. 4th Alvey Vision Conf., 1988, pp. 59–64. [3] A. Jackson, A. Kassner, D. Annesley-Williams, H. Reid, X.-P. Zhu, and K.-L. Li, “Abnormalities in the recirculation phase of contrast agent bolus passage in cerebral glioblastomas: Comparison with relative blood volume and tumor grade,” AJNR Amer. J. Neuroradiol., vol. 23, pp. 7–14, 2002. [4] B. Julesz, E. N. Gilbert, L. A. Shepp, and H. L. Frisch, “Inability of humans to discriminate between visual textures that agree in secondorder statistics-revisited,” Perception, vol. 2, pp. 391–405, 1973. [5] B. Julesz, “Textons, the elements of texture perception, and their interaction,” Nature, vol. 290, pp. 91–97, 1981. [6] V. Kovalev, M. Petrou, and J. Suckling, “Detection of structural differences between the brains of schizophrenic patients and controls,” Psych. Res.: Neuroimag., vol. 124, pp. 177–189, 2003. [7] M. Kubicki, C. F. Westin, P. G. Nestor, C. G. Wible, M. Frumin, S. E. Maier, R. Kikinis, F. A. Jolesz, R. W. McCarley, and M. E. Shenton, “Cingulate fasciculus integrity disruption in schizophrenia: A magnetic resonance diffusion tensor imaging study,” Biol Psych., vol. 54, pp. 1171–80, 2003. [8] N. Paragios and R. Deriche, “Coupled geodesic active regions for image segmentation: A level set approach,” in Proc. Eur. Conf. Computer Vision, 2001, pp. 224–240. [9] M. Petrou, “The differentiating filter approach to edge detection,” Adv. Electron. Electron. Phys., vol. 88, pp. 297–345, 1994. [10] A. Pfefferbaum, E. V. Sullivan, M. Hedehus, M. Moseley, and K. O. Lim, “Brain gray and white matter transverse relaxation time in schizophrenia,” Psych. Res., vol. 91, pp. 93–100, 1999. [11] I. Pitas and A. N. Venetsanopoulos, “Edge detectors based on nonlinear filters,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-8, no. 4, pp. 538–550, Jul. 1986. [12] W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C: The Art of Scientific Computing. Cambridge, U.K.: Univ. Cambridge Press, 1988. [13] T. Sigmundsson, J. Suckling, M. Maier, S. Williams, E. T. Bullmore, K. E. Greenwood, R. Fukuda, M. A. Ro, and B. K. Toone, “Structural abnormalities in frontal, temporal and limbic regions and interconnecting white matter tracts in schizophrenic patients with prominent negative symptoms,” Amer. J. Psychol., vol. 158, pp. 234–243, 2001. [14] J. Suckling, M. J. Brammer, A. Lingford-Hughes, and E. T. Bullmore, “Removal of extracerebral tissues in dual-echo magnetic resonance images via linear scale-space features,” Magn. Reson. Imag., vol. 17, pp. 247–256, 1999. [15] Z. Sun, F. Wang, L. Cui, J. Breeze, X Du, X Wang, Z. Cong, H. Zhang, B. Li, N. Hong, and D. Zhang, “Abnormal anterior cingulum in patients with schizophrenia: A diffusion tensor imaging study,” Neuroreport, vol. 14, pp. 1833–1836, 2003. [16] M. Tuceryan, “Moment based texture segmentation,” Pattern Recognit. Lett., vol. 15, pp. 659–668, 1994. [17] X Zhang and A. Webb, “Design of a capacitively decoupled transmit/ receive NMR phased array for high field microscopy at 14.1 T,” J. Magn. Res., vol. 170, pp. 149–155, 2004. [18] [Online]. Available: http://www.bruker-biospin.de/MRI/applications/ med0.html [19] [Online]. Available: http://www.bruker-biospin.de/MRI/applications/ biov12.html

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 10, OCTOBER 2006

Maria Petrou (SM’05) studied physics at the Aristotle University of Thessaloniki, Thessaloniki, Greece, applied mathematics at Cambridge University, Cambridge, U.K., and she received the Ph.D. degree from the Institute of Astronomy, Cambridge University. She has been working on image processing and computer vision since 1986. She was Professor of image analysis at the University of Surrey, Guildford, U.K., from 1998 to 2005. She is currently the Professor of signal processing at Imperial College, London, U.K. She has published more than 300 scientific papers on astronomy, remote sensing, computer vision, machine learning, color analysis, industrial inspection, and medical signal and image processing. She has coauthored two books Image Processing: The Fundamentals (Wiley, 1999) and Image Processing: Dealing With Texture (Wiley, 2006). A full list of publications and other details can be found at http://www.commsp.ee.ic.ac.uk/mcpetrou. Dr. Petrou is a Fellow of the Royal Academy of Engineering and a Fellow of IAPR. She has served as the Chairman of the Technical Committee for Remote Sensing of IAPR, the Chairman of the British Machine Vision Association (BMVA), as an Associate Editor of IEEE TRANSACTIONS ON IMAGE PROCESSING, and as the Newsletter Editor and Treasurer of IAPR.

Vassili A. Kovalev received the degree in applied mathematics in 1978 and the Ph.D. degree in engineering cybernetics in 1984, both from the Tomsk Polytechnic University, Russia. He was a Senior Lecturer at the Tomsk Polytechnic University, Head of the Image Analysis laboratory of the Institute of Mathematics, and later a habilitation practitioner at the United Institute of Informatics Problems, National Academy of Sciences of Belarus. Currently, he is a Research Fellow at the University of Surrey, Guildford, U.K. He has been working in the field of image analysis since 1989 and has published about 100 papers in the areas of medical imaging, texture feature extraction, color image processing, image retrieval, and object recognition. His current research interests are in 3-D volumetric image analysis, microscopy image analysis, and color image processing with applications in neurology, cancer diagnosis, and color conversion for color-blind people in multimedia systems.

Jürgen R. Reichenbach received the M.S. and Ph.D. degrees in physics from the University of Karlsruhe, Karlsruhe, Germany, in 1988 and 1992, respectively. He was a Postdoctoral Fellow at the University of Montpellier, France, in 1993, and at the Heinrich-Heine University Dusseldorf, Germany, in 1994. Following two years as a Visiting Associate Scientist at the Mallinckrodt Institute of Radiology, St. Louis, MO, he joined the Friedrich-Schiller University, Jena, Germany. He heads a research team working on the development of new measurement techniques in the field of magnetic resonance imaging.