Using Belief Function Theory to Deal with Uncertainties and ...

Using Belief Function Theory to Deal with Uncertainties and Imprecisions in Image Processing Benoˆıt Lelandais, Isabelle Gardin, Laurent Mouchard, Pierre Vera, and Su Ruan

Abstract. In imaging, physical phenomena and acquisition system often induce an alteration of the information. It results in the presence of noise and partial volume effect corresponding respectively to uncertainties and imprecisions. To cope with these different imperfections, we propose a method based on information fusion using Belief function theory. First, it takes advantage of neighborhood information and combination rules on mono-modal images in order to reduce uncertainties due to noise while considering imprecisions due to partial volume effect on disjunctions. Imprecisions are then reduced using information coming from multi-modal images. Results obtained on simulated images using various signal to noise ratio and medical images show its ability to segment multi-modal images having both noise and partial volume effect.

1 Introduction In imaging, two distinct problems lead to ambiguities from a spatial point of view: uncertain information due to noise and imprecise information due to lack of knowledge at the transition between areas. At this transition, the information carried by voxels is more ambiguous than the one suffering from noise. Both uncertainties and imprecisions have a negative effect on image processing. Benoˆıt Lelandais ⋅ Laurent Mouchard ⋅ Su Ruan University of Rouen, LITIS EA 4108 - QuantIF, 22 bd Gambetta, 76183 Rouen, France e-mail: [email protected], [email protected],[email protected] Isabelle Gardin ⋅ Pierre Vera Centre Henri-Becquerel, Department of nuclear medicine, 1 rue d’Amiens, 76038 Rouen, France & University of Rouen, LITIS EA 4108 - QuantIF, 22 bd Gambetta, 76183 Rouen, France e-mail: [email protected], [email protected] T. Denœux & M.-H. Masson (Eds.): Belief Functions: Theory & Appl., AISC 164, pp. 197–204. springerlink.com © Springer-Verlag Berlin Heidelberg 2012

198

B. Lelandais et al.

Belief function theory (BFT) [1, 2, 3] is particularly well suited to represent information from partial and unreliable knowledge. In [4, 5], authors propose to use BFT to reduce uncertainties and imprecisions using conjunctive combination of neighboring voxels. On one hand, it allows to reduce noise and on the other hand, to highlight conflicting areas mainly present at the transition between areas where PVE occurs due to the fact that information is extremely ambiguous in a spatial context. Therefore, results obtained by these authors allow to represent both segmented regions and contours. BFT has the advantage to manipulate not only singletons but also disjunctions. This gives the ability of explicitly representing both uncertainties and imprecisions. One of the difficulties resides in the modeling of disjunctions, while they make it possible to take into consideration the lack of knowledge. In [6], author proposes to use fuzzy morphological operators to transfer for each voxel a part of belief on disjunctions according to its neighborhood. This method is interesting, but considers uncertainties and imprecisions in the same way. By using BFT, our aim is two-fold: first, we reduce uncertainties due to noise, then imprecisions due to Partial Volume Effect (PVE) which corresponds to the lack of knowledge at the transition between areas. At first, our method operates a disjunctive combination followed by a conjunctive combination of neighboring information on mono-modal images. The disjunctive combination allows to transfer both uncertain and imprecise informations on disjunctions. Then, the conjunctive combination is applied to reduce uncertainties due to noise while maintaining representation of imprecise information at the boundaries between areas on disjunctions. In order to remove some imprecise informations, a multi-modal image fusion is also proposed. We take benefit from the complementarity of images to reduce imprecisions. The method is used for the fusion of multi-modal PET (Positron Emission Tomography) medical images of the same patient using three radiotracers which give respectively information on tumor glucose metabolism, cell proliferation, and hypoxia (inadequate supply of oxygen). These images are of major interest for the treatment of lung cancer by radiotherapy, but need a relevant treatment considering both their important noise and partial volume effect. First, we present our method, based on the fusion in mono-modal images, followed by the multi-modal fusion of informations. Then, the validation of the method is done on simulated data. Finally, the method is applied on multi-modal PET images.

2 Information Fusion Using Belief Function Theory for Reducing Uncertainties and Imprecisions 2.1 Fusion for Reducing Uncertainties While Considering Imprecision on Mono-Modal Images Partial knowledge as uncertainties and imprecisions are taken into account by assigning Basic Belief Assignments (BBA) m over different subsets of the considered

Using BFT to Deal with Uncertainties and Imprecisions in Image Processing

199

frame of discernment Ω = {ω1 , ω2 ,... , ωC }. m is defined as a mapping from 2Ω to [0,1] verifying ∑A⊆Ω m(A) = 1. From initial BBA, to reduce uncertainties while considering imprecisions, we propose to take benefit from both neighborhood information and combination operators [4, 5]. The way in which the neighborhood contribution has been carried out is as follow: let Φ (Vi ) be a set of P voxels Vk (with k ∈ {1,...,P}), surrounding a voxel Vi , and including Vi . Because of the different distances separating Vk and Vi , we propose to associate for each voxel in Φ (Vi ) a coefficient αk that depends on the distance separating it from Vi . It is computed by: αk = exp((Vk −Vi )2 /σ 2 ), with √ FWHM = 2 2 log2σ the Full Weight at Half Maximum corresponding to the spatial resolution of our images. 2.1.1

Disjunctive Combination of Neighboring Voxels

First, the influence of a voxel Vk from Φ (Vi ) is weighted by the coefficient αk . The BBA on A ≠ ∅ and on ∅ can be calculated using the expressions: ∀A ≠ ∅ mV′ k (A) = αk mVk (A), mV′ k (∅) = 1 − αk + αk mVk (∅)

(1)

Thus, the further away from Vi the voxel Vk is, the lower its contribution to the computation will be. The transfer to the empty set is interpreted as a non-commitment toward all the other hypotheses, and allows, before applying a disjunctive combination, to reduce the influence of mVk (A) proportionally to αk . The Fuzzy C-Means algorithm (FCM) [7] is used in order to initialize mVk (A). After this step, and in order to transfert uncertain and imprecise data to disjunctions, the following disjunctive combination is performed for each voxel: MVi (.) =

∪ mV′ k (.) ◯

(2)

Vk ∈Φ (Vi )

It follows that nonzero masses are assigned to disjunctions. Higher they are, more different the informations carried by the neighboring voxels are. It is especially true on the edges between areas where PVE occurs and for voxels located in a very noisy environment. This operator, usually used when at least one source is reliable, can be used at that time. It is reasonable to assume that at least one of the voxels in the neighborhood gives a reliable information. After the disjunctive combination, we can assume that all sources become reliable since operator acts as a discounting of spatially ambiguous sources. 2.1.2

Estimation Using Disjunctive Combination in FCM

Disjunctive step allows to transfert, from initial BBA computed using FCM, uncertain and imprecise information on disjunctions. We also propose to integrate our disjunctive combination of neighboring voxels inside FCM algorithm. This process

200

B. Lelandais et al.

is used for updating centroids and computing membership degrees with adapted data. After the fusion, the obtained data is less ambiguous. 2.1.3

Conjunctive Combination of Neighboring Voxels

To reduce uncertainties without impacting the ambiguities brought by the imprecisions, the opposite operation has been proposed. It consists in the conjunctive combination of neighboring voxels Vk discounted. First, the discounting is done according to the coefficient αk by transferring a part of belief on the set Ω : MV′′k (A) = αk MVk (A), ∀A ≠ Ω MV′′k (Ω ) = 1 − αk + αk MVk (Ω )

(3)

The discount process allows to reduce the influence of voxels which are far from Vi before doing the conjunctive combination using Dempster’s rule given by: MVi (.) =

⊕

Vk ∈Φ (Vi )

MV′′k (.)

(4)

Since all sources are reliable (thanks to disjunctive combination), it is appropriate to use this operator. This step allows to remove ambiguities due to noise, by transferring their belief on the singletons, while the voxels at the boundaries between areas remain represented on disjunctions.

2.2 Fusion for Reducing Imprecisions Using Multi-modal Informations Having different information coming from other sources, we propose to take benefit from this information in order to reduce imprecisions due to PVE. Information coming from mono-modal image does not allow to reduce imprecisions. If two sources of information are available, reducing the imprecision with BFT is possible by using the conjunctive rule of combination. Let m1 and m2 be two fully reliable BBA. Their fusion is defined as follow: ∩ 2 (A) = ∑ m1 (B)m2 (C) m1 m

(5)

B∩C=A

We propose an information fusion method based on conjunctive rule to deal with multi-modal images. Furthermore, we choose to use an external contextual knowledge to reduce imprecisions for our application on PET images (section 4).The external knowledge is learned from PET phantom images containing spheres whose volumes are known. It consists in applying the conjunctive combination of the current source with the external knowledge whose BBA has two focal elements: the subset A of Ω to reinforce and Ω .

Using BFT to Deal with Uncertainties and Imprecisions in Image Processing

201

3 Validation The efficiency of our method is evaluated and compared to the method proposed in [6] on simulated images by measuring the recognition rates (rates of pixels correctly labelled) according to several Signal to Noise Ratio (SNR) varying from 1.5 to 6. The simulated images (Fig. 1) consist in a square surrounding by a background (images with two classes). The simulated images are blurred with a Gaussian filter whose FWHM vary according to the SNR, and noised with a Gaussian filter whose standard deviation is inversely proportional to the SNR. Note that BBA presented on Fig. 1(a) and (b) correspond to a simulated image with SNR of 5. The proposed method is applied on the simulated images. At the end of the disjunctive combination of each pixel with its neighborhood, the belief masses are spread over the hypotheses {ω1 }, {ω2 } and {ω1 , ω2 } (Fig. 1(f), (g) and (h)). The belief of each pixel for which the information is ambiguous is mainly represented on the hypothesis {ω1 , ω2 }. The result, using the conjunctive combination of each pixel with its neighborhood is presented Fig. 1(i), (j) and (k). Inside areas, the method provides high beliefs in favor of {ω1 } and {ω2 }. The uncertainties due to noise are therefore reduced. Within the transitions between areas, the belief is mainly represented on the hypothesis {ω1 , ω2 }, highlighting the imprecision. For comparison, we present Fig. 1(c), (d) and (e) the result of the modeling using the method proposed in [6]. This method considers both noisy and fuzzy information

performance 1 (a)

{ω1 }

(b)

0.9

{ω2 }

0.8 (c)

{ω1 }

(d)

{ω2 }

(e)

Ω

(f)

{ω1 }

(g)

{ω2 }

(h)

Ω

(i)

{ω1 }

(j)

{ω2 }

(k)

Ω

0.7 0.6 1.5

2

2.5

3

3.5

4

4.5

5

5.5

6

SNR

(l)

Fig. 1 Results of the fusion of mono-modal simulated images. (a) and (b) are initial BBA. (c), (d) and (e) show the BBA with the method proposed in [6]. (f), (g) and (h) present the BBA after the disjunctive combination. (i), (j) and (k) show the BBA applying then the conjunctive combination (our method). On (l) are presented the recognition rates according to different SNR on simulated images using only FCM (black curve), both FCM and the method proposed in [6] (orange curve), the conjunctive rules (blue curve), the disjunctive and the conjunctive rules (green curve), and using disjunctive rule integrated in FCM followed by conjunctive rule (red curve).

202

B. Lelandais et al.

as imprecision. Note that the conjunctive combination of neighboring pixels is an important step of our method to reduce noise and transfer fuzzy information on disjunctions. Fig. 1(l) presents the recognition rates using FCM, and applying our method, and the method proposed in [6] according to different Signal to Noise Ratio (SNR). To measure recognition rates for each SNR, the methods are here applied on two images having the same SNR, and results are then fused using the Dempster’s rule in order to reduce both uncertainties and imprecisions. As we can see, our method gives better recognition rates than the method proposed in [6]. In addition, note that the conjunctive combination of neighboring pixels allows to improve the results. Moreover, when disjunctive rule is integrated in FCM, performances are lightly improved.

4 Application to Multi-modal PET Images for Functional Tumor Localization The proposed method is also applied for multi-modal fusion of PET functional medical images (Fig. 2) to localize the tumor. These images are obtained after injection of a tracer specific to a studied function. From tracer FDG, FLT and FMISO, three type of PET images are obtained for a patient. Their characteristics are respectively glucose metabolism, cell proliferation and hypoxia (inadequate supply of oxygen). The FDG provides a good definition of the tumor target volume, especially ganglionic [8]. The FLT has a better tumor specificity than FDG [9] and lets us to envisage increasing the frequency of radiation therapy sessions on hyper-proliferative lesions. Finally, FMISO defines hypoxic tumors for which an irradiation dose escalation can be envisaged to improve the treatment [10]. The three PET images allow the distinction of areas that can be represented by five singletons, namely healthy tissue {N} (Normal), those with an important glucose Metabolism {M}, an important cell Proliferation {P}, a significant hypoxia {H}, and tissues with a Full uptake {F}: where tissues need an increasing of both the radiation therapy frequency and the dose. For each image, estimation step as proposed has been applied in order to obtain degrees of belief over two hypotheses and their union as presented in Table 1.

(a) FDG

(b) FLT

(c) FMISO

Fig. 2 Transverse slices for one patient with lung cancer. (a) Glucose metabolism PET image, (b) cell proliferation PET image, (c) hypoxia PET image. The area of interest (tumor lesion) is located in the rectangle.

Using BFT to Deal with Uncertainties and Imprecisions in Image Processing

203

Table 1 Hypotheses considered for PET images in order to fuse them coherently. Image Low uptake High uptake Ω FDG {N} {M, P, H, F} {N, M, P, H, F} {N, M, H} {P, F} {N, M, P, H, F} FLT {H, F} {N, M, P, H, F} FMISO {N, M, P}

(a)

{M, P, H, F}

(b) Ω

(g) ∅

(h) {M}

(c) {P, F}

(i) {P}

(d) Ω

(j) {H}

(e) {H, F}

(f) Ω

(k) {F}

(l) max

Fig. 3 Images showing results of our method of fusion on multi-modal PET images. (a) to (f) show the BBA assigned to each voxel after the estimation step. (g) to (l) are the results applying the Dempster’s rule of combination. (g) correspond to the areas of conflict. (h) to (k) are the plausibility corresponding to our hypotheses of interest. Finally, (l) is the segmented image using the maximum of plausibility.

On PET images, partial volume effect depends both on the size of the high uptake area and the contrast between areas. It becomes also important to reinforce BBA of high uptake areas according to its contrast and its volume. We chose to apply a reinforcement with a knowledge depending on contrast and volume. A learning is first carried out on PET images for which the high uptake volume is known in order to determine the parameters of reinforcing. This step allows to reduce imprecisions on mono-modal PET images. Results obtained from multi-modal PET images (Fig. 2) are presented in Fig. 3. From Fig. 3(a) to (f) are presented BBA corresponding to high uptake tissues and imprecise information after applying our fuzzy clustering method on each image followed by the conjunctive combination of neighboring voxels. On one hand, we observe that noisy information is removed from areas corresponding to high uptake tissues. On the other hand, we can see that areas corresponding to partial volume effect and medium uptake are mainly assigned to the vacuous BBA. Fig. 3(g) to (k) present the result of the multi-modal PET image fusion using first the reinforcing and then the Dempster’s rule [1]. They present both the BBA corresponding to the conflict, and the plausibility corresponding to our hypotheses of interest: {M}, {P}, {H} and {F}. Note that the conflict corresponds to a high uptake in FLT and FMISO and a low uptake in FDG. Finally, Fig. 3(l) correspond to the segmented tumor using the maximum of plausibility. This image is of great interest for the radiotherapist in order to adapt dose deliverance according to the functional tumor tissues.

204

B. Lelandais et al.

5 Discussion-Conclusion Currently, in medical images, very few authors consider both spatial uncertainties and imprecisions in the information modeling with the BFT [6]. We propose to perform a fusion of neighboring information by a disjunctive combination followed by a conjunctive combination. This method allows to deal with both types of imperfection. In addition, we suggest to integrate the disjunctive combination in FCM in order to compute centroids only with certain and precise information. Finally, we propose to take benefit from prior knowledge in order to reduce imprecisions. As shown from the results on simulated and medical images, the interest of our method is that the uncertainties due to noise are largely removed, and that the imprecision at the boundaries between regions is taken into account in the modeling. Moreover, considering large amount of noise, our method outperform a simple FCM and the method proposed in [6]. The method is generic since it can be applied whatever the distribution of initial beliefs is. In future work, we will test our method on a larger database to assess the robustness of the method, and on other types of images to confirm its genericity.

References 1. Dempster, A.P.: Upper and lower probabilities induced by a multivalued mapping. Ann. Math. Stat. 38, 225–339 (1967) 2. Shafer, G.: A mathematical theory of evidence. Princeton University Press, Princeton (1976) 3. Smets, P., Kennes, R.: The Transferable Belief Model. Artif. Intell. 66, 191–234 (1994) 4. Capelle, A.S., Colot, O., Fernandez-Maloigne, C.: Evidential segmentation scheme of multi-echo MR images for the detection of brain tumors using neighborhood information. Inf. Fusion 5(3), 203–216 (2004) 5. Zhang, P., Gardin, I., Vannoorenberghe, P.: Information fusion using evidence theory for segmentation of medical images. In: Int. Colloq. on Inf. Fusion, vol. 1, pp. 265–272 (2007) 6. Bloch, I.: Defining belief functions using mathematical morphology - Application to image fusion under imprecision. Int. J of Approx. Reason. 48(2), 437–465 (2008) 7. Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press, New York (1981) 8. Gould, M.K., Kuschner, W.G., Rydzak, C.E., et al.: Test performance of positron emission tomography and computed tomography for mediastinal staging in patients with nonsmall-cell lung cancer: a meta-analysis. Ann. Int. Med. 139, 879–892 (2003) 9. Xu, B., Guan, Z., Liu, C., et al.: Can multimodality imaging using 18F-FDG/18F-FLT PET/CT benefit the diagnosis and management of patients with pulmonary lesions? Eur. J. Nucl. Med. Mol. Imaging 38(2), 285–292 (2011) 10. Choi, W., Lee, S.W., Park, S.H., et al.: Planning study for available dose of hypoxic tumor volume using fluorine-18-labeled fluoromisonidazole positron emission tomography for treatment of the head and neck cancer. Radiother. Oncol. 97(2), 176–182 (2010)