Brain CT Image Similarity Retrieval Method Based on ... - IEEE Xplore

574

IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, VOL. 18, NO. 2, MARCH 2014

Brain CT Image Similarity Retrieval Method Based on Uncertain Location Graph Haiwei Pan, Pengyuan Li, Qing Li, Senior Member, IEEE, Qilong Han, Xiaoning Feng, and Linlin Gao

Abstract—A number of brain computed tomography (CT) images stored in hospitals that contain valuable information should be shared to support computer-aided diagnosis systems. Finding the similar brain CT images from the brain CT image database can effectively help doctors diagnose based on the earlier cases. However, the similarity retrieval for brain CT images requires much higher accuracy than the general images. In this paper, a new model of uncertain location graph (ULG) is presented for brain CT image modeling and similarity retrieval. According to the characteristics of brain CT image, we propose a novel method to model brain CT image to ULG based on brain CT image texture. Then, a scheme for ULG similarity retrieval is introduced. Furthermore, an effective index structure is applied to reduce the searching time. Experimental results reveal that our method functions well on brain CT images similarity retrieval with higher accuracy and efficiency. Index Terms—Image modeling, image similarity retrieval, medical image, uncertain graph.

I. INTRODUCTION VER the past decades, the medical imaging technology such as computed tomography (CT) and magnetic resonance imaging (MRI) help doctors to diagnose diseases with the medical images. Because medical images improve the level of diagnostic accuracy significantly, an increasing number of patients are asked to obtain various medical images from radiology departments to highlight the suspected pathology. Hence, a vast amount of brain CT images are generated from hospitals every year and computerized methods are needed to solve this problem of the increasing amount of data. Patient-to-patient comparison, especially image-to-image comparison can provide doctors more valuable knowledge that is extremely useful during the whole process.

O

Manuscript received January 9, 2013; revised May 11, 2013 and June 26, 2013; accepted July 17, 2013. Date of publication August 15, 2013; date of current version March 3, 2014. The work was supported in part by the National Natural Science Foundation of China under Grant 61272184, Grant 61202090, and Grant 61100007, in part by the Natural Science Foundation of Heilongjiang Province under Grant F200903, Grant F201016, Grant F201024, and Grant F201130, in part by the Program for New Century Excellent Talents in Universities (NCET-11-0829), in part by the Fundamental Research Funds for the Central Universities under Grant HEUCF100609 and Grant HEUCFT1202, and in part by the Science and Technology Innovation Talents Special Fund of Harbin under Grant RC2010QN010024. H. Pan, P. Li, Q. Han, X. Feng, and L. Gao are with the College of Computer Science and Technology, Harbin Engineering University (HEU), Harbin 150001, China (e-mail: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]). Q. Li is with the Department of Computer Science, City University of Hong Kong, Kowloon, Hong Kong (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/JBHI.2013.2274798

A brain CT image not only contains information about the picture itself, but also indicates a series of treatments for a particular patient. By using image-to-image comparison doctors will find similar images that come from different patients. These patients may have a high probability of getting same disease because they have the similar pathological characteristics in their images. According to this domain knowledge, we believe that finding similar images from the brain CT image database will significantly assist doctors in finding patients who may get the same disease. And it will also help doctors to make diagnoses by acquiring information from the previous diagnoses and results. Up to now, there are two categories of image retrieval methods: 1) description-based image retrieval, which is performed using object retrieval based on image descriptions (such as keywords et al.), and 2) content-based image retrieval (CBIR), which supports retrieval based on the image content [1]. Due to the image always having complicated semantic information and the difficulty in describing an image with limited words, CBIR is an increasingly popular discipline in computer science [2]. Some simple and elegant methods were developed for CBIR. Swain and Ballard [3] proposed a method based on color histogram; Liu and Yu [4] introduced a method based on cooccurrence matrix; and Quellec et al. [5] proposed a method by wavelet-based CBIR. But brain CT image similarity retrieval is different from general image similarity retrieval [6], [7]. The brain CT image similarity retrieval has three challenges: 1) It is hard to recognize the object of medical interest from the images. Usually, medical experts are needed to annotate images manually, but annotation is time-consuming, laborious, and expensive work. 2) The object within the brain CT image has complicated properties so that it is hard to be literally described. 3) Brain CT image similarity retrieval requires high accuracy; little changes of the size or the location of the object can lead to different results. Only few attempts have been previously reported related to brain CT images similarity retrieval. Azhar and Basir [8] presented an image registration-based retrieval framework, and this technique can help medical experts in quantifying, localizing, and tracking disease. Rahman et al. [9] proposed a classification-driven image retrieval framework based on image filtering and similarity fusion by employing supervised learning techniques. Wang et al. [10] presented an image retrieval method based on feedback. However, these efficient methods all depend on image descriptors, and good brain CT image descriptors will lead to better retrieval results.

2168-2194 © 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

PAN et al.: BRAIN CT IMAGE SIMILARITY RETRIEVAL METHOD BASED ON UNCERTAIN LOCATION GRAPH

Texture is an important spatial feature for the doctors’ diagnosis because the texture can describe the objects naturally and even present the gray-level value variation of adjacent pixels. Devrim et al. [11] used LBP [12] and spatial indexing for brain MR image retrieval. And several researches [13]–[15] focused on shape description and matching problem. But most texture descriptors are described with static value that is calculated by some texture features. It has less consideration about the uncertainty and structure of the texture. At the same time, graph mining is a growing research topic because more and more objects can be naturally modeled by graphs. Some methods about graph learning and matching [16]–[18] were proposed for normal image retrieval. In these works, the image was described with feature points that cannot represent the texture of brain CT image. Specifically, they did not consider the texture uncertainty in brain CT image. A novel model of uncertain graphs [19]–[21] enlarges the application of graph model for practical objects. But this model, which is a kind of probability graphs cannot be applied directly to the image. To the best of our knowledge, there is no research on brain CT image similarity retrieval with uncertain graphs. In this paper, we first propose a novel model of uncertain location graph (ULG), which is a kind of uncertain graph based on the uncertainty of vertex’s location. Second, a new method is introduced to model brain CT images to ULGs based on the texture of images. A novel concept of texel that is the fundamental of texture is applied to maintain the structure of the texture. Then, an ULG-matching scheme is presented for brain CT image similarity retrieval. Furthermore, we propose an index structure and an approximate ULG-matching algorithm to speed up the brain CT image retrieval process. The main contributions of this paper are summarized as follows: 1) Propose a novel model of ULG with more mobility. 2) Use ULG for brain CT image modeling and similarity retrieval. 3) Introduce an approximate algorithm for brain CT image similarity retrieval with a new index structure. This paper is organized as follows: Section II introduces a preprocessing work. Section III proposes the model of ULG and gives a method for modeling brain CT image to ULG. Section IV describes the ULG similarity retrieval method (ULGR). Extensive experiments are shown in Section V. Section VI concludes the paper and highlights the future works. II. PREPROCESSING Through the observation of a large number of brain CT images, we find out that the brain CT image has the following advantages: 1) The size of the brain in the brain CT images is relatively stable. 2) The texture location of the brain CT image always has the same semantic. 3) The brain CT image has a simple background. Fig. 1(a) shows an original brain CT image. According to our doctors’ survey, we know that the hypodense (dark) struc-

575

Fig. 1. Process of preprocessing. (a) Original image. (b) ROI image. (c) Hierarchical texture image. (d) Size-uniformed texture image (TI).

tures indicate much more information than hyperdense (bright) structures. This also means that the texture of the hypodense structures is more important than the texture of the hyperdense structures during the texture comparison process. With this domain knowledge, Algorithm 1 gives different part of texture different value based on gray-level histogram to express the different importance of the different part of texture. Therefore, Algorithm 1 transforms the brain CT image to a size-uniformed hierarchical texture image, and it is essential to transform a brain CT image to an ULG. Example 1: In Algorithm 1, a brain CT image contains m × n pixels corresponding to a matrix IMm ×n . The value of IM(i, j) is the gray-level value of the jth column of the ith-row pixel in the image. The value of IM(i, j) smaller, the pixel from the image located ith column of the jth row darker. Algorithm 1 first extracts the region of interest (ROI) of the original image with parabolic shape, as shown in Fig. 1(a), semi-automatically. Then, the dark background is removed, as shown in Fig. 1(b). Let us calculate the gray-level histogram of the ROI and find the k-layer partition array of par[k]. With the Canny edge detector [22], a hierarchical texture image is generated. Furthermore, we correct the region of brain texture to upright semi-automatically [23]. Fig. 1(c) shows an example of the hierarchical texture image after upright correction. The output of Algorithm 1 is the size-uniformed texture image TI with the size of Column × Row as shown in Fig. 1(d). In this paper, Column = 161 and

576


Fig. 3.

Fig. 2.

Uncertainty of ULG.

Row = 151, and these two parameters depend on the resolution of the original data. With this preprocessing process, every original image in brain CT image database can be transformed to a size-uniformed texture image TI, and it also corresponds to a matrix TM. TM(i, j) is the gray-level value of the jth column of the ith-row pixel in TI and ∀TM(i, j) = 0 in the image means that there is a texture through the location (i, j). III. BRAIN CT IMAGE TO ULG In this section, we first give the definition of ULG and then use an example to show the uncertainty that is based on the vertex’s location. Second, we propose a modeling method. Then, a concept of texel is applied as a part of ULG to reduce the complexity of the ULG and texture. Because the texel consists of a sequence of vertices, we describe the texel mobility based on the vertex mobility and structure of the texel.

Distances between adjacent pixels.

Since the vertices in the ULG have mobility, the edges in the ULG are also not static. The edge (V (i), V (j)) between V (i) and V (j) can do some changes by mob(V (i)) and mob(V (j)) limited. Example 2: There is an ULG Gsam ple = {{V (1), V (2), V (3)}, {(V (1), V (2)), (V (2), V (3)), (V (1), V (3))}, and L, P, T = {T (1), T (2)}}, where T (1) = {V (1), V (2)} and T (2) = {V (3)} in Fig. 2(a). Every vertex has different importance values P (V (2)) > P (V (3)) > P (V (1)), and from (1), we know that mob(V (2)) < mob(V (3)) < mob(V (1)). The circle around the vertex is the boundary of vertex mobility area where the vertex is the center and the mobility mob of the vertex is the radius. The uncertainty of ULG is mainly that the vertex of the graph can move by its mobility area constraint. V (1) , V (2) , and V (3) are the locations to which the vertices V (1), V (2), and V (3) may be moved, and we consider that it is also a graph Gsam ple even though the vertex has been moved. Fig. 2(b) shows an illegal move of ULG Gsam ple because the location of V (2) is out of the mobility area of vertex V (2). From Example 1, one could observe that the model of ULG is different from the recently proposed classical graph and uncertain graph. The model of ULG contains more uncertainty, specifically about the uncertainty of vertex’s location.

A. Uncertain Location Graph

B. Modeling Method

As usual, we assume that the graph G = (V, E) is an abstract representation with the static vertex set V and the edge set E. An ULG is a kind of undirected graph G = (V, E), in which every vertex in the graph has the uncertain location. Every ULG is in a coordinate system, and the coordinate system is shown in Fig. 2. Definition 1: An ULG is a system G = (V, E, L, P, T), where V is the vertex set, E is the edge set, L: V→{1, 2, 3, . . . , n} is a function assigning labels to vertices, P: V→[0,1] is a function assigning importance values to vertices, T = {T(1), T(2),. . ., T(m)}, and T(i) is a subset of V. In this paper, the V (i) is a vertex of the graph with the label i by L functions, and V (i) has the location or the relative location (x, y). The P (V (i)) is the importance of the vertex V (i), which means that if the vertex V (i) moves, it will cause the P (V (i)) impact on the graph. The maximum P (V (i)) = 1 means that if the V (i) moves a little, it has no similarity than earlier. The minimum P (V (i)) = 0 means that it has no impact to the graph no matter how far V (i) moves. As we can see, P (V (i)) also means that the mobility of the V (i)—low importance means high mobility and vice versa. We define the mobility of vertex V (i) as

After the preprocessing work, we have got the uniformed texture image TI, and it corresponds to a matrix TM. There are five steps to transform the uniformed texture image TI to an ULG. Vertex: TM(i, j) is the gray-level value of the jth column of the ith row pixel in the TI, and ∀TM(i, j) = 0 in the image TI means that there is a texture through the location (i, j). So, we define there is a vertex with location (i, j) in the ULG. Edge: Because the vertex of ULG is based on pixel, the distance of any pair of adjacent √ pixels are shown in Fig. 3. There is no distance larger than 2, so we give the definition of edge as {((i, j),√(p, q)) ∈ E| TM(i, j) = 0, TM(p, q) = 0, dis((i, j), (p, q)) ≤ 2}. P: During the preprocessing, Algorithm 1 gives the different value to TM(i, j) based on their gray-level value, and with the domain knowledge, it is also an importance value of the vertex (i, j). Therefore, the function P is given by

mob (V (i)) =

1 . P (V (i))

(1)

P (i, j) = TM (i, j) .

(2)

Definition 2: A texel T(k) = {V(a), V(i), V(j), . . . , V(b)} is a path with no repeated vertices from V(a) to V(b) in the ULG, where ∀i, j, T(i) ∩ T(j) = Ø, and V = {T(1) ∪ T(2) ∪. . .∪ T(m)}. For image retrieval, texture feature is always too complex to be described and feature point is always insufficient to represent


577

Fig. 5. Flexibility of texel in ULG. (a) Example of basic texel tolerance. (b) Example of extended texel tolerance.

is located at (i, j) and whose gray-level value is TM(i, j) = 0. An edge is generated √ if the distance between any pair of vertices is not larger than 2. T set is generated by Algorithm 2, and the segmentation result is shown in Fig. 4(d). Function L gives the label to every vertex during the segmentation process to let texel T (i) be T (i) = {V (j), V (j + 1), V (j + 2), . . . , V (j + n)}. It will be helpful in later operation. With these steps, we can transform a brain CT image to an ULG. Fig. 4. Example of texture image to ULG. (a) Uniformed TI. (b) Partial enlarged view of (a). (c) Partial ULG without T set. (d) Partial ULG with T set.

the total texture feature. In this paper, we describe the texture feature with texels. With the definition of texel, we know that the texel is the fundamental element of texture, just as the picture described by pixel. A texel, which consists of a sequence of vertices, is a fully connected fragment of the texture and maintains the structure of a sectional texture. T: T = {T (1), T (2), . . . , T (m)} is a texel set. T is the segmentation of ULG. Actually, T is the segmentation of the texture. Algorithm 2 gives the detail to generate the texel set T , and for a texel, the following is the generating process: 1) Find a start point. 2) Find a vertex that has an edge with last vertex and that has never been labeled. If the vertex has more than one vertex connected and unlabeled, we choose the vertex according the following rule: i) Choose the vertex whose importance value is similar to the last one. ii) If the vertices have the same importance value, we choose the vertex randomly. 3) Execute step 2) until no vertex is found. Within Algorithm 2, the variable nodemun is used for labeling the vertex and the variable tnum is used for labeling the texel in ULG G. L: Function L labels the vertex in order during the generating T process. It seems like a topological sorting process. Algorithm 2 gives the explanation about how L works. Example 3: Fig. 4(a) shows a size-uniformed texture image TI, and Fig. 4(b) shows the partial enlarged view of Fig. 4(a). In this example, every vertex in Fig. 4(c) corresponds to a pixel that

C. Texel Mobility Since the vertex in the ULG has the mobility, the texel that consists of a sequence of vertices also has the mobility that we called texel tolerance. As (1) shows, the mobility of V (i) depends on P (V (i)), the texel tolerance depends on the importance value as well. For each vertex on the texel, there is an edge to the next vertex. The basic area into which the texel can be moved is formulated by the vertex mobility area. A different vertex on the texel may have a different vertex mobility area, but the flexibility of the texel is holistic. Therefore, we define the basic flexibility of texel tolerance standard btt as btt (T (j)) =

mob (V (i)) , n

V (i) ∈ T (j)

(3)

where n is the number of vertices in T (j). It means the vertex on the texel can be moved under its texel tolerance standard btt constraint. Two examples are shown in Fig. 5(a); texel Ta is the original texel, and texels Tb and Tc are the texels into which texel Ta might be moved. The circle in Fig. 5(a) is the vertex mobility area where the btt is the radius. Nevertheless, we find that the mobility is not adequate enough. When a part of texel moved merely and the other parts of the texel moved farther than btt, we also recon that they are the same. According to Weber–Fechner law, subjective sensation is proportional to the logarithm of the stimulus intensity [24], which means that if a stimulus varies as geometric progression, the corresponding perception will be altered in an arithmetic progression. Because the appearance of texel is irregular, we take the length of the texel as the stimulus and 2 as the common ratio of the geometric progression. Then,

578


the extend texel tolerance standard ett is defined as btt(T (k)), btt(T (k)) > log2 length ett(V (i)) = log2 length, btt(T (k)) ≤ log2 length

(4)

where the length is the hop from the longest vertex that is inside its mobility area. Fig. 5(b) shows two examples that texel Tb and Tc are the texels that texel Ta may moved with the ett constraint. From these examples, we can find that the length that we mentioned in (4) is not static and that the ett depends on the variability of the sequence of vertices and vertex’s sequence number. This also means that the texel in ULG has a dynamic mobility that is based on the structure of the texel. IV. ULG SIMILARITY RETRIEVAL PROBLEM Generally, graph matching is a problem to establish a correspondence between the nodes and the edges of two graphs when they satisfy some constraints. ULG similarity retrieval problem is like the graph-matching problem. We denote G = {V, E, L, P, T } as the query ULG and G = {V , E , L , P , T } as an ULG of the ULG set Gdb = {G , G1 , G2 , . . . , Gn }, which will be searched. In this section, we first give the definition of similarity between ULGs. Then, a basic scheme for ULGR is proposed. To speed up the image retrieval process, an efficiency index is applied. Further, an advanced ULGR with the index is introduced. A. Similarity Between ULGs As described earlier, the ULG has a high sensitivity about the location of the vertex. The vertex’s mobility is based on its importance value and the structure of the texel. In this subsection, two levels of correspondence have been defined: matching and similar. With these definitions, the similarity of ULGs can be calculated and a corollary will be proposed for future ULG similarity retrieval. Definition 3: Vertices V(p) and V (q) are the matching vertices if the distance between V(p) and V (q) satisfies dis(V(p), V (q)) ≤ mob(V(p)) + mob(V (q)). Definition 4: Texels T(i) and T (j) are the matching texels if every vertex V(p) on the vertex T(i) has a correspondence to the vertex V (q) on the texel T (j) and every pair of vertices V(p) and V (q) satisfies dis(V(p), V (q)) ≤ btt(T(i)) + btt(T (j)). Fig. 6(a) shows an example of texel Tb matching texel Ta. Because every vertex on texel Tb has a corresponding vertex on texel Ta and satisfies the constraint. The basic constraint line is a line with the distance dis = btt(Tb) + btt(Ta) from texel Ta. Definition 5: Texel T (j) is similar to T(i) if every vertex V(p) on the texel T(i) has a correspondence to the vertex V (q) on the texel T (j) and every pair of vertices V(p) and V (q) satisfies the . constraint dis(V(p), V (q)) ≤ btt(T(i)) + btt(T (j)) + loglength 2 Where the length is the hop from the vertex V (q) to V (n), which V (n) is the farthest vertex has been matched. Fig. 6(b) shows an example of texel Tb and Tc similar to texel Ta. For texel Tb, extend constraint line is a line that has the distance from texel Ta. dis = btt(T a) + btt(Tb) + loglength 2 Definition 6: Texel T (j) is partially similar to T(i) if part of vertices V(p) on texel T(i) have correspondence to vertices V (q)

Fig. 6. Matching problem of two texels. (a) Example of texel matching. (b) Example of texel similarity. (c) Example of texel partly similarity.

on texel T (j) and satisfy dis(V(p), V (q)) ≤ btt(T(i)) + btt(T (j)) . + loglength 2 Where the length is the hop from the vertex V (q) to V (n), which V (n) is the farthest vertex has been matched. The similarity between two texels st is calculated as later and PT(i) is the vertex set that consists of the vertices satisfying the constraint V (p)∈P T (i) P (V (p)) . (5) st(T (i), T (j)) = V (l)∈T (i) P (V (l)) Fig. 6(c) shows an example that texels Tb and Tc are partially similar to texel Ta. Further, the similarity between two ULGs is based on their texels similarity. Definition 7: The similarity sg of ULG G to ULG G can be calculated as follows: T (i)∈T (st(T (i), T (j))∗ V (m )∈T (i) P (V (m))) sg(G , G) = T (i)∈T V (m )∈T (i) P (V (m)) T (i)∈T st(T (i), T (j)) (6) = n where n is the number of texels in T. Definition 7 gives the definition of similarity between two ULGs, and it pays more attention to the important texel that contains more important vertex or the longer one. Corollary: If a pair of vertices V(p) and V (q) consists of matching vertices and another pair of vertices V(m) and V (n) satisfies the constraint dis(V(m), V (n)) ≤ btt(T(i)) + btt(T (j)) , {V(p), V(m) ∈ T(i), V (q), V (n) ∈ T (j)}, where + loglength 2 the length is the hop from V(p) to V(m), we guess texel T (j) is


Fig. 7.

579

How we can guess the texels that may be similar.

similar or partially similar to T(i). We call V(p), V (q) to be the start points and V(m), V (n) to be the end points. In Fig. 7, there are a number of pairs of vertices satisfying the constraint proposed in the corollary. With the top-down approach, the longest similar texel should be found first because the longest texel contains several shorter texels. It is good to choose a, d as the start points and e, c as the end points, and just one comparison between texels Ta and Tb is required. It is good to choose c, f as the start points and b, g as the end points for the comparison of texels Ta and Tc. Even, there are some vertices between g and f on texel Tc that do not satisfy the constraint, but this is the best way to compare all the vertices that may satisfy the constraint. Because of reducing of the compare time and the searching time, the closer to the terminal point the better to choose with this theory. B. Basic ULG Similarity Retrieval Method In classical graph theory, graph matching is an NP-complete problem. With the series of definitions that we have presented earlier, an approximate algorithm will be proposed to reduce the ULG-matching problem to a P problem. Algorithm 3 gives a basic scheme for ULG similarity retrieval. It compares each texel in the query graph with every texel in the ULG database. Suppose that there is a query ULG G with m texels and that there are n ULGs with average t texels in Gdb ; then O(m × n × t) comparisons are needed to compare the texels. If we assume that every texel has the same number of vertices l, then there O(m × n × t × l × l × l/2) calculations are needed. The main problem of Algorithm 3 is that it is time-consuming to compare the impossible similar texels, and for each pair of potential texels it needs O(l × l × l/2) calculations in average. The ideal method is to just compare the possible similar texels with T (i), which is a texel in query ULG only. Therefore, the problem is how to find the texels that are possible similar to T (i). C. Index for ULG Similarity Retrieval The best improvement must be finding the texel set T wl, which consists of the texels that may similar to texel T (i) directly. In this paper, every ULG is in a Column × Row plane. With the corollary, we design the index structure shown in Fig. 8 to find the texel set T wl, which consists of the texels that may be partially similar to texel T (i) from V (b) to V (y).

Fig. 8.

Index for medical image similarity search.

The size of pixel index in Fig. 8 is given by Column × Row. Every point PI(i, j), which is located at (i, j) in pixel index, has a trans table TT(i, j), which records the texels that have ever crossed the (i, j) location from different brain CT images. And every item in the trans table TT(i, j) corresponds the texel information that it belongs to. All texels information is recorded in the texture table. From the corollary, we know that to find the texel set T wl that may be partially similar to texel T (i) from V (b) to V (y) we need to find the pair of vertex V (bb) and V (yy), which satisfy the constraints discussed in the corollary. Algorithm 4 gives the description about how the index structure works to find the texel set T wl, which may be partially similar to texel T (i) from V (b) to V (y). The main idea of Algorithm 4 is: At first, we set V (b) as the start point and V (y) as the end point: 1) V − is a location set that consists of the location (p, q) that matches with the start point V (b). 2) V + is a location set that consists of the location (p, q) that is similar to the end point V (y). 3) With pixel index, we can get the texel set wl− , which is a union set of TT(p, q) and which corresponds to every location in V − . Similarly, we can get the texel set wl+ , which is a union set of TT(p, q), which corresponds to every location in V + .

580


4) Get the texel set T wl = wl+ ∩ wl− , which may be partially similar to texel T (i) from V (b) to V (y) by corollary direction. Then, we should also consider the situation in which V (y) is the start point and V (b) is the end point. After the aforementioned four similar steps, we can get the texel set T wl , which may be partially similar to texel T (i), where V (y) is the start point and V (b) is the end point. 5) The final potential similar texel set is T wl = T wl ∪ T wl . The index shown in Fig. 8 gives us a possibility to find the potential similar texels, and Algorithm 4 gives us a way to find the texel set T wl, which consists of the texels that may be partially similar to texel T (i) from V (b) to V (y). D. ULG Similarity Retrieval With Index (ULGR-Index) ULGR is time-consuming because it requires a lot of time to compare impossible texels. The advanced idea is to just compare the texels that may be similar. The index and Algorithm 4 give us a scheme to find the texel set T wl, which consists of the texels that may be partially similar to texel T (i) from V (b) to V (y). The ULG similarity retrieval with index is described in Algorithm 5. It just needs O(m × l × p × l/4 × bttavg ), where p is the number of the texel that may be similar to texel T (i) and bttavg is the average of all btt(T (i)) in G. Algorithm 5 includes the following steps: 1) For each texel T (i) in G, find a pair of vertices as the start point and end point. 2) Find the texel set T wl, which consists of the texels that may be similar to T (i). 3) For each texel T (j) in T wl, calculate st(T (i), T (j)) by (5) and save it into the score matrix scorelistm ×n , where m is the number of ULG in Gdb and n is the number of texel in G. 4) Execute step 2) until algorithm cannot find a pair of vertices as the start point or end point. 5) Execute step 1) until all texels in G have been calculated. 6) With the scorelist, compute the similarity score for each ULG in Gdb by (6) and give the result of rank.

TABLE I TIME-CONSUMING OF THE FIVE METHODS

V. EXPERIMENTS In this section, four experiments were implemented to demonstrate and compare the performance of our methods for brain CT image retrieval. For comparison, the two methods that we have proposed earlier—basic ULGR and ULG similarity retrieval with index (ULGR-index)—and the other three algorithms were also implemented—KLT [25], [26], Sp-LBP [11], and LBP [12]. KLT is a feature tracker, which selects regions that can be tracked well, and valuates the consistency of features between nonconsecutive frames. LBP is a local texture descriptor with the grayscale invariant. Sp-LBP is the LBP approach with spatial indexing. In our experiments, we use eight pixels as a symmetric neighbor set and the spatial indexing is with three annular splits and eight angular splits. The database used in these experiments is from 50 patient subjects with 9 brain CT slices. Three simulated bias field from the BrainWeb [27] are utilized to simulate other 1550 images [11]. Therefore, our experiment data include 2000 brain CT images. All the experiments are implemented in C/C++ using Intel Core(2) processor 2.66 GHz and 2 GB of RAM. A. Time-Consuming Analysis For a database with 2000 brain CT images, Table I shows the time-consuming of five methods. LBP and Sp-LBP are faster


Fig. 9.

581

Time-consuming comparison between ULGR and ULGR-index. Fig. 10.

than other three methods because these two methods describe texture feature by static value with less consideration about texture structure and uncertainty, but they are also efficient. KLT, which is one of most popular algorithms, is the slowest in this retrieval process because it takes the graph isomorphism problem to consideration. Graph isomorphism problem in graph matching is the most useful but time-consuming. However, it is not necessary for brain CT image retrieval process. Fig. 9 shows the comparison of the time consumed between ULGR and ULGR-index. From Fig. 9, we can see that ULGRindex is always faster than ULGR whether in small database or in the database with 2000 images. The time-consuming increasing speed of ULGR is about 2–3 s per 100 images constantly. The time-consuming increasing speed of ULGR-index is under 1 s per 100 images, and with the increase of database, the time-consuming increasing speed is getting down. This comparison of the time consumed between ULGR and ULGR-index demonstrates that ULGR-index is much faster than ULGR. B. Precision Analysis Performance of image similarity retrieval is generally measured by precision and recall, but for brain CT images, we think each of them to be relevant. In this experiment, we simulated people just caring about the top 20 results and we use the precision of the top 20 results to measure the performance of each methods. Precision is calculated by precision =

relevant returns returns

(7)

where “returns” is the number of images that feedback and in this experiment returns = 20, “relevant returns” is the number of the right images in the top 20 results that is decided by doctors. Fig. 10 shows the maximum precision, average precision, and minimum precision of five different methods that are based on 50 independent queries. From Fig. 10, we can see that ULGR is almost the same with ULGR-index in three parameters. These two methods hold higher score in maximum precision, average precision, and minimum precision. Significantly, the average precision of ULGR and ULGR-index is about 0.7 higher than the maximum precision of the other three methods. And the maximum precision of ULGR and ULGR-index is about 0.45 higher than the average precision of the other three methods.

Precision of the five methods.

Some examples of search results are also shown in Fig. 11. Images 67–70 are from a patient’s brain CT image with the adjacent slice. That is, if we find an image whose number is between 67 and 70 and is similar to an image whose number is between 67 and 70, then that might be a right answer because they come from one patient’s brain CT images. The first row in Fig. 11 in the image will be retrieved in the database, and the other five rows are the top five similar images. Fig. 11 shows that the top five results of ULGR and ULGR-index are the same. With the medical knowledge directed, we can believe that ULGR and ULGR-index could find the image with the similar texture better than the other three methods. This also means that ULGR and ULGR-index that are based on the model of ULG function better than other models for brain CT image similarity retrieval. C. Sensitivity Analysis Precision and recall are generally used to measure the performance of image similarity retrieval, but in the real world, specifically for brain CT images, little change is crucial. Fig. 12(a) shows a brain CT image from a normal human being, and Fig. 12(b) shows an image we put a simulated field in Fig. 12(a). Although they look similar to each other, doctors just focus on the pathology lesions. Therefore, we measure the sensitivity as Sensitivity =

S(a) − SC(a) S(a)

(8)

where S(a) is the relevant results set of image a and SC(a) is the relevant results set of little-changed image a, as shown in Fig. 12(b), and we used the top 20 results as the relevant results. This experiment is designed based on the practical requirements like distinguishing the normal images and the abnormal images that have little differences from the normal images for early diagnosis and distinguishing the change of pathology lesions in brain CT images to evaluate the status of disease. Fig. 12(c) shows the maximum sensitivity, average sensitivity, and minimum sensitivity of the five methods. ULGR is almost the same with ULGR-index, and these two methods have a highest score in maximum sensitivity, average sensitivity, and minimum sensitivity. The minimum sensitivity of ULGR and

582

Fig. 11.


Example for precision of the five methods.

Fig. 12. Sensitivity of the four methods. (a) Original image. (b) Simulated image. (c) Sensitivity of the five methods.

ULGR-index is about 0.77, which is much higher than the maximum sensitivity of the other three methods. After this experiment, one could observe that ULGR and ULGR-index can distinguish the images that have a little change better than the other three methods. For brain CT images, the model of ULG can focus on the change of textures better, specifically for normal and abnormal brain CT images.

D. Score Distribution Analysis In this experiment, we propose a novel concept of distribution ability to evaluate the score distribution of the search result. Example 4: Assume that we use methods A and B for image retrieval in image database that has 100 images. The result of method A divides 100 images into two categories—50 images scored 1 and others scored 0.5. Differently, method B divides 100 images into 100 categories that also means method B gives 100 different scores to 100 images. Now, if we want to get the top 20 results, method A has less ability to give us the accurate results than method B. Because method B gives more information (such as score, tag, and so on) to divide the images into more categories, we call the ability, which can give the result with more information, as the distribution ability.

Fig. 13.

Distribution of scores for the five methods.

From Example 4, we can observe that the distribution ability is a concept to evaluate how much information the method can give to the result. Fig. 13 is the statistic of the score distribution by 200 queries. From Fig. 13, we can find that the line of ULGR, ULGR-index, Sp-LBP, and LBP scores concentrated between 0 and 1. But LBP and Sp-LBP score distributions are concentrated between 0.85 and 1. With the knowledge we get from Example 4, LBP and Sp-LBP score distributions cannot give us enough information about the retrieval results. In information theory, entropy is a measure of the uncertainty associated with a random variable [28]. For image retrieval, every score distribution is the information that the database give us. In the real world, every image in the database is independent. For once query, every image in the database will have a score x between 0 and 1. And the score x has the possibility Pos(x), that means the image has Pos(x) to get score x for the query image. Therefore, we calculate the entropy as the distribution ability (9) H=− Pos (x) ln(Pos(x)) . The entropy scores of the five methods are shown in Table II. The entropy scores of ULGR, ULGR-index, and KLT are much greater than the other two methods. That is, the ULGR,


TABLE II SCORE OF DISTRIBUTION ABILITY

583

REFERENCES

[7] H. Pan, X. Xie, W. Zhang, and J. Li, “Mining image sequence similarity patterns in brain images,” in Proc. PRICAI 2006: Trends Artif. Intelligence. Berlin, Germany: Springer, pp. 965–969. [8] Q. Azhar and O. Basir, “Semantic image retrieval in magnetic resonance brain volumes,” IEEE Trans. Inf. Technol. Biomed., vol. 16, no. 3, pp. 348– 355, May 2012. [9] M. Rahman, S. Antani, and G. Thoma, “A learning-based similarity fusion and filtering approach for biomedical image retrieval using SVM classification and relevance feedback,” IEEE Trans. Inf. Technol. Biomed., vol. 15, no. 4, pp. 640–646, Jul. 2011. [10] R. Wang, H. Pan, Q. Han, J. Gu, and P. Li, “Medical image retrieval method based on relevance feedback,” in Advanced Data Mining and Applications. Berlin, Germany: Springer, 2012, pp. 650–662. [11] U. Devrim, A. Ekin, and R. S. Jasinschi, “Local structure-based region-ofinterest retrieval in brain MR images,” IEEE Trans. Inf. Technol. Biomed., vol. 14, no. 4, pp. 897–903, Jul. 2010. [12] O. Timo, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 7, pp. 971–987, Jul. 2002. [13] X. Xu, D. Lee, S. Antani, and L. R. Long, “A spine X-ray image retrieval system using partial shape matching,” IEEE Trans. Inf. Technol. Biomed., vol. 12, no. 1, pp. 100–108, Jan. 2008. [14] D. Zhang and G. Lu, “Review of shape representation and description techniques,” Pattern Recognit., vol. 37, no. 1, pp. 1–19, 2004. [15] G. V. Pedrosa, C. A. Z. Barcelos, and M. A. Batista, “An image retrieval system using shape salience points,” in Proc. 2011 IEEE Int. Symp. Circuits Syst., pp. 2797–2800. [16] T. S. Caetano, J. J. McAuley, C. Li, Q. V. Le, and A. J. Smola, “Learning graph matching,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 6, pp. 1048–1058, Jun. 2009. [17] Y. Zhu, L. Qin, J. X. Yu, Y. Ke, and X. Lin, “High efficiency and quality: Large graphs matching,” in Proc. 20th ACM Int. Conf. Inf. Knowl. Manage., 2011, pp. 1755–1764. [18] T. Cour, P. Srinivasan, and J. Shi, “Balanced graph matching,” Adv. Neural Inf. Process. Syst., vol. 19, p. 313, 2007. [19] Z. Zou, J. Li, H. Gao, and S. Zhang, “Mining frequent subgraph patterns from uncertain graph data,” IEEE Trans. Knowl. Data Eng., vol. 22, no. 9, pp. 1203–1218, Sep. 2010. [20] Z. Zou, J. Li, H. Gao, and S. Zhang, “Finding top-k maximal cliques in an uncertain graph,” in Proc. IEEE 26th Int. Conf. Data Eng., 2010, pp. 649–652. [21] J. Li, Z. Zou, and H. Gao, “Mining frequent subgraphs over uncertain graph databases under probabilistic semantics,” VLDB J., vol. 21, no. 6, pp. 753–777, 2012. [22] J. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-8, no. 6, pp. 679–698, Nov. 1986. [23] A. Fitzgibbon, M. Pilu, and R. B. Fisher, “Direct least square fitting of ellipses,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 5, pp. 476– 480, May 1999. [24] A. S. Reber, E. S. Reber, and R. Allen, The Penguin Dictionary of Psychology, vol. 1. London, U.K.: Penguin, 1985. [25] B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in Proc. 7th Int. Joint Conf. Artif. Intell., 1981, pp. 674–679. [26] S. Birchfiedl, “Derivation of KLT tracking equation,” unpublished. [27] J. G. Sled, A. P. Zijdenbos, and A. C. Evans, “A nonparametric method for automatic correction o f intensity nonuniformity in MRI data,” IEEE Trans. Med. Imag., vol. 17, no. 1, pp. 87–97, Feb. 1998. [28] T. M. Cover and J. A. Thomas, Elements of Information Theory. New York, NY, USA: Wiley, 1991.

[1] J. Han and M Kamber, Data Mining: Concepts and Techniques, 2nd ed. Beijing, China: China Machine Press, 2006, pp. 396–399. [2] R. Datta, D. Joshi, J. Li, and J. Z. Wang, “Image retrieval: Ideas, influences, and trends of the new age,” ACM Comput. Surveys (CSUR), vol. 40, no. 2, pp. 5–60, 2008. [3] M. J. Swain and D. H. Ballard, “Color indexing,” Int. J. Comput. Vis., vol. 7, no. 1, pp. 11–32, 1991. [4] G. Liu and Y. J. Yu, “Image retrieval based on the texton co-occurrence matrix,” Pattern Recognit., vol. 41, no. 12, pp. 3521–3527, 2008. [5] G. Quellec, M. Lamard, G. Cazuguel, B. Cochener, and C. Roux, “Fast wavelet-based image characterization for highly adaptive image retrieval,” IEEE Trans. Image Process., vol. 21, no. 4, pp. 1613–1623, Apr. 2012. [6] H. M¨uller and T. M. Deserno, “Content-based medical image retrieval,” in Biomedical Image Processing. Berlin, Germany: Springer, 2011, pp. 471–494.

Haiwei Pan received the Ph.D. degree from Harbin Institute of Technology, Harbin, China. He is currently an Associate Professor and an Assistant Dean at Harbin Engineering University, Harbin. His current research interests include database, data mining, medical image mining, graph data processing and massive data processing. He has authored or coauthored more than 40 publications in these and related areas. He has presided over the national natural science foundation of China, Heilongjiang Province Natural Science Fund, postdoctoral fund project in Heilongjiang Province, and other projects.

ULGR-index, and KLT methods can distinguish the similarity in more detail. We could also assume that there are a large number of images at present; the methods with more entropy can divide images into more categories. VI. CONCLUSION In this paper, we proposed a novel model of ULG. The uncertainty of ULG that is based on vertex’s location is quite different from other models. Then, we described a method to transform the brain CT image to ULG based on brain CT image texture. In addition, a basic ULGR is introduced. To speed up the brain CT image similarity retrieval process, an index and an ULGR with index ULGR-index are proposed. By these processes, we can retrieve brain CT images that are in the brain CT image database. A series of experiments were carried out to demonstrate and compare the performance of the proposed method for medical image similarity retrieval. Experimental results show that ULGR and ULGR-index methods can find the image with similar texture more accurately. Second, ULGR and ULGR-index methods can distinguish the medical images having little difference from each other. Third, ULGR and ULGR-index methods have more ability to distinguish the similarity of a large number of images. By this series of comparison, we can observe that the method of ULGR-index has the same ability as that of ULGR. However, ULGR-index method has much advantage in time complexity. ULGR and ULGR-index methods are based on the model of ULG that is fairly generic and can be extended to different applications. Future work may include other practical applications with the ULG model and reducing the ULG-matching time.

584


Pengyuan Li received the B.E. degree in computer science and technology from Zhengzhou University, Zhengzhou, China, in 2011. He is currently working toward the Master’s degree in computer software and theory from Harbin Engineering University, Harbin, China. His current research interests include medical image mining and graph data mining.

Qing Li (SM’02) received the B.Eng. degree from Hunan University, Changsha, China, and the M.Sc. and Ph.D. degrees from the University of Southern California, Los Angeles, USA, all in computer science. He is currently a Professor at the City University of Hong Kong, Kowloon, Hong Kong. His current research interests include object modeling, multimedia databases, social media, and recommender systems. He has authored or coauthored more than 300 publications in these and related areas. Dr. Li is a Fellow of IET and a Member of ACM SIGMOD and the IEEE Technical Committee on Data Engineering. He is the chairman of the Hong Kong Web Society and is a steering committee member of DASFAA, ICWL, and WISE Society.

Qilong Han received the Ph.D. degree from Harbin Institute of Technology, Harbin, China. He is currently an Associate Professor in the Department of Computer Science and Technology, Harbin Engineering University (HEU), Harbin. Before joining HEU in 2006, he held positions at Harbin Institute Technology and the Petrochina Daqing Petrochemical Company. His current research interests include spatiotemporal data mining, graph mining, and sensitive data protection.

Xiaoning Feng received the Ph.D. degree from Harbin Engineering University, Harbin, China. He is currently an Associate Professor in the Department of Computer Science and Technology, Harbin Engineering University. He was engaged in petri theory, large-scale modeling of complex systems, distributed simulation, and etc. in his Academic career in recent ten years. He got some rewards and not only took charge of but also participated in several projects from the government. He has published over 20 academic articles.

Linlin Gao received the B.E. degree in computer science from the Huanghuai College, Henan, China, in 2012. She is a currently working toward the Ph.D. degree in the Department of Computer Science, Harbin Engineering University, Harbin, China. Her current research interests include data mining and database.