Modified Jakubowski Shape Transducer for Detecting Osteophytes and Erosions in Finger Joints Marzena Bielecka1 , Andrzej Bielecki2 , Mariusz Korkosz3, Marek Skomorowski2, Wadim Wojciechowski4 , and Bartosz Zieliński2 1
3
Department of Geoinformatics and Applied Computer Science, Faculty of Geology, Geophysics and Environmental Protection, AGH University of Science and Technology, Mickiewicza 30, 30-059 Cracow, Poland
[email protected] 2 Institute of Computer Science, Jagiellonian University, Łojasiewicza 6, 30-348 Cracow, Poland {bielecki,skomorowski}@ii.uj.edu.pl,
[email protected] Division of Rheumatology, Departement of Internal Medicine and Gerontology, Jagiellonian University Hospital, Śniadeckich 10, 31-531 Cracow, Poland
[email protected] 4 Department of Radiology, Jagiellonian University Hospital, Kopernika 19, 31-531 Cracow, Poland
[email protected]
Abstract. In this paper, a syntactic method of pattern recognition is applied to hand radiographs interpretation, in order to recognize erosions and osteophytes in the finger joints. It is shown that, the classical Jakubowski transducer does not distinguish contours of healthy bones from contours of affected bones. Therefore, the modifications of the transducer are introduced. It is demonstrated, that the modified transducer correctly recognizes the classes of bone shapes obtained based on the medical classification: healthy bone class, erosion bone class and osteophyte bone class. Keywords: Syntactic method of pattern recognition, Medical imaging, Computer assisted rheumatic diagnosis.
1
Introduction
Arthritis and musculoskeletal disorders are more prevalent and frequent causes of disability than heart disease or cancer [11]. There are a number of inflammatory as well as non-inflammatory diseases within the scope of rheumatology and diagnostic radiology. It is essential to distinguish between inflammatory disorders, which can be fatal, and non-inflammatory disorders, which are relatively harmless and can occur in the majority of people aged around 65. To give a diagnosis, A. Dobnikar, U. Lotrič, and B. Šter (Eds.): ICANNGA 2011, Part II, LNCS 6594, pp. 147–155, 2011. c Springer-Verlag Berlin Heidelberg 2011
148
M. Bielecka et al.
an X-ray is taken of the patients hand and symmetric metacarpophalangeal joint spaces and interphalangeal joint spaces are analyzed [14]. Thus, the changes in border of finger joints surfaces observed on hand radiographs are a crucial point in medical diagnosis and support important information for estimation of therapy efficiency. However, they are difficult to detect in an X-ray picture when examined by a human expert, due to the quantity of joints. On the other hand, it is extremely important to diagnose pathological changes in the early stages of a disease, which means that differences in the order of 0.5mm between the contours of pathologically changed bones and unaffected ones need to be identified. The possibility of performing such analysis by a computer system is a key point for diagnosis support. Therefore, studies concerning possibilities of implementation such systems are topic of numerous publications [12,13,16] (see other references in [6]). These researches are a part of the extensive stream of studies concerning artificial intelligence methods application in medical image understanding [15].
a
b
c
d
e
Fig. 1. Healthy joint (a), bones with osteophytes (b, c) and joints with erosions (d, e) radiograph
This paper is a continuation of studies described in [2,3,4,5,6,17,18], concerning automatic hand radiographs analysis. In the previous papers the preprocessing and joint location algorithms were presented. At the beginning, the applied approach turned to be effective in about 90% of cases [6], the algorithm was then improved in [18] and efficiency at 97% was achieved. Based on those locations, the algorithm identifying the borders of the upper and lower joint surfaces was proposed [5]. The preliminary analysis of such borders due to erosions detection is studied in [2,4]. In this paper, a syntactic method of pattern recognition is applied to hand radiographs interpretation, in order to recognize erosions and osteophytes in the finger joints. Example of the healthy joint radiograph and joints with osteophytes and erosions are shown in Fig.1(a), Fig.1(b,c) and Fig.1(d,e), respectively. Possible location of the osteophytes and erosions are shown as bold line in Fig.2(a) and Fig.2(b), respectively. It is shown that, the classical Jakubowski transducer [8] does not distinguish contours of healthy bones from contours of
Application of Shape Description
a
149
b
Fig. 2. Contours with possible locations where osteophytes (a) and erosions (b) may occur marked by bold line
affected bones. Therefore, the modifications of the transducer are introduced. It is demonstrated, that the modified transducer correctly recognizes the classes of bone shapes obtained based on the medical classification: healthy bone class, osteophyte bone class and erosion bone class. The paper is organized in the following way. The shape description methodology is recalled in section 2. In section 3, Jakubowski transducer is used for bone contours analysis and the necessary modifications are introduced.
2
Shape Description Methodology
Let us recall a formalism presented in [7,9,8,10], where basic unit of the analysed pattern is one of the sixteen primitives from set PRIM, being line segments or quarters of a circle (see Fig.3a). It should be mentioned that bi-indexation enumerating primitives plays a crucial role in the contour analysis. Let us also recall definition of a contour k = p1 p2 ... pm , where p1 , p2 , ..., pm are successive primitives of the contour k. Symbols pi pi+1 denotes that pi is connected to pi+1 , such that hd(pi ) = tl(pi+1 ), where hd(pk ) and tl(pk ) corresponds to head and tail of the primitive pk (see Fig.3b). Characterological description of contour k is chain of successive primitive types defined as char(k) = si1 j1 si2 j2 ...sim jm . Moreover, Qo is defined as set of primitives from the o-th quarter, for o = 1, 2, 3, 4, therefore: Qo = {sij : (j = o) ∨ (i = 1 ∧ j = o ⊕ 1)}, where o ⊕ 1 = 1 if o = 4 and o ⊕ 1 = o + 1 otherwise.
150
M. Bielecka et al.
a
b
Fig. 3. Set PRIM (a) and construction of primitive (b)
A contour k with char(k) = v such that v ∈ Q+ i ∧(length(v) > 1∨(length(v) = 1 ∧ v ∈ Qi \ (Qi⊕3 ∪ Qi⊕1 ))) is said to be the contour from the singular quadrant ((i)-singuad for short). In other words, the (i)-singuad is a contour composed of primitives from the ith quadrant. + Given contours k , k such that char(k ) ∈ Q+ i , char(k ) ∈ Qj , and / Qi , and if j = i ⊕ 2 then b ∈ Qj \ (Qj⊕3 ∪ Qj⊕1 ). If char(f irst(k )) = b ∈ k = k k , we say that k crates the so called (i,j)-biquad with char(k) = ˙ ). The first primitive of k i.e. f irst(k ) is called a switch enchar(k )char(k coded by the string ij named the basic mark. Furthermore, according to definition 10, paper [8], transducer is a 5-tuple: T = (G, Σ, Δ, δ, G0 ), where G is a finite nonempty set of states, Σ is a finite nonempty input alphabet, Δ is a finite nonempty output alphabet, G0 is a finite nonempty set of start states, G0 ⊂ G and δ is a finite subset of G × Σ ∗ × Δ∗ × G. Intuitively, if (q, u, v, q ) ∈ δ, it means that if the machine is in the state q and the string u ∈ Σ ∗ is given as an input, then the state of the machine is changed into the state q and v ∈ Δ∗ becomes the machine output.
3
Bone Contour Analysis
The transducer Tm = ({q1 , q2 , q3 , q4 }, S, {1, 2, 3, 4}, δ, {q1, q2 , q3 , q4 }), where δ is given by the graph depicted in Fig.4 was proposed by Jakubowski in [8]. If u causes the transition from the state qi to qj , i = j, then u designates the switch of an (i, j)-biquad, what simply means, that there is a switch between ith and
Application of Shape Description
151
Fig. 4. δ function of the original transducer from paper [8], Fig.14b
j th quarter. Therefore, for each analysed contour, chain of biquads is taken as the result of transition. If transducer with δ function is used in case of the bone, it usually can not distinguish the healthy bone contours from contours of the bone with osteophyte or erosion. As an example, let us consider the simplified contours presented in Fig.5. Contour presented in Fig.5(a) presents no pathological changes. However contour in Fig.5(b) is convex, what means that it contains osteophyte. On the other hand, contour in Fig.5(c) is concave, that is why it contains erosion. However, it can be easily verified, that all three contours are represented by the same biquad description 32.21, despite the fact that they represents healthy bone, bone with osteophyte and bone with erosion, respectively. Wherefore, authors had to modified the transducer to differentiate those three classes of bones. For this purpose, δ function was created as modification of the original δ function. Thus, new function behaves differently in case of primitives placed at the border of two quarters (s11 , s12 , s13 and s14 ). To better understand the changes, let assume that k is fragment of the contour which characterological description char(k) = sj s1o , where the first primitive was already classified by transducer to j th quarter and the second primitive is placed at the border of two quarters. Then, in case of function δ the biquad value is described by function:
c
b
a
d
M. Bielecka et al.
152
e
Fig. 5. Example of the healthy contour (a), contours with osteophytes (b and d) and contours with erosions (c and e). Number near primitive represent the quarter to which this primitives belong to. If the primitive first index equals 1, there are two numbers, as such primitive is placed between two quarters.
⎧ ,o = j ∨ o = j ⊕ 1 ⎨ none biquads(δ) = j(j ⊕ 1) , o = j ⊕ 2 ⎩ j(j ⊕ 3) , o = j ⊕ 3 On the other hand, modified function δ works differently for two last cases: ⎧ ,o = j ∨o = j ⊕ 1 ⎨ none biquads(δ ) = j(j ⊕ 2) , o = j ⊕ 2 ⎩ j(j ⊕ 2) , o = j ⊕ 3 It can be easily verified, that all three contours represented by the same chain of biquad 32.21 in case of δ function are represented by three different chains of biquads in case of δ function - see Tab.1. The changes in transducer were introduced due to the fact that in healthy bone, the angles between successive primitives are bigger than 90◦ , what can be observed in Fig.1a. If angles are equal or smaller than 90◦ , it means that bone contour contains pathological changes - osteophyte if an acute or right angle is inside of the bone and erosion if an acute or right angle is outside of the bone - see 5b and 5c, respectively. Original δ function does not take such regularity into account and in many cases does not differentiate contours from different bone classes.
Application of Shape Description
153
Fig. 6. δ function of the transducer, created based on the original δ function from Fig.4 Table 1. δ biquad description, δ biquad description and medical assignment of contours from Fig.5 Figure δ biquad description δ biquad description osteophyte or erosion Fig.5a 32.21 32.21 none Fig.5b 32.21 31 osteophyte Fig.5c 32.21 31.13.31 erosion Fig.5d 31 31 osteophyte Fig.5e 32.23.31 31.13.31 erosion
Moreover, it has to be stressed that introduced δ function not only differentiates two contours with the same δ biquad description, but also integrates some contours with different δ biquad description. The integration can be observed in case of Fig.5b and Fig.5d, as well as in case of Fig.5c and Fig.5e. In both pairs, biquad description generated by δ function is different for both contours, but description generated by δ is identical (see Tab.1). However, it turns out that this is an advantage, because δ function generates the same biquad description for contours with the same pathological change - either both have osteophyte, or both have erosion. Naturally, the examples in Fig.5 are quite simple, due to the fact, that they contain 45◦ , 90◦ and 135◦ angles only. However in reality, the set of angles
154
M. Bielecka et al.
between parts of contours will be much bigger. Therefore, some kind of fuzzy representation of the angles might help improve robustness and portability of the proposed methodology.
4
Concluding Remarks
As, has been presented, transducer introduced by Jakubowski and modified in this paper can be used to distinguish contour of the healthy bone from contour of the bones with erosion and osteophyte. That kind of diversification is required to build an intelligent system for joint diseases diagnosis. In such system, the most important will be analysis of the highest level features such as: the presence and location of osteophyte, the presence and location of erosion and joint space narrowing. The first two features can be described using a special algebraic approach described in [1], what will be the topic of the next publication. To recapitulate, the final system will be hierarchical one, with the following levels (starting from the lowest to highest level): preprocessing [6,17,18], contour shape description and joint space width analysis [2,4], algebraic language for coding highest level features in syntactic way and expert system to diagnose joint diseases. It has to be noted, that the system will be used as an aid in radiological diagnosis of the hand radiographs.
References 1. Bielecka, M.: Syntactic segmentation of graph function type curves. Machine Graphics and Vision 16, 39–55 (2007) 2. Bielecka, M., Bielecki, A., Korkosz, M., Skomorowski, M., Wojciechowski, W., Zieliński, B.: Application of shape description methodology to hand radiographs interpretation. In: Bolc, L., Tadeusiewicz, R., Chmielewski, L.J., Wojciechowski, K. (eds.) ICCVG 2010. LNCS, vol. 6374, pp. 11–18. Springer, Heidelberg (2010) 3. Bielecka, M., Skomorowski, M., Bielecki, A.: Fuzzy syntactic approach to pattern recognition and scene analysis. Intelligent Control Systems and Optimization, Robotics and Automation 1, 29–35 (2007) 4. Bielecka, M., Skomorowski, M., Zieliński, B.: A fuzzy shape descriptor and inference by fuzzy relaxation with application to description of bones contours at hand radiographs. In: Kolehmainen, M., Toivanen, P., Beliczynski, B. (eds.) ICANNGA 2009. LNCS, vol. 5495, pp. 469–478. Springer, Heidelberg (2009) 5. Bielecki, A., Korkosz, M., Wojciechowski, W., Zieliński, B.: Identifying the borders of the upper and lower metacarpophalangeal joint surfaces on hand radiographs. In: Rutkowski, L., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2010. LNCS, vol. 6113, pp. 589–596. Springer, Heidelberg (2010) 6. Bielecki, A., Korkosz, M., Zieliński, B.: Hand radiographs preprocessing, image representation in the finger regions and joint space width measurements for image interpretation. Pattern Recognition 41(12), 3786–3798 (2008) 7. Jakubowski, R.: Syntactic characterization of machine parts shapes. Cybernetics and Systems 13, 1–24 (1982) 8. Jakubowski, R.: Extraction of shape features for syntactic recognition of mechanical parts. IEEE Transactions on Systems, Man and Cybernetics 15(5), 642–651 (1985)
Application of Shape Description
155
9. Jakubowski, R.: A structural representation of shape and its features. Information Sciences 39, 129–151 (1986) 10. Jakubowski, R., Bielecki, A., Chmielnicki, W.: Data structure for storing drawing being then analysed for purposes of CAD. Archiwa Informatyki Teoretycznej i Stosowanej 1, 51–70 (1993) 11. Liang, M., Esdaile, J., Klippel, J., Dieppe, P.: Impact and Cost Effectiveness of Rheumatologic Care in Rheumatology. Mosby International, London (1998) 12. Ogiela, M.R., Tadeusiewicz, R., Ogiela, L.: Image languages in intelligent radiological palm diagnostics. Pattern Recognition 39, 2157–2165 (2006) 13. Sharp, J., Gardner, J., Bennett, E.: Computer-based methods for measuring joint space and estimating erosion volume in the finger and wrist joints of patients with rheumatoid arthritis. Arthritis & Rheumatism 43(6), 1378–1386 (2000) 14. Szczeklik, A., Zimmermann-Górska, I.: Injury Disease (in Polish). Medycyna Praktyczna, Warszawa (2006) 15. Tadeusiewicz, R., Ogiela, M.R.: Medical image understanding technology. Studies in fuzziness and soft computing. Springer, Heidelberg (2004) 16. Tadeusiewicz, R., Ogiela, M.R.: Picture languages in automatic radiological palm interpretation. International Journal of Applied Mathematics and Computer Science 15(2), 305–312 (2005) 17. Zieliński, B.: A fully-automated algorithm dedicated to computing metacarpophalangeal and interphalangeal joint cavity widths. Schedae Informaticae 16, 47–67 (2007) 18. Zieliński, B.: Hand radiograph analysis and joint space location improvement for image interpretation. Schedae Informaticae 17/18, 45–61 (2009)
