VISUAL SIGNAL PROCESSING AND IMAGE. UNDERSTANDING IN BIOMEDICAL SYSTEMS. Marek R. Ogiela, Ryszard Tadeusiewicz. University of Mining and ...
VISUAL SIGNAL PROCESSING AND IMAGE UNDERSTANDING IN BIOMEDICAL SYSTEMS Marek R. Ogiela, Ryszard Tadeusiewicz University of Mining and Metallurgy, Institute of Automatics Al. Mickewicza 30, PL-30.059 Krakow, Poland interpretation and semantic understanding of this type of data. These features result in that the linguistic approach, presented in the further part of this paper, used to create universal systems supporting decision-taking processes have wide application in clinical practice; in the h t w e it will find frequent application in tasks of medical diagnosis support and machine understanding of image data.
ABSTRACT The paper deals with theoretical and application-related issues connected with possibilities to create intelligent bio-medical and multimedia systems based on graph linguistic formalisms, originating from A1 methods. Such systems can be directed at tasks supporting medical diagnosis, multimedia indexation as well as classificationand cognitive semantic perception of the content of medical images. The said possibilities can be obtained owing to the use of structural and semantic reasoning processes in such systems, conducted pursuant to linguistic rules of aPPropriatelY defined graph and sequential grammars, describing the semantics of selected MRI and RTG images.
The approach prepared by authors assumes the use of graph and languages to create meaning descriptions of the analysed structures, c a v i n g important information relating to the image semantics of pathological lesions and context information specifying the neighbourhood of the analysed object. The analysis conducted by the system is based on parsing of languages defining the semantics of such images; apart from a classification of a given image it leads also to deeper reasoning specifying the essence of the analysed lesions and medical conclusions resulting from them. A synthesis of the thus obtained information allows one also to create effective mechanisms used in indexation tasks and in searching for specialist image data in scattered General scheme of such analysis is presented in Fig. 1.
1. INTRODUCTION This paper presents new opportunities of linguistic algorithm applications in the field of artificial intelligence, used to create intelligent biomedical systems. These systems are applied in various tasks supporting decisions taken in the area of Medical Imaging. Such systems, in particular Decision Support and Multimedia Systems can be based on the methods of structural analysis of medical imaging and are directed at offering possibilities of automatic
INTELLIDEN7 B!DMLDIEAL INFORMATION SYSTEMS
. .-
.,,.~ . . . . ~
'._
_,,'
,'
'''.
' .,""' ezowm
..{
'.
sGantic iiieipret;ti&,& ,
___" -Medical Co@"sians
,,
.'
i
' ,
..
.
,
\.
'. ....
Figure 1, General schema of medical panem analysis in Biomedical Information Systems On the lefl: images of spinal cord In the case of machine understanding, the information biomedical systems can specify the semantics of the image by means of syntactic reasoning, executing appropriate, defined semantic procedures. This is the approach proposed
0-7803-7761-3/03/$17.00 02003 IEEE
by the authors to create specialist DSS systems. These systems were prepared for interpretation tasks of many diseases of various organs, including ureter calculi, pancreas neoplasm and myocardial ischaemia [5,6].
V-17
In this paper we shall present an extension of the applications used so far onto a class of MRI images of the spinal cord and RTG images of renal sinuses. The general operation chart of such systems is presented on Fig. 1.
2. INTELLIGENT BIOMEDICAL SYSTEMS IN INTERPRETATION OF MRI VISUALIZATIONS
The general approach in the creation of such systems is the initial interpretation and specification of features, conducted by means of image pre-processing, with a view to detect features important for the analysed objects on a selected type of medical images. These features are subsequently described with the use of a formal language generated by an appropriately defined attributed grammar. Properties described in this way can be later reproduced in the course of structural reasoning conducted by the prepared information system.
While presenting methods of image understanding used in Intelligent Information Systems, we shall relate to examples of medical images. Application of structural pattem analysis methods to extract important semantic information from images will be shown on the example of analysis of spinal cord images.
The main advantage of this approach is its possibility to interpret the meaning of a much bigger class of images than the ones, which were used for the writing of the formal language. This results from the fact that the used grammar rules generalise the descriptions introduced and allow one to interpret new cases, previously not defined.
For a such interpretation of the mentioned structures and for a verification of lesion advancement level, a graph grammar of type EDT [7],and an attributed context-free grammar of type LALR ( I ) and have been proposed. These methods have been applied to describe changes in the width of different structures, visible in graphs. These graphs are obtained thanks to applying at the image pre-processing stage, a straightening transformation, which enables the production graphs of straightened structures, while preserving the morphological changes occurring in them. Before coming to the stage of perception of the changes it is necessary to preserve the sequence of basic image operations [4]. The aim is to obtain width profiles, which show the pathological changes occurring in analysed structures. During the initial analysis the following operations are executed: segmentation and skeletonisation, and application of a straightening transformation to transform the contour of the analysed structure in twodimensional space into one-dimensional graph form, which shows a profile of a straightened organ. Details of this operation and the advantages resulting from the application of this transformation are presented in [6]. To define the primary components on the obtained width graphs as well as terminal linguistic symbols describing these components, an algorithm of line approximation was used [SI. As a result, for every graph we received a sequence of vectors approximating its external borders. Next, to every of the thus determined sectors, terminal symbols are assigned, depending on the angle of its slope. In every case the result of this operation is a received sequence of terminal symbols, which are the input to syntax analysers.
In the case of analysis of backbone and spinal cord MRI images, the recognition objective is to detect and diagnose lesions, which could evidence a whole range of various disease units: beginning with myelomeningocele, numerous forms of inflammatory conditions or cerebral or spinal cord ischaemia, ending with most serious cases of intra- and extradullary tumours. Un unambiguous identification of all units with the use of one recognising software is extremely difficult due to rather subtle differences, decisive for the correct classification of every one of them. As it tums out, however, structural analysis can tum out to be extremely useful in the specification of the degree of the disease unit development by means of specifying the size of lesions in the morphology of the cord and by defining the compression of the spinal cord and meninges [Z].
For analysis of this structure a context-free grammar was prepared. It allows us to identify the symptoms, and to draw diagnostic conclusions conceming the essence of the visible pathology. The general form of the context-free grammar is the following: G = (VN, VT, SP, STS), where VN - set of non-terminal symbols, VT - set of terminal symbols, SP set of production, STS - starting symbol of grammar. In the case of analysis of spinal cord image, the grammar was defined in the following way: VN = {LESION,NARROWING, ENLARGEMENT, H, E, NI
VT= (h, e , n ) forhs[-IO", lO"],ea(lO", 180°),ne(-10",-180") STS = LESION
SP:
LESION + ENLARGEMENT Lesion=enlargement LESION +NARROWING Lesion=narrowing ENLARGEMENT -) E H N 1 E N I E H NARROWING -1NHEINEINH . H+h/hH w.+=w,,+w,; h,,:=h,,+h, E+eleE ... N+nlnN ... This grammar permits us to detect different forms of narrowing and enlargements, which characterize the different disease units (neoplasm or inflammation processes).
v-18
yCALIX-MJR)
Using attributes permits to calculate the numerical parameters of detected morphological changes which allows us to characterize the degree of lesion development. The simplicity of this grammar results mainly from the big generation capacity of context-free grammars, understood mainly as possibilities to describe complex shapes by means of a small number of introductory rules, that is grammar productions. A completely different situation occurs in the analysis of pathological lesions in the main pancreatic ducts, where a bigger number of symptoms and the variety of their shapes result in a necessity to introduce a more complex grammar [5]
3. GRAPH-BASED ANALYSIS OF VISUAL DATA An example of understanding the gist of morphological lesions of the shape wit the use the graph grammars will be presented on an actual task in which an analysis of the correctness of the renal pelvis and renal calyxes is to be made on the visualised urograph images (Fig. 2). For this purpose a grammar of the EDT class has been proposed [5].
+ PELVIS-REN(xCALIX-MJR + PELVIS-REN(xCAL1X-MJR + PELVIS-REN(yCALIX-MJR
yCALIX-MJR ZCALIX-MJR) KALIX-MJR) zCALIX-MJR)
2. CALIX-MJR + calix-mjr(yCALIX-MIN CALIX-MIN) + calix_mjr(xCALIX_MINKALIX-MIN ICALIX-MIN) + calix-mjr(xCALIX-MIN gCALIX-MIN) 4 calix-mjr(xCALIX-MIN nCALIX-MIN) 3. CALIXMIN
+ calix-minhapilla zppapilla)
+ calix-min(xpapil1a) I calix-min(vpapilla) I calix-min(zpapilla) + calix-min(xpapilla ypapilla) I calix-min(xpapil1a zpapilla)
The first group of production defines the different kinds of normal renal pelvis i.e. having two or three smaller calyxes. The succeeding productions define the form of bigger calyxes formed from two or more smaller calyxes. The last group defines the proper form of renal papillae, which obtains a fork form during the skeletonisation, which means it finishes with short branches, which arise only when it is concave to the interior of a smaller calyx. Convex forms during skeletonisation are thinned to the line without end branches, which results from the properties of skeletonisation algorithms.
4. RESULTS
Figure 2. Attempt at automatic understanding of the shape of renal pelvises. Urogram of healthy renal pelvis and calyxes together with skeleton and graph description obtained after approximating of
skeleton
To analysis and understanding these structures it will be suggested an expansive graph grammar defined with the aim of analysing the skeleton morphology of investigated renal pelvises and renal calyxes (Fig. 2). For such analysis the following graph grammar was used: GEo,=(Z. r, r, P, Z), where Z = CN U XT is a set of terminal and non-terminal vertex labels, r is a function which assigns to the graph vertex the number of its consequents, Z is a finite set of starting graphs, - is a set of edge labels, and P- is a set of production.
r
Owing to the application of the presented grammars it is possible to interpret effectively various kinds of lesions of investigated structures. In the case of syntactical analysis using such grammars, a recognition program delivers complete information concerning the visual morphological irregularities and the semantic information connecting them. Analysis of the morphological changes was carried out on the basis of a set of dozens of images (spinal cord images and also, in earlier research urogram, coronograms and ERCP images). For the presented grammars parser were constructed to serve the syntactical analysis. Because of analysis of these images, in each case we have obtained the kind of recognized lesion and a sequence of production numbers, which lead to grammar derivation of shape description of such lesions. Such sequences create a proper description of analysed shapes and have been stored in indexing record for multimedia searches.
The results obtained owing to the application of the characterized methods, confirm the immense opportunities &= {pelvis-ren, calix-mjr, calixmin, papilla) offered by syntactic methods in the diagnosis of medical ZN= {PELVIS-REN, CALIX-MJR, CALIX-MIN) r =(x, y , ~ forye[-30",30"],xe(30", ) I X O " ) , Z E ( - ~ ~ ~ , - I X O ~ ) visualizations with pathological lesions. The efficiency of recognition and mining of information important for the Z = {PELVIS-REN} analysed images, with semantic character exceeded the P- is a set ofproduction: threshold of 93%. In Fig. 3 presented examples which show I . PELVIS-REN 4 PELVIS-REN(xCAL1X-MJR
v-19
description ofthe lesions in question for pancreatic duct and spinal cord images. The recognition process is based on
production's numbers generated as output string by syntax analysers. Recognized symptoms are marked by a hold line.
. Figure 3. Results of disease symptom interpretation in Biomedical Diagnosis Support System
5. CONCLUSION As a result of research conducted on possibilities of application of linguistic algorithms o f syntactic pattem analysis and classification, it was possible to construct intelligent biomedical information systems allowing to recognize, and understand important lesions visible on analysed patterns. These methods have turned out to be not only very efficient in direct morphological deformation recognition on various medical images, but also allow to introduce very effective methods o f creation meaning description of the examined images as well as semantic representation. Research conducted proves that systems discussed io this paper enable such an intelligent pattem understanding. Owing to this, they allow to recognise efficiently pathological lesions of dangerous diseases. Understanding the image content may be used for an effective support to the diagnostic process. The fact of automatic understanding of the image content can have numerous further applications: for example such information can be used to monitor the therapeutic processes or to forecast the disease development as well as the patient's future state. What is more, by gaining an efficient method guaranteeing automatic understanding of the medical image content, we can treat results obtained with this method as special descriptions indexing images in a big multimedia database. Preparing of this form of indexation, first for images stored in a data base and later also for images in the form of queries addressed to that data base will allow one to enhance the process of context search for image information. Currently this process of context-data search in multimedia databases is considerably hampered.
6. ACKNOWLEDGEMENT This work was supported by University of Mining and Metallurgy under Grant No. 10.10.120.39.
7. REFERENCES [I] Bankman I . , editor. Handbook of Medical Imaging: Processing m d Andysis, Academic Press. San Diego. 2002. R.Di//PrrenriaIdiag,~osis in Magnefic Resonance Iniaging, Thieme, Stuttgan.2001
[ 2 ] Burgener F . A . ,Meyers S.P.,and Tan
[3] Javidi B., editor. Image Recognilion and Classrfication.
Marcel Dekker, New York, 2002. [4] Leondes C. T., editor. Image processing and recognifion, Academic Press, San Diego, 1998.
pattern
[ 5 ] Ogiela M. R., and Tadeusiewicz R. "Syntactic pattern
recognition for X-ray diagnosis of pancreatic cancer", IEEE Engineering In Medicine and Biology Magazine, / 9 : 6 :94.105, 2000.
[ 6 ] Ogiela, M. R. and Tadeusiewicz R. "Syntactic reasoning and pattern recognition for analysis of coronary artery images", Artijicial Inrelligence in Medicine, 26: 145-159, 2002.
[7] Ogiela M. R., Tadeusiewicz R., and Ogiela L. "Syntactic Pattem Analysis in Visual Signal Processing and Image Understanding", Proc. of the Inrernarional Conference on Fundamenlals of Electronic, Communications and Computer Science-lCFS2002.Toky0, Japan, 2002,pp.l3:10- 13:14.
v-20
