Generating Population Specific Shape Models of the

Aus dem Institut für Biomedizinische Bildanalyse

Generating Population Specific Shape Models of the Middle Ear Cavity

Masterarbeit zur Erlangung des Titels „Master of Science Medizinische Informatik“ der Privaten Universität für Gesundheitswissenschaften, Medizinische Informatik und Technik

vorgelegt von Omar Alshafi, Dr. aus dem Sudan

Hall in Tirol, 2004

Betreuer und erster Referent: Zweiter Referent: Annahme durch Prüfungssekretariat am von

2

TABLE OF CONTENTS TABLE OF FIGURES .......................................................................................................................... 6 REMARKS ............................................................................................................................................ 8 ACKNOWLEDGEMENT .................................................................................................................... 9 LIST OF ABBREVIATIONS............................................................................................................. 10 GLOSSARY ......................................................................................................................................... 11 0 INTRODUCTION ............................................................................................................................ 12 1 STATE OF RESEARCH ................................................................................................................. 13 1.1 ANATOMY OF THE MIDDLE EAR ................................................................................................... 13 1.1.1 CONCEPT OF THE ANATOMY ........................................................................................................ 13 1.1.2 Regional Anatomy of the Human Ear .......................................................................................... 13 1.1.3 THE MIDDLE EAR (CAVUM TYMPANI; EARDRUM; TYMPANUM)................................................. 14 1.1.4 THE OSSICLES ............................................................................................................................... 15 1.1.5 THE MUSCLES............................................................................................................................... 15 1.1.6 WALLS OF THE TYMPANIC CAVITY .............................................................................................. 16 1.2 IMAGE SEGMENTATION ................................................................................................................. 17 1.2.1 CONCEPT AND DEFINITION:.......................................................................................................... 17 1.2.2 OBJECTIVES OF SEGMENTATION: ................................................................................................. 17 1.2.3 MEDICAL APPLICATIONS:............................................................................................................. 18 1.2.4 METHODS AND TECHNIQUES:....................................................................................................... 18 1.2.4.1 Image Segmentation using Region Based Techniques.............................................................. 18 A. Thresholding..................................................................................................................................... 18 B. Clustering.......................................................................................................................................... 19 C. Region Growing............................................................................................................................... 19 D. Region Splitting................................................................................................................................ 20 E. Region Splitting and Merging.......................................................................................................... 20 1.2.4.2 Image Segmentation using Edge Based Techniques ................................................................. 20 1.2.4.3 Image Segmentation using Deformable Techniques ................................................................. 20 A. Deformable Models .......................................................................................................................... 21 B. Deformable Organisms..................................................................................................................... 21 C. Deformable M-Reps ......................................................................................................................... 21 1.3 IMAGE MODELING ......................................................................................................................... 22 A. MODELING ........................................................................................................................................ 22 B. MEDIAL REPRESENTATION ............................................................................................................... 22 1.3.1 M-REPS AND MEDIAL ATOM ......................................................................................................... 22 1.3.1.1 Definition of m-reps .................................................................................................................. 22 1.3.1.2 The geometry of medial atom: hubs with spokes ..................................................................... 23 1.3.1.3 The single-figure medial mesh .................................................................................................. 24 1.2.1.4 Connectivity and topology ........................................................................................................ 25 1.3.2 FIGURAL HIERARCHIES ................................................................................................................. 26

3

1.3.3 INTERPOLATION OF MEDIAL ATOMS ............................................................................................. 26 1.3.4 ADVANTAGES AND LIMITATIONS OF M-REPS: .............................................................................. 26 1.3.4.1 Advantages ................................................................................................................................ 26 1.3.4.2 Limitations................................................................................................................................. 27 2 OBJECTIVES................................................................................................................................... 28 2.1 SEGMENTATION ............................................................................................................................. 28 2.2 MODELING ...................................................................................................................................... 28 2.3 MEAN MODEL: ............................................................................................................................... 28 2.4 EVALUATION: ................................................................................................................................. 28 3 METHODS ....................................................................................................................................... 29 3.1 CT IMAGES ..................................................................................................................................... 29 3.2 AMIRA 3.0 ....................................................................................................................................... 29 3.2.1 OVERVIEW .................................................................................................................................... 29 3.2.2 SEGMENTATION EDITOR ............................................................................................................... 30 3.2.3 HOW TO SEGMENT IN AMIRA ........................................................................................................ 30 3.3 PABLO ............................................................................................................................................. 32 3.3.1 DEFINITION ................................................................................................................................... 32 3.3.2 OBJECT HIERARCHY ..................................................................................................................... 32 3.3.3 THE FITTING OF THE MODEL TO THE IMAGE ................................................................................. 33 3.4 MATHEMATICA .............................................................................................................................. 36 3.5 PCA - PGA ..................................................................................................................................... 36 3.5.1 PCA DEFINITION .......................................................................................................................... 36 3.5.2 PCA APPLICATIONS ..................................................................................................................... 37 3.5.3 THE STEPS OF PERFORMING PCA................................................................................................. 37 3.5.4 GENERALIZATION OF PCA ........................................................................................................... 38 3.6 MATLAB ....................................................................................................................................... 39 4 RESULTS.......................................................................................................................................... 40 4.1 THE MEAN MODEL ........................................................................................................................ 40 4.2 DISTRIBUTION OF THE MEAN MODEL .......................................................................................... 41 4.3 VARIATIONS.................................................................................................................................... 43 4.3.1 FIRST PRINCIPAL COMPONENT ..................................................................................................... 43 4.3.2 SECOND PRINCIPAL COMPONENT ................................................................................................. 43 4.3.3 THIRD PRINCIPAL COMPONENT .................................................................................................... 44 4.3.4 FOURTH PRINCIPAL COMPONENT ................................................................................................. 45 4.3.5 FIFTH PRINCIPAL COMPONENT ..................................................................................................... 45 4.4 EVALUATION WITH COMPARISON ................................................................................................ 46 4.4.1 WITH MATHEMATICA ................................................................................................................... 46 4.4.2 WITH MTLAB............................................................................................................................... 50 5 DISCUSSION ................................................................................................................................... 51 5.1 PROBLEM OF MANUAL SEGMENTATION ...................................................................................... 52 5.2 MODEL VARIATION: VALIDITY AND QUALITY ............................................................................ 52 5.3 PROBLEMS ...................................................................................................................................... 54 5.4 OUTLOOK ....................................................................................................................................... 54

4

6 SUMMARIES ................................................................................................................................... 55 6.1 ABSTRACT....................................................................................................................................... 55 6.2 ZUSAMMENFASSUNG ...................................................................................................................... 56 REFERENCES .................................................................................................................................... 57

5

TABLE OF FIGURES Figure 1.1: Anatomy of the Human Ear [4]...................................................................................................... 13 Figure 1.2: The Medial Wall and Part of the Posterior and Anterior Walls of the Right Tympanic Cavity, Lateral View. [25] ...................................................................................................................................... 14 Figure 1.3: Middle Ear – Ossicles [26]:-[1] Head, [2] Malleus, [3] Handle, [4] Tympanic Membrane, [5] Stapes, [6] Base (foot plate), and [7] Incus. ............................................................................................. 15 Figure 1.4: Medial Atom (for an internal mesh position, implying two boundary sections) and Medial End-Atom (for a mesh edge position, implying a section of boundary crest)....................................... 24 Figure 1.5: Three [a] 8x3, [b] 5x3 and [c] 7x3 m-rep Models (yellow atoms) ................................................ 25 Figure 1.6: Examples of Medial Mesh Topologies............................................................................................ 25 Figure 3.1: CT Image of Tympanic Membrane................................................................................................ 29 Figure 3.2: How to Select Segmentation Editor................................................................................................ 30 Figure 3.3: Slice 161 without and with Segmentation ...................................................................................... 31 Figure 3.4 [a]: SurfaceGen and SurfaceView ................................................................................................... 31 Figure 3.4 [b] Surface Reconstruction of a Tympanic Membrane ................................................................. 31 Figure 3.5: The Adjustment of Display Control............................................................................................... 33 Figure 3.6[a]: The Model before and during the Manual Initialization Process ........................................... 34 Figure 3.6[b]: Object Ensemble Stage............................................................................................................... 34 Figure 3.7: The Model [a] after the manual initialization and [b] after the Automatic Fitting Process...... 35 Figure 3.8: Atom Stage Process ......................................................................................................................... 36 Figure 4.1: Atom Grids (5x3) with Hidden and Wire Frame Surface............................................................ 40 Figure 4.2: Common Shape Model of the Population with Solid and Wire Frame Surface......................... 40 Figure 4.3: Distribution of the I and II Principal Components....................................................................... 41 Figure 4.4: Distribution of Eigenvalues (Principal Components)................................................................... 42 Figure 4.5: Variations from the First Principal Component (left: -2σ, middle: mean, right: +2σ).............. 43 Figure 4.6: Variations from the Second Principal Component (left: -2σ, middle: mean, right: +2σ).......... 44

6

Figure 4.7: Variations from the Third Principal Component (left: -2σ, middle: mean, right: +2σ)............ 44 Figure 4.8: Variations from the Fourth Principal Component (left: -2σ, middle: mean, right: +2σ).......... 45 Figure 4.9: Variations from the Fifth Principal Component (left: -2σ, middle: mean, right: +2σ) ............. 45 Figure 4.10: Distribution of the I and II Principal Components (First MM) ................................................ 46 Figure 4.11: Distribution of the I and II Principal Components (Second MM) ............................................ 47 Figure 4.12: Distribution of Eigenvalues (Principal Components) – First MM ............................................ 48 Figure 4.13: Distribution of Eigenvalues (Principal Components) – Second MM ........................................ 49 Figure 4.14: The Two Mean Models in Comparison (MM1-blue + MM2-red) ............................................. 51

7

Remarks

•

This work is written according to the American spelling and punctuation of the English language.

•

References are written with little changes according to the MLA style [1] and quoted in the text in numbers like [1] above.

•

[…] contains numbers of references, or numbers of parts of a figure.

•

(…) contains synonyms, explanations, figures, or abbreviations.

•

{…} used as mathematical set sign in this work.

8

Acknowledgement First of all I would like to express my bottomless gratitude to my supervisor Univ. Prof. Dr. Rainer Schubert for giving me the chance and support to write this work at IBIA. I would like also to thank my second tutor Dr. Karl Fritscher who has supported me during this work and was always very patient to answer all my questions. I express my thanks to Prof. Rudolf Leuwer; Department for ENT Surgery, University Hospital Eppendorf, University of Hamburg; who assisted us with his medical expertise and has provided us with the CT data of tympanic cavities. The members and students of IBIA were helpful and I would like to thank all of them for their friendly help. In particular, I would like to thank Mag. Walter Draxl, the chancellor of UMIT, who has supported me during this work and always raises my spirits.

“Medical real world analysis from images depends on both descriptions of the real world geometry and probability functions on these geometric descriptions.” Stephen Pizer [17]

9

List of Abbreviations M-Reps: Medial Representation MM: Mean Model (common shape model) 2D: 2-Dimentional 3D: 3-Dimentional CT: Computed Tomography MRI: Magnetic Resonance Imaging MMA: Multiscale Medial Axis PCA: Principal Components Analysis PGA: Principal Geodesic Analysis

MIDAG: Medical Image Display and Analysis Group IBIA: Institute for Biomedical Image Analysis UMIT: University for Health Sciences, Medical Informatics and Technology

10

Glossary •

An Object is a solid, tangible entity in the physical world.

•

A Shape is defined by [20] as: 1. corresponding landmarks and space warp with interpolation. 2. a high-dimensional warping between image data and application of the deformation to a segmented object template. 3. a parameterization of object surfaces. 4. an extraction of characteristic surface features. 5. an extraction of the medial axis and a graph description.

•

A Figure is a main component of an object or a protrusion, an indentation, a hole, or an associated nearby or internally contained object. It is also, in a simple definition, the part of the object associated with a particular sheet.

•

A Single Figure is a sheet of medial atoms, which is interpolated from the model formed by a net (i.e., a mesh or chain) of medial atoms (hence the name m-reps).

•

A Single-Figure Medial Mesh is the structural unit of the m-rep model and is made up of medial atoms linked in chains or grids -- the single-figure m-reps in 3D are made up of 1D chains or 2D lattices of medial atoms.

•

Figural Coordinate System is the whole object-intrinsic coordinate system.

•

Anatomy is the science of body structures and the relationships among structures.

•

Segmentation is spatial partitioning of an image into its constituent parts, or isolating specific objects in an image.

•

M-Reps are a multiscale medial means for modeling and rendering 3D solid geometry.

•

Pablo is a model-building and image-analysis research tool for applying m-reps to medical image segmentation and modeling, particularly directed at tasks in 3D CT and MRI data analysis.

11

0 INTRODUCTION Statistical shape analysis is a well-known approach [10] to study diseases as pathological conditions versus physiological controls or also to study variations of an anatomical structure among a population. The understanding of the biological variations and patho-anatomical changes would be improved by proper usage of anatomical shape information. Most diseases of the middle ear cavity are very likely to result from morphological alterations and they, consequently, change the anatomical form of the cavity. As the tympanic cavity is a hollow space within the tympanic bone, its form is constrained by the shape and growth process of this surrounding tympanic bone. Because the high resolution CT scans, however, reveals middle ear bony variations and abnormalities in great details, 3D shape models could be generated. These 3D shape models represent inter-individual and population-specific shape variations as a meaningful tool for future diagnosis and surgical planning. Medial representation method (m-reps) which is developed by Pizer and his colleagues at MIDAG [15] would be a suitable approach for the generation of these shape models. M-reps is a straight forward method to stablish geometric correspondence between the individuals of a population without altering the individual shape. Based on previous experiences with m-reps, one of them is a study at IBIA [10], the purpose of this study is to build a control model (common shape model or mean model) based on a healthy population to see the suitability of m-reps method to generate a mean model of tympanic cavity and the ability of that mean model in analysis of specific and characteristic shape features and variations. The study has been performed by segmenting the individuals of tympanic cavities and modeling them and the result was a mean model of these individuals. Objectives, methods, results and discussion of this work will be presented in the following pages.

12

1 STATE OF RESEARCH 1.1 Anatomy of the Middle Ear 1.1.1 Concept of the Anatomy Anatomy [4] is the science of structures and the relationships between them with the help of dissection, microscope, and a variety of imaging techniques. Moreover, anatomy can be classified into many disciplines like macroscopic and microscopic, systemic and regional, radiological and the recent emerging [24] computational anatomy. For this work on the middle ear, it is the radiographic anatomy to be concerned (refer to section 3.1) and the computed tomography has been used as imaging modality.

1.1.2 Regional Anatomy of the Human Ear The ear [4], or organ of hearing, is divided into three main regions: the external ear, which collects sound waves and channels them inward; the middle ear or tympanic membrane, which conveys sound vibrations to the oval window; and the internal ear or labyrinth, which houses the receptors for hearing and equilibrium.

Figure 1.1: Anatomy of the Human Ear [4]

13

1.1.3 The Middle Ear (Cavum Tympani; Eardrum; Tympanum) The middle ear [4] is a small irregular air-filled cavity within the temporal bone that is lined by epithelium. Therefore it is not a grown organ in a common sense, but its form is constrained by the shape and growth process of the surrounding tympanic bone. It is separated from the external ear by the eardrum and from the internal ear by a thin bony partition that contains two small membrane-covered openings: the oval window and the round window.

Figure 1.2: The Medial Wall and Part of the Posterior and Anterior Walls of the Right Tympanic Cavity, Lateral View. [25]

14

1.1.4 The Ossicles Extending across the middle ear and attached to it by ligaments are the three smallest bones in the body, the auditory ossicles, which are connected by synovial joints. The bones, named for their shapes, are the malleus, incus, and stapes – commonly called the hammer, anvil, and stirrup, respectively. The “handle” of the malleus attaches to the internal surface of the eardrum. The head of the malleus articulates with the body of the incus. The incus, the middle bone of the series, articulates with the body of the stapes. The base or footplate of the stapes fits into the oval window. Directly below the oval window is another opening, the round window, which is enclosed by a membrane which is called the secondary tympanic membrane (see Figure 1.3).

Figure 1.3: Middle Ear – Ossicles [26]:-[1] Head, [2] Malleus, [3] Handle, [4] Tympanic Membrane, [5] Stapes, [6] Base (foot plate), and [7] Incus.

1.1.5 The Muscles The muscles which are attached to the ossicles are two tiny skeletal muscles, the tensor tympani muscle and the stapedius muscle. The tensor tympani muscle limits movement and increases tension on the eardrum to prevent damage to the inner ear from loud noises. The innervation of this muscle is through the mandibular branch of the trigeminal nerve (cranial nerve V). The stapedius muscle, which is the smallest of all skeletal muscles and innervated by the facial nerve (cranial nerve VII), protects the oval window by dampening large vibrations of the stapes due to loud noises, but it also decreases the sensitivity of hearing.

15

1.1.6 Walls of the Tympanic Cavity The tympanic cavity can be compared [5] with a narrow room (3-6 mm broad, volume 0.8 ml) which is occupied by the three tympanic bones.

Anatomic Terminology: Paries –pl. parietes- is defined as the wall of a cavity or hollow organ [2].

1. Roof: Paries tegmentalis. 2. Floor: Paries jugularis.

Epitympanic Cavity. Hypotympanic Cavity.

3. Medial Wall: Paries labyrinthicus. 4. Lateral Wall: Paries membranaceus 5. Anterior Wall: Paries caroticus. It contains an opening that leads directly into the auditory (pharyngotympanic) tube, commonly known as the Eustachian tube. 6. Posterior Wall: Paries mastoideus. Medial, lateral, anterior, and posterior walls are the borders of

Mesotympanic Cavity.

16

1.2 Image Segmentation Image segmentation is a broad and active field of research not only in medical imaging but also in a variety of other fields such as computer vision or satellite imagery. In this section, a general precise review about segmentation will be provided including the concept and definition (1.2.1), objectives of segmentation (1.2.2), medical applications (1.2.3), and methods and techniques (1.2.4) of medical image segmentation.

1.2.1 Concept and Definition: Image segmentation is a very broad term, and is used in many different ways in imaging science, particularly in image processing. But usually the general concept of segmentation means one of two things: spatial partitioning of an image into its constituent parts, or isolating specific objects in an image. Therefore, the image segmentation, defined by subdivision of image into regions (called also classes and subsets), is a very important area in image processing, which often has a large impact on image analysis and interpretation. The level to which the subdivision is carried depends on the problem being solved. Klaus Engelke et al. [12] have described segmentation as an integral part of image processing. There is no general agreement among authors regarding where image processing stops and image analysis {also called image understanding} starts, but sometimes, as suggested by Gonzalez [6], a distinction is made by defining the image processing as a discipline in which both the input and output of a process are images while the inputs in image analysis are generally images but the outputs are attributes extracted from those images. The area of image analysis is in between image processing and computer vision and the logical place of overlap between image processing and image analysis is the area of recognition of individual regions or objects in an image. Since segmentation requires classification of pixels, it is often treated as a pattern recognition problem and addressed with related techniques. Image classification, not to be confused with segmentation, means identifying what an object in an image is, or what type of object each pixel belongs to.

1.2.2 Objectives of Segmentation: The definition of the goal of segmentation varies according to the goal of the study and the type of the image data. The principal goal of segmentation is to divide an image into regions that are homogenous with respect to one or more characteristics or features. Therefore, it may be useful to classify image pixels into anatomical regions, such as bones muscles and blood vessels, or into pathological regions, such as cancer, tissue deformities, and multiple sclerosis lesions.

17

In some studies the goal is to divide the entire image into subregions such as the white matter, gray matter, and cerebrospinal fluid spaces of the brain, while in others one specific structure has to be extracted, for example breast tumors from magnetic resonance images. Segmentation can also be used as an initial step for visualization and compression.

1.2.3 Medical Applications: Segmentation is an important tool in medical image processing and it has been useful in many medical applications. Some of these applications are detection of the coronary border in angiograms, multiple sclerosis lesion quantification, surgery simulations and planning, measuring tumor volume and its response to therapy, and functional mapping. For more medical applications refer to [7] but generally in medical imaging, segmentation is important for feature extraction, image measurements, and image display.

1.2.4 Methods and Techniques: The most commonly used segmentation methods can be classified into two broad categories: I.

Region-based segmentation methods that look for the regions of special characteristics and they include pixel-oriented, texture-oriented, object-oriented, and scene-oriented methods, and

II. Edge-based segmentation methods that look for edges between regions with different characteristics. Although there are a wide variety of segmentation techniques, there is no one standard segmentation technique that can produce satisfactory results for all imaging applications. Segmentation techniques can be divided into classes in many ways, depending on classification scheme: 1. Manual, semiautomatic, and automatic techniques. 2. Pixel-based, region-based, and edge-based techniques. 3. Classical and deformable models = (parametric, geometric) techniques. 4. Statistical, fuzzy, and neural network techniques. 5. Volumetric segmentation and partial volume segmentation techniques.

1.2.4.1 Image Segmentation using Region Based Techniques Region-based techniques segment an image into a number of regions starting from one or more seed points that based on definitive criteria of uniformity. Some examples of these criteria are mean intensity value, texture and color.

A. Thresholding Thresholding is a common region-based segmentation method. In this technique a threshold is selected and an image is divided into groups of pixels with values greater or equal to the threshold. A

18

single threshold can segment the image into only two regions, a background and a foreground. The Thresholding methods include, at least, global methods based on gray-level histograms, global methods based on local properties (e.g., local mean value and standard deviation), local threshold selection, and dynamic thresholding. Thresholds are global or local; global by selecting only one threshold, based on the image histogram, for the entire image and local by selecting a threshold that depends on local properties of some image regions. If the local thresholds are selected independently for each pixel (or groups of pixels), thresholding is called dynamic or adaptive. Computationally, global thresholding is simple and fast while the local one is more expensive. Global thresholding works well on images that contain objects with uniform intensity values on contrasting background. However, it fails if there is a low contrast between the object and the background, if the image is noisy, or if the background intensity varies significantly across the image. Local thresholding can be determined by splitting the image or examining the threshold. Splitting the image is done by dividing the image first into rectangular overlapping subimages which should be large enough to include both objects and background pixels, and then calculating the histograms for each subimage. Examining the image intensities in the neighborhood of each pixel is performed by selecting a threshold using the mean value of the local intensity distribution or other statistics.

B. Clustering Clustering [7] achieves region segmentation by dividing the image into sets or clusters of pixels that have strong similarity in the feature space. Fuzzy clustering is the oldest fuzzy approach to image segmentation. Algorithms such as fuzzy c-means (FCM), possibilistic c-means (PCM) and k-means can be used to build clusters (segments).

C. Region Growing Unlike the thresholding which focuses on the difference of pixel intensities, the region growing, also called region merging, focuses on groups of pixels with similar intensities. Region growing starts with a pixel or a group of pixels (called seeds) that belong to the structure of interest. The general algorithm of region growing consists of:•

choosing the seeds by an operator or an automatic seed finding procedure,

•

examining the neighboring pixels at a time and adding them to the growing region, if they are sufficiently similar based on a uniformity test,

•

continuing the procedure until no more pixels can be added, and

•

representing the object by all pixels that have been accepted during the growing procedure.

A very popular example for this algorithm is the one proposed by Adams and Bischof and an interesting modification of region growing technique called hill climbing was proposed by Bankman et al. [7] for detecting microcalcifications in mammograms.

19

Region growing has some problems like merging of the examining regions with the regions that do not belong to the object of interest, if the homogeneity criterion is improperly chosen. Therefore the results of region growing depend strongly on the selection of the homogeneity criterion. Another problem is that different starting points may not grow into identical regions. The advantage of region growing is that it is capable of correctly segmenting regions that have the same properties and are spatially separated. Another advantage is that it generates connected regions.

D. Region Splitting The steps of this algorithm are:•

It takes the opposite approach to region growing by starting from the entire image as one region,

•

If the entire image does not meet the homogeneity criteria, it is split into 4 subimages (or 8 in 3D),

•

The procedure is applied recursively on each subimage until every subimage meets the homogeneity criteria,

•

Then, after the completion of the procedure, the image can be represented as a quadtree (data structure in 2D in which each parent node has 4 children) or as an octree (data structure in 3D in which each parent node has 8 children).

E. Region Splitting and Merging As the main disadvantage of region splitting is that the final subimage may contain adjacent regions with identical properties, it is useful to combine the two approaches, region splitting and region growing to address this problem and to add together the advantages of both approaches.

1.2.4.2 Image Segmentation using Edge Based Techniques An edge or boundary on an image is defined by the local pixel intensity gradient. A gradient is an approximation of the first order derivative of the image function. The strategy of edge-based segmentation algorithms is to find object boundaries and segment regions enclosed by the boundaries. These algorithms could include the most gradient operators, graph searching, border tracing and contour following techniques.

1.2.4.3 Image Segmentation using Deformable Techniques “Deformable techniques” is a common term coined by this study to refer to the segmentation techniques that include deformable models, deformable organisms, and deformable m-reps which will be briefly discussed in this section.

20

A. Deformable Models The deformable models have become one of the most successful research areas in medical image analysis. Although deformable models were originally developed for applications to problems in computer vision and computer graphics, their potential for use in medical image analysis has been quickly realized. Terzopoulos [7] introduced the theory of continuous (multidimensional) deformable models in a Lagrangian dynamic setting. But the underlying theory of deformable templates in order to extract image features was introduced 15 years before by Fishler and Widrow. [10] The mathematical underpinnings of deformable models represent the interdisciplinarity of geometry to represent the object shape, physics to impose constraints on the variations of the shape over space and time, and approximation theory to provide the formal foundations of mechanisms for fitting the models to measured data. Deformable models are curves or surfaces that can move under internal forces, defined within the curve to keep the model smooth and external forces which are computed by image data in order to move the model toward an object boundary. Basically, deformable models can be divided into:•

Parametric deformable models: They represent curves and surfaces explicitly in their parametric forms during deformation with the advantage of direct interaction. However, changes in topology, such as splitting or merging parts during deformation are very difficult to implement by using parametric models. Generally there are two different possibilities to formulate them: energy minimizing formulation and dynamic force formulation.

•

Geometric deformable models: They are introduced by Caselles and Malladi [10] to overcome the problems of parametric deformable models like changes in topology. The geometric deformable models are, however, based on curve evolution theory and level set methods.

B. Deformable Organisms Deformable organisms is a new approach introduced to biomedical image analysis by McInerney et al. [19] and it combines deformable methodologies with concepts from the field of artificial life. Details about this technique are to be found in [19].

C. Deformable M-Reps Deformable m-reps are a good tool for modeling (refer to section 1.3). They are, however, developed for segmentation.

21

1.3 Image Modeling A. Modeling There are different surface and volume based methods for modeling as well as approaches based on finite element methods, active shape models (ASM), harmonic maps, which are used for shape modeling. Methods for representing objects in 2D and 3D images can be divided into two categories: the category that describes the boundary of the objects and the category that describes the inside of the objects. [16] The methods that describe the inside of objects have certain advantages of multilocality and they can be divided into three types [16]: the type of globally prescribed components (e.g. geons), the type of generalized cylinders, and the type of medial representations. Many researchers have preferred the medial representations because the other two types require arbitrary decisions on how to choose the axis and the cross-sectional forms.

B. Medial Representation Pizer et al. [15] introduced a medial based approach (m-reps) which provides a straight forward method to establish geometric correspondence between the individuals of a population without altering the individual shape. M-reps are a work in progress and they grew out of a work on MMAs (multiscale medial axis) and core-based image analysis in computer graphics. They are developed by Pizer and his colleagues (Fritch, Wilson, Chen, and Thall) in image analysis at MIDAG [9]. The principal idea behind them was to reverse the medial transform from a boundary implying a medial description to a mesh of medial atoms implying boundaries (i.e., from an unstable to a stable relation), and to define objects medially by allowing the medial structure to imply the boundaries. They are founded on the need for a multiscale representation for image-analysis based on geometric models.

1.3.1 M-reps and medial atom 1.3.1.1 Definition of m-reps M-reps are a multiscale medial means for modeling and rendering 3D solid geometry. The representation is based on figural models, which define objects at coarse scale by a hierarchy of figures {multi-object complex- object- figure- subfigure}.

22

By assuming from the start that a stable skeletal representation requires insensitivity to fine surface detail, m-reps divide the shape characteristics of an object into two parts: •

a coarse-scale, medial representation based on object width but with a width-proportional boundary tolerance, and

•

a fine-scale, boundary representation giving perturbations within the width-proportional tolerances at each surface location.

1.3.1.2 The geometry of medial atom: hubs with spokes M-reps are based on the medial atom, a discrete point on an implied medial locus which designates paired points (medial involutes) on the object boundary. These medial atoms are then grouped into a linear chain or planar mesh to define a single-figure m-rep, representing a discrete medial sampling of an implied solid object. As the medial atom is the basis of m-rep model, its geometry should be described. An m-rep for a generic figure is a 2-manifold of medial atoms. An interior medial atom is a medial position at which two vectors (called port and starboard sails) of equal length r share. Or in other words the medial atom is a discrete point built on an implied medial locus and it is defined as a 4-tuple m = {x, F, r, θ} consisting of: •

x ∈ R3 , the skeletal position or hub of the inscribed sphere and it gives the central location of the solid section of figure that is being represented by the atom,

•

r Є R+ , the local width defined as the radius of the sphere, which is the distance from the skeletal position to two or more implied boundary positions. r gives the scale of the solid section of figure that is being represented by the atom. Therefore, it provides a local ruler for the object.

•

F ∈ SO (3), an orthogonal local frame parameterized by (b, b┴, n), where n is the normal to the medial manifold, and b is the direction in the tangent plane of the fastest narrowing of the implied boundary sections. F gives the orientation of the solid section of figure that is being represented by the atom. That is, it provides a local figural compass for the object.

•

θ ∈ [0, π], the object angle determining the angulation of the implied sections of boundary relative to b. [15,13]

23

Figure 1.4: Medial Atom (for an internal mesh position, implying two boundary sections) and Medial End-Atom (for a mesh edge position, implying a section of boundary crest)

The medial atom implies two opposing boundary points, y0, y1, with respective boundary normals, n0, n1, which are given by n0 = cos ( θ ) b – sin ( θ ) n,

y0 = x + r n0,

n1 = cos ( θ ) b + sin ( θ ) n,

y1 = x + r n1

1.3.1.3 The single-figure medial mesh Pizer has defined the figure intuitively in [15] as “a main component of an object or a protrusion, an indentation, a hole, or an associated nearby or internally contained object”. In [16] he has defined the figure in 2D as “an unbranching section of an object with two related sides (sections of boundary) and an end (may be a cycle and no end) such that marching up the two sides monotonically, each towards the same end, respects the intuitive relatedness of points opposite each other on the two sides” A single figure [15] is a sheet of medial atoms, which is interpolated from the model formed by a net (i.e., a mesh or chain) of medial atoms (hence the name m-reps). Each atom models a solid region by a position and width and also a local figural frame giving figural directions and an object angle between opposing, corresponding positions on the boundary implied by the m-rep. An m-rep represents a single object as a set of connected meshes of medial atoms. There is a main figure and possibly a set of connected subfigures, each represented by a single quadrilateral (or other topological) mesh of medial atoms.

24

The single-figure medial mesh is the structural unit of the m-rep model and is made up of medial atoms linked in chains or grids -- the single-figure m-reps in 3D are made up of 1D chains or 2D lattices of medial atoms [9]

[a]

[b]

[c]

Figure 1.5: Three [a] 8x3, [b] 5x3 and [c] 7x3 m-rep Models (yellow atoms)

Multi-figure objects may then be created hierarchically by joining figures to one another as protrusions (additive subfigures), indentations (subtractive subfigures), or associated, neighboring figures.

1.2.1.4 Connectivity and topology Primitively and locally, medial structure can be categorized according to dimensional symmetry as spheres, whose symmetry is about a point; cylinders whose symmetry is about a 1D axis; and slabs whose symmetry is about a 2D sheet. Only the quad-figure (see Figure 1.6) is in active use in current m-reps research. For more information about connectivity and topology refer to [9].

Figure 1.6: Examples of Medial Mesh Topologies

25

1.3.2 Figural hierarchies A boundary displacement exceeding the allowable tolerance is represented as an attached subfigure because m-reps do not use a branching skeletal structure. This allows the object’s shape to be divided into several possible types of figural relationships: •

Figure-subfigure relationships, where the parent figure is the main body of the object;

•

Co-figure relationships, where no unique parent-child relationship is apparent;

•

Figure-self relationships, where, e.g., the “tail” of a figure may be attached as a protrusion from part of the same figure;

•

Figure-neighbor-figure relationships, where associated figures may be unattached to the main figure.

For more information and figures, refer to [9].

1.3.3 Interpolation of medial atoms To construct a boundary for a given sampled medial mesh, there are numerous methods which are based on the boundary involutes, surface normals and other information provided by the medial atoms. These methods include simple boundary tessellations based on involutes, implicit surface methods, fitting of spline-based or subdivision surfaces to the boundary, or adaptive refinement of boundary meshes based on interpolative-refinement of the medial mesh to generate a fine mesh of involutes. For more details about the interpolation of medial atoms, refer to [9].

1.3.4 Advantages and limitations of m-reps: 1.3.4.1 Advantages •

The multi-local nature of m-reps over local b-reps: The global shape information provided by the medial skeleta gives m-reps a figural basis of shape description much more directly than in b-rep (boundary representation) modeling. M-reps thus allow medial-based figural deformation and provide natural movement for articulated models.

•

The stability with tolerance versus traditional Blum medial axes: “Tolerance in m-reps is based on a statistical description of shape, with objects having a probabilistic nature—a mean description and a distribution of possible deviations from this mean [9]”. The 1D medial curves are non-generic primitives in Blum medial axes and because of this they are extremely intolerant of boundary noise and perturbation while the 1D chains in m-reps are generic primitives and tolerant—means that the boundary perturbations within tolerance do not affect the medial structure. Therefore, one of the advantages of m-reps is to distinguish object deformations into along-object deviations, namely elongations and bendings, and acrossobject deviations, namely bulgings and indentations.

26

•

Natural separation of object shape into intuitive, figural hierarchies: Because of their hierarchical nature, m-reps have advantages for animation and deformable modeling. The figural coordinates allow the figural hierarchy to impose its own coordinates for figurally-based shape description and deformation.

•

The power gained by using displacement subdivision techniques for fine-scale modelling: Displaced subdivision surfaces are an ideal representation for m-rep boundaries because they restrict boundary displacement to be within the desired medially implied tolerances and provide precise, fine-scale modeling of the surfaces within the framework of the coarse shape description. This allows coarse shape modifications to be made skeletally and independent of the fine-scale detail.

•

The stochastic model generation: The stochastic model generation is possible because of the m-reps` design as robust statistical shape descriptors. M-reps were designed, as tool in medical image analysis, to allow statistical description of shape variation across a population. Thus, an m-rep can be used to generate multiple instances of an object with natural variations in its structure.

1.3.4.2 Limitations •

Quadmeshes and their limitations: Despite the many advantages of regular quadrangular meshes, the necessity for a constant number of rows and columns is at least a theoretical and as well, perhaps, as a practical drawback. There are many objects that do not readily adapt to quadmesh representation.

•

Surface discontinuities: The medial skeleton of an m-rep alone implies a blobby object with corners, creases, and other first-order discontinuities by fuzziness of the representation. But this is not a problem in Pablo, which is a model-building and image-analysis research tool for applying m-reps to medical image segmentation and modeling, because most of the anatomical objects being modeled are somewhat blobby in nature.

•

The correspondence problem: Correspondence is not a problem for synthetic model generation but the users of m-reps in image analysis must face the problem of establishing correspondence between m-reps fitted to different members of a sampled population.

•

Discrete m-reps versus cm-reps: Thall stated that the continuous m-reps are Blum axis methods at heart and because of this they do not avoid the intolerances and overprecision of Blum medial methods.

27

2 OBJECTIVES 2.1 Segmentation Segmentation is the first and important step for image modeling and generating a common shape model for a population. Therefore, one of the objectives of this work is to segment 10 CT images of the middle ear. The software tool for segmentation is chosen to be Amira 3.0 which is a 3D visualization and modeling software system. Moreover, the method of segmentation was chosen to be manual interaction to help in overcoming the problem of missing definitive anatomical borders in some parts of middle ear (refer to section 1.1).

2.2 Modeling After the process of segmentation, the results of segmentation should be used for image modeling by using the m-reps method which provides inter-individual correspondence. Pablo software has to be used for this modeling.

2.3 Mean Model A mean model has to be generated from all individuals of the tympanic cavity. This mean model must be used to study and analyze the shape and shape variations of the populations of the middle ears

2.4 Evaluation To investigate how far the resulting models are depending on user interaction, evaluation of modeling process is achieved by comparing two mean models, one is the generated mean model of this work and the other one is the mean model generated by an expert user. Therefore, identical segmented images shall be used. Two different users shall initialize and optimize the mean models of these identical segmented images.

28

3 METHODS 3.1 CT Images For this work, 10 CT scans of anatomical specimens of tympanic bones have been used. They are considered being in a physiological and not in pathological state. The tympanic cavity is a hollow space within the tympanic bone. Therefore, it is not a grown organ in a common sense, but its form is constrained by the shape and growth process of the surrounding tympanic bone (see Figure 3.1).

Figure 3.1: CT Image of Tympanic Membrane

3.2 Amira 3.0 3.2.1 Overview Amira 3.0 [28] is a 3D visualization and modeling software system. Its basic system components are modules and data objects. Modules, which are represented by little icons in the object pool, are used to visualize data objects or to perform some computational operations on them. Data objects of specific types are created automatically from file input data or as output of module computations. Parameters of data objects and modules can be modified in Amira’s interaction area. Amira provides a large number of modules types allowing the visualization of various kinds of scientific data and the creation of polygonal models from 3D images. The functions of Amira are the following: •

direct volume rendering

•

isosurfaces

•

segmentation

•

surface reconstruction

29

•

surface simplification

•

generation of tetrahedral grids

Amira is successfully used in a number of different application areas, among which are medicine, biology, physics, geophysics, astrophysics, and computational fluid dynamics. In this work two functions of Amira have been used. They are the segmentation using Segmentation Editor and the surface reconstruction using the SurfaceGen module.

3.2.2 Segmentation Editor The segmentation editor is a tool for interactively segmenting 3D image data. For the review of segmentation methods and techniques, refer to section 1.2 in this work. The segmentation editor window has the following major parts: •

Material List: for adding, removing and renaming materials.

•

Tool Box: several tools for interactive manipulation of segmentation.

•

Info Area: for displaying some basic information like the grey value.

•

Menu Bar: from the menu bar additional tools and filters can be accessed.

•

Image Viewer(s): the biggest part and is covered by one or four image viewers

3.2.3 How to segment in Amira •

Load the image in Amira viewer.

•

Select the image segmentation editor (see Figure 3.2).

Figure 3.2: How to Select Segmentation Editor •

Manually segment all the slices of middle ear (see Figure 3.3).

30

Figure 3.3: Slice 161 without and with Segmentation

•

Generate a surface with SurfaceGen and SurfaceView (see Figure 3.4 [a], [b]).

Figure 3.4 [a]: SurfaceGen and SurfaceView

Figure 3.4 [b] Surface Reconstruction of a Tympanic Membrane

31

3.3 Pablo 3.3.1 Definition Pablo [27] is a model-building and image-analysis research tool for applying m-reps to medical image segmentation, particularly directed at tasks in 3D CT and MRI data analysis. It was developed by a team of researchers at MIDAG. A reference problem rose while writing this section about Pablo software because the recent version of Pablo has no user guide and the user guide [27] of the previous version of Pablo software has been used. Pablo allows the hand-building of (multifigure) m-rep models, letting the user create and manipulate m-reps and position them relative to an orthogonal-slice-based view of a 3D dataset. Besides performing operations on arbitrary sets of atoms, Pablo can also interpolate between the atoms in a figure to compute both smooth medial sheets and the middle of the implied boundary. Moreover, it can fit a model to the underlying image in many ways: manually under user control, whole-model using a similarity transform over all figures, per objects using geometric constraints, and per figure using similarity transform.

3.3.2 Object Hierarchy A model is contained entirely in a single text file called an m3d file. An md3 file contains the following information: Model o

Figures (set) •

Figure ID: [0..N]

•

Figure’s Name (user assigned)

•

Display color: Red, Green, Blue

•

positivePolarity: 1 for positive, 0 for negative

•

positiveSpace: 1 for protrusion figure, 0 for indentation figure

•

type: QuadFigure

•

numRows

•

numColumns

•

atoms (Mesh) r: length of Y vectors elongation: [1..infinity] orientation: qx, qy, qz, qx selection flag mark flag theta angle: degrees type: EndAtom/StandardAtom position: x, y, z

32

o

Figure Trees (set) •

Parent figure ID

•

childLinks (set) blendAmount: [0..1] blendExtent: [0..1] child figure ID childLinks (set) childAtom ID (u, v, t) coordinate of parents

3.3.3 The fitting of the model to the image This can be done by performing the following steps:•

The image (e.g., P15_CT_Pablo_blur_raw3) has to be loaded. raw3 is the file type for images.

•

The display control has to be adjusted (see Figure 3.5):

(1) Image: in the image of display control there are possibilities of adjusting boundary sagittally, coronally, or axially. (2) Surface: includes Surface Type and Subdivision. Surface Type displays a surface interpolated from the atoms as dots, wire frame, or a tiled opaque surface. None hides the surface. Subdivision is the density of the dots or tile vertices and higher levels yield a smoother surface. Subdivision has been adjusted to level 3 where the geometric entity is the main figure and the transformation is the similarity plus elongation.

Figure 3.5: The Adjustment of Display Control •

A mean model has been generated in a previous study [18] and this mean model (e.g., mean_rigid.m3d) has to be loaded from the file. m3d is the file type for models.

33

•

The mean model defined above has to be fitted to the image with the maneuvers of manual initializations (see Figure 3.6 [a] and Figure 3.7 [a]) like rotation, shifting, and scaling (maximizing and minimizing). They are performed according to the similarity transform and shape variations.

•

With the object ensemble stage next to optimizer control from windows, the manual process of fitting the model to the image can be seen if it is optimal or not. If the current match in object ensemble stage is over -60, then the process should be optimal (see Figure 3.6 [b]). In this process the whole model is adjusted.

Figure 3.6[a]: The Model before and during the Manual Initialization Process

Figure 3.6[b]: Object Ensemble Stage

34

•

After the manual initializations, the automatic fitting process can be started (see Figure 3.7 [b]). The automatic fitting process, however, operates an objective function over a set of geometric transformations available at one of these scale levels. The objective function is the sum of two terms, namely, a term that measures the geometric typicality of the deformed mrep and the geometry to image match term. The Geometric typicality is the comparison of corresponding points in an m-rep before and after deformation while the geometry to image match is the comparison of intensities at corresponding positions [11].

Figure 3.7: The Model [a] after the manual initialization and [b] after the Automatic Fitting Process

•

Then the image match should be computed with the Atom Stage from Optimizer Control (see Figure 3.8). During this process each medial atom in the model will be deformed, in turn and recursively until the convergence of the objective function is achieved. This involves translation, rotation, and scaling of each individual atom of the medial mesh.

35

Figure 3.8: Atom Stage Process Boundary Stage has not been used in this work.

3.4 Mathematica Mathematica is an integrated environment for technical computing. It is used throughout the sciences – physical, biological, social and other. The development of Mathematica has been carried out at Wolfram Research by a world-class team. For more information about Mathematica, refer to [29]. In this work Mathematica has been used to compute the Principal Components Analysis (refer to section 4.3).

3.5 PCA - PGA Statistical shape analysis is emerging as an important area of image processing and computer vision with the primary goal of describing the variability of a population of geometric objects like anatomical structures in medical images. PCA is a standard technique for statistical shape analysis to describe model variability. However, PCA is only applicable when model parameters are elements of a Euclidean vector space. Therefore, it does not handle more complex representations of shape.

3.5.1 PCA Definition PCA [21] is a way of identifying patterns in data, and expressing the data in such a way as to highlight their similarities and differences. PCA is a powerful tool for analyzing data because of the difficulty of finding patterns in data of high dimension and the unavailability of representing these data graphically.

36

3.5.2 PCA Applications PCA has proven to be useful for understanding geometric variability in populations of parameterized objects. However, other applications of PCA include, besides the finding of patterns, representation and image compression.

3.5.3 The Steps of Performing PCA •

Get the data: The data are the 3D dataset (dimensions x, y, z) of the axes of the models.

•

Subtract the mean: For PCA to work properly, the mean has to be subtracted from each of the data dimensions. The mean subtracted is the average across each dimension.

Equation 1 •

Calculate the covariance matrix:

Equation 2

•

Calculate the eigenvectors and eigenvalues of the covariance matrix:

Definition: If A is n x n matrix, then a nonzero vector x in Rn is called an eigenvector of A if Ax is a scalar multiple of x; that is Ax = λx for some scalar λ. The scalar λ is called an eigenvalue of A, and x is said to be an eigenvector of A corresponding to λ.

37

•

Choosing components and forming a feature vector: The eigenvector with the highest eigenvalue is the principal component of the data set. In general, once eigenvectors are found from the covariance matrix, the next step is to order them by eigenvalue, highest to lowest. Feature vector is constructed by taking the eigenvectors which are kept from the list of eigenvectors, and a matrix is formed with these eigenvectors in columns.

FeatureVector = (eig1 eig2 …eign)

•

Deriving the new data set: This is the final step in PCA. When the feature vector is chosen, we simply take the transpose of the vector and multiply it on the left of the original data set, transposed.

FinalData = RowFeatureVector x RowDataAdjust,

RowFeatureVector is the matrix with the eigenvectors in the columns transposed, and RowDataAdjust is the mean-adjusted data transposed.

3.5.4 Generalization of PCA As the m-reps are representations of geometry based on the medial axis description providing complex variability in terms of bending, twisting, and widening, the medial parameters are not naturally elements of a Euclidean space, but they are in fact elements of nonlinear Riemannian symmetric space [13]. The generalization of PCA to the manifold setting is called Principal Geodesic Analysis (PGA).

PCA (Principal Component Analysis)

Euclidean Vector Space

PGA (Principal Geodesic Analysis)

Riemannian Symmetric Space

For more information about the PGA, refer to [13].

38

3.6 MATLAB MATLAB is a high-performance language for technical computing. It integrates computation, visualization, and programming in an easy-to-use environment where problems and solutions are expressed in familiar mathematical notion. Typical uses include:•

Math and computation

•

Algorithm development

•

Data acquisition

•

Modeling, simulation, and prototyping

•

Data analysis, exploration, and visualization

•

Scientific and engineering graphics

•

Application development, including GUI building

MATLAB is an interactive system whose basic data element is an array that does not require dimensioning. The MATLAB system consists of five main parts: development environment, the MATLAB mathematical function library, the MATLAB language, graphics, and the MATLAB application program interface (API). For more information about MATLAB, refer to [30]. In this work MATLAB has been used to compute the distances between the atoms of the two models (refer to section 4.3).

39

4 RESULTS 4.1 The Mean Model Using the m-reps method with an atom grid containing 15(5x3) atoms (see Figure 4.1), a common shape model of the population has been generated (see Figure 4.2) with the use of the Pablo settings: neighbor penalty = 0.9 and geometric weight = 0.35,

Figure 4.1: Atom Grids (5x3) with Hidden and Wire Frame Surface

Figure 4.2: Common Shape Model of the Population with Solid and Wire Frame Surface

40

4.2 Distribution of the Mean Model Application of principal geodesic analysis has shown that all the individuals are lying within a Gaussian distribution within a range of ± 2.5 standard deviation (see Figure 4.3).

Figure 4.3: Distribution of the I and II Principal Components Principal Component (PC)

Covering of Shape Space %

Accumulated %

First PC

30.5

30.5

Second PC

19.9

50.4

Third PC

14.1

64.5

Fourth PC

09.9

74.4

Fifth PC

07.4

81.8

Sixth PC

06.1

87.9

Seventh PC

04.8

92.7

Eighth PC

04.3

97.0

Ninth PC

03.0

100.0

Referring to the above table, the first three principal components cover a shape space of 64.5 % in the mean model and the first five components a shape space of 81.8 % while the nine principal components are covering 100 % (see the table plus Figures 4.3 and 4.4).

41

100

80

60

40

20

1

2

3

4

5

6

7

8

9

Figure 4.4: Distribution of Eigenvalues (Principal Components)

42

4.3 Variations 4.3.1 First Principal Component The first principal component which covers a space of 30.5 % reveals that the major variation among the population is a stretching and compression mainly along the axis of the cavity going through epitympanon (–paries tegmentalis-) and hypotympanon (-paries jugularis-). The first principal component shows a stretching along negative standard deviations (see Figure 4.5 left) and a compression along positive standard deviations of the model (see Figure 4.5 right). There is also an enlargement of the epitympanon along negative standard deviations and decreasing of the same part along positive standard deviations. Moreover, the variations of the first principal component are caused by differences mainly in the epitympanon and hypotympanon of the tympanic cavity (see Figure 4.5).

Figure 4.5: Variations from the First Principal Component (left: -2σ, middle: mean, right: +2σ)

4.3.2 Second Principal Component The second principal component which covers a space of 19.9 % reveals that its variations among the population is a stretching along negative standard deviations and compression along positive deviations. These changes are mainly along the axis of the cavity going through the anterior (–paries caroticus-) and posterior (–paries mastoideus-) walls of the middle ear. The major part of this variation is caused by differences in the size of the mesotympanon (see Figure 4.6).

43

Figure 4.6: Variations from the Second Principal Component (left: -2σ, middle: mean, right: +2σ)

4.3.3 Third Principal Component The third principal component covers a space of 14.1 % and reveals a medial and lateral bending (kinking) of the hypotympanon along the axis going through the lateral (-paries membranaceus-) and medial (-paries labyrinthicus-) walls of the tympanic cavity (see Figure 4.7 left and right).

Figure 4.7: Variations from the Third Principal Component (left: -2σ, middle: mean, right: +2σ)

44

4.3.4 Fourth Principal Component This principal component covers a space of 9.9 % and mainly reveals the bulging of the hypotympanon along the axis that going through the lateral and medial walls of the tympanic cavity (see Figure 4.8 right).

Figure 4.8: Variations from the Fourth Principal Component (left: -2σ, middle: mean, right: +2σ)

4.3.5 Fifth Principal Component This principal component covers only a space of 7.4 % and reveals a narrowing of the epitympanon along longitudinal axis that goes along the lateral and medial walls of the middle ear (see Figure 4.9).

Figure 4.9: Variations from the Fifth Principal Component (left: -2σ, middle: mean, right: +2σ)

45

4.4 Evaluation with Comparison 4.4.1 with Mathematica Two mean models have been generated, one is the result of this work and the second one is generated by an expert user. The two mean models are based on the same manual segmentation method and the same manual initializations technique and, therefore, depend on the subjective interaction of the user. They have been investigated and analyzed to see if they are more or less different from each other. They will be referred to as the first and the second mean models (MM). All the individuals of both mean models are lying within a Gaussian distribution in within a range of ± 2.5 standard deviation (see Figures 4.10 and 4.11).

Figure 4.10: Distribution of the I and II Principal Components (First MM)

46

Figure 4.11: Distribution of the I and II Principal Components (Second MM)

In the following table a comparison between the two mean models has been performed. The diagram in Figure 4.11 is reflected and in its right position has been compared to the diagram of the Figure 4.10. The result of this comparison between the horizontal and vertical axes of the two diagrams is that, the first, third and eighth mean models are reflected in the y axis while there is no mean model that is reflected in the x axis.

st

1 M nd

X & Y Axes

X & Y Axes

(0.4,-0.6) [+x, -y]

(0.5,0.2) [+x, +y]

M

(0.1,-1.1) [+x, -y]

(0.1,-0.3) [+x, -y]

rd

3 M

(-0.3,-0.6) [-x, -y]

(-0.5,0.2) [-x, +y]

th

4 M

(-0.4-0.6) [-x, -y]

(-0.3,-1.6) [-x, -y]

5th M

2

(2.0,-0.7) [+x, -y]

(1.8,-0.7) [+x, -y]

th

(0.1,0) [+x, 0]

(0.4,-0.7) [+x, -y]

th

(-1.3,-0.1) [-x, -y]

(-1.3,0) [-x, 0]

th

(-1,0.6) [-x, +y]

(-0.7,-0.2) [-x, -y]

th

9 M

(1.0,2.3) [+x, +y]

(1.2,2.0) [+x, +y]

th

(-0.7,0.7) [-x, +y]

(-1.15,1.2) [-x, +y]

6 M 7 M 8 M 10 M

Application of principal geodesic analysis (PGA) has shown that the first five principal components cover a shape space of 78.1 % in the first mean model and 81.9 % in the second mean model while the nine principal components are covering 95.4 in the first mean model and 100 % in the second one (see the tables plus Figures 4.12 and 4.13).

47

Principal Component (PC)

Covering of Shape Space %

Accumulated %

First PC

30.5

30.5

Second PC

19.9

50.4

Third PC

14.1

64.5

Fourth PC

09.9

74.4

Fifth PC

07.4

81.8

Sixth PC

06.1

87.9

Seventh PC

04.8

92.7

Eighth PC

04.3

97.0

Ninth PC

03.0

100.0

Principal Component (PC)

Covering Shape Space %

Accumulated %

First PC

30.5

30.5

Second PC

20.0

50.5

Third PC

14.0

64.5

Fourth PC

10.0

74.5

Fifth PC

07.4

81.9

Sixth PC

06.0

87.9

Seventh PC

04.8

92.7

Eighth PC

04.3

97.0

Ninth PC

03.0

100

100

80

60

40

20

1

2

3

4

5

6

7

8

9

Figure 4.12: Distribution of Eigenvalues (Principal Components) – First MM

48

Figure 4.13: Distribution of Eigenvalues (Principal Components) – Second MM

49

4.4.2 with MTLAB With MATLAB the distance between each of the same two atoms of the 15 atoms of both mean models has been calculated Generally we can consider the above three atoms representing the epitympanon, the middle six, from the fourth atom until the ninth one, representing the mesotympanon, and the last six representing the hypotympanon (see the table and Figure 5.1). The distance is generally enlarged in upper and lower parts of the middle ear while the middle part has shown small distances between its atoms. Distance between the atoms of the two mean models* Dist. [01] Atom =0.0017 unit cube

= 0.255 mm

Dist. [02] Atom = 0.0012 unit cube

= 0.180 mm


= 0.195 mm


= 0.255 mm


= 0.285 mm


= 0.060 mm


= 0.075 mm


= 0.045 mm


= 0.285 mm


= 0.225 mm


= 0.315 mm


= 0.300 mm


= 0.075 mm


= 0.255 mm


= 0.360 mm

* In this table Dist. [01] means the distance in mm between the corresponding first atoms of the two mean models (To calculate the pixel, the unit cube has been divided by the resolution of the image 1/512 = 0.001953125). Since the length of the mean model is 16.0 mm and its width is 9.0 mm, the result of the above table is that the distances between the corresponding atoms of the models are very small compared to the model’s size. Therefore, the two mean models are always almost identical.

50

[b]

Figure 4.14: The Two Mean Models in Comparison (MM1-blue + MM2-red)

51

5 DISCUSSION 5.1 Problem of Manual Segmentation The segmentation process is highly important for the process of modeling because modelbuilding depends upon the single-figure objects that result from the process of segmentation. The segmentation can be performed automatically, semi-automatically, or manually. The application of fully automated segmentation is extremely challenging and not realistic in the biomedical images because of the large biological variations. Therefore, in this work, the manual slice-by-slice technique of segmentation using Amira software has been chosen, with strict abiding to anatomical definitions, to overcome the problem of the missing of radiological borders of anatomical structures in some parts of the middle ear cavity (refer to section 1.2). To perform the manual segmentation in the epitympanon, the junction between the epitympanic recess and mastoid antrum was chosen as a boundary to separate the tympanic cavity from the mastoid cells [10]. As a borderline between the auditory tube and hypotympanon, a line has been chosen. This line connects between processus cochleariformis (see Figure 1.2) and the edge at the crossover between tympanic cavity and auditory tube. One possibility to improve the accuracy of the model near the border between the tympanic cavity and the auditory tube would be the generation of an additional model of the auditory tube. Then the correlations between the shape of the two models can be compared and analyzed, and the anatomical borders between both models can be also well defined. A model also for the middle and inner ear can be generated because the two parts of the ear are anatomically and functionally connected.

5.2 Model Variation: Validity and Quality Shape representation and analysis has been considered a difficult and challenging problem in computer vision and image analysis [20] and it is more difficult in biomedical image analysis because of the biological shape variability. Pizer [17] has considered image analysis to be a misnomer and he states that: “we should see ourselves as being in real world analysis with images as evidence”. By extending Pizer’s statement, this helps in understanding the biological real world of the patient’s anatomy or physiology. The main idea of this study was to generate a common shape model of the tympanic cavity with its variations using the m-reps technique and Pablo software. This objective is successfully achieved with the results demonstrating the suitability of m-reps approach. Medial models are so useful in achievement of real world analysis [17] and the mean model generated by this study is in this direction of doing real world analysis which needs models of the real world.

52

This work describes all the important steps of generating that mean model from the manual segmentation of CT images to the generation of the mean model and comparing it with another mean model generated by an expert user (refer to 4.4).

The medial sheet for modeling has been intuitively chosen to be an atom grid containing 15 (5x3) atoms to be used for each individual object. Each of these atoms holds a tupel of parameters describing the characteristics of the local shape of the tympanic cavity model. Therefore, in the above figure the upper and middle tupels describe the epitympanon and mesotympanon respectively while the lower tupels are describing the hypotympanon. The results of refining this model in an interactive process for each object produce atom grids with different individual shapes representing interindividual variations. The inter-individual shape variations have been analyzed and quantified using principal geodesic analysis (PGA) derived by Fletcher et al. [14], which is an extension of principal components analysis (PCA) to be used in a Riemannian symmetric space.

Validity The common shape models depend upon the manual segmentation method and the manual initializations technique. The segmentations and the fitting of the model to the images have been performed totally manual from different two users. The resulting two models are almost identical – therefore the experiment shows that the optimization process is very robust and it’s dependency on the manual interactions is small. The results of this study show that a mean model of tympanic cavity, despite the missing of clear anatomical borders of some parts of middle ear, can be generated and it can provide a facility to analyze the complex shape of the anatomical structure of the middle ear and its variations. The mean model could also be used to help in differentiation of physiological variations of the middle ear from pathological ones as the mean model is an important tool for understanding anatomical structures

53

from medical images. The validation of the mean model and its statistical interpretations, therefore, are encouraging; providing robust and accurate results for clinical use and acts as a basis for further research.

Quality The analysis of the common shape model reveals characteristic directions of variations within the populations and has the ability to identify areas in the tympanic cavity where variations in shape are quite distinct in contrast to other parts which are more constant throughout the population. These directions of variations are represented by the first three principal components.

5.3 Problems The number of the CT images used for this study was not large enough and a large number of CT images (like 60 CT images) for a study like this would be optimal recommendable to refine the results. The comparison has been done between two mean models and it would be very recommendable if the number of mean models to be compared is 10, for example. A third problem emerges from the fact that the m-reps are not good in modeling bumps in the figures. A boundary displacement exceeding the allowable tolerance is represented as an attached subfigure because m-reps do not use a branching skeletal structure. This allows the object’s shape to be divided into several possible types of figural relationships. Much time has been spent in the manual segmentation of the CT individuals and in the manual fitting process of the model to the images. These manual processes have been repeated many times for the sake of accuracy or after some mistakes.

5.4 Outlook An extension of this work would be a project of generating more mean models of tympanic cavities from more CT individuals. The modeling process can be performed by m-reps and other modeling method (e.g. point based) and then the mean models resulting from the different modeling methods would be compared. Because the segmentation and modeling manual processes need much time, automatic or semiautomatic techniques would be wishful for these processes.

54

6 SUMMARIES 6.1 Abstract Statistical shape analysis is an important tool for understanding complex anatomical structures and their variations in biomedical images. One of the important shape analysis technique is the medial representation (m-reps) which is developed by Pizer and his colleagues in image analysis at MIDAG. M-reps are a multiscale medial means for modeling and rendering 3D solid geometry and the main idea behind them was to reverse the medial transform from a boundary implying a medial description to a mesh of medial atoms implying boundaries and to define objects medially by allowing the medial structure to imply the boundaries. The purpose of this study was to generate a mean model from a population of tympanic cavities using m-reps technique and software called Pablo. This purpose has been successfully achieved with the results demonstrating the suitability of m-reps approach. The middle ear, or tympanic cavity, is the intermediate portion of the ear of higher vertebrates consisting typically of a small air-filled membranelined chamber in the temporal bone continuous with the nasopharynx through the Eustachian tube, separated from the external ear by the tympanic membrane and from the inner ear by fenestrae, and containing a chain of three ossicles that extends from the tympanic membrane to the oval window and transmits vibrations to the inner ear. As the segmentation is a highly important step for modeling, 10 CT images of middle ear were manually segmented using Amira software. Then a mean model generated by a previous study was used to generate the mean model of the population in this work using the m-reps method with an atom grid containing 15 (5x3) atoms. The variations of this mean model have been analyzed and quantified by using the principal components analysis to show the directions of variations in some parts of tympanic cavity in the population. A comparison has been done between the mean model of this work and another mean model generated by an expert user. The mean models depend upon the manual interaction of the users in the stages of segmenting the images and fitting the model to the images. All the individuals of both mean models have been found to be lying within the Gaussian distribution and the distance between the corresponding atoms of both mean models is calculated. This experimental comparison showed only small dependencies of the resulting models from the user interactions. The major variations of the first principal components analysis are stretching and compression along the axis from superior to inferior parts of the tympanic cavity while the main changes of the second principal components analysis are also stretching and compression but along the axis from anterior to posterior parts of the tympanic cavity. Bending of the hypotympanon occurs in the third principal components analysis along the axis from lateral to medial walls and bulging of the hypotympanon occurs along the same axis in the fourth principal components analysis. Narrowing of the epitympanon is the variation of the fifth components analysis along the longitudinal axis.

55

6.2 Zusammenfassung Die statistische Formanalyse ist eine wichtige Methode um komplexe anatomische Strukturen und ihre biologischen Variationen zu verstehen. Eine Methode zur statistischen Formanalyse ist die mediale Representation (m-reps), die von Pizer et al. entwickelt wurde. M-reps ist eine Methode zur hierarchischen Modellierung von 3-dimensionalen Objekten. Die Hauptidee hinter m-reps ist es, die Form eines Objekts anhand von medialen Ebenen, bestehend aus Atomen, welche den Rand eines Objektes implizieren, zu beschreiben. Das Ziel dieser Studie ist es, ein gemeinsames „Common Shape“ – Modell (1.MM) von 10 CT Bilder des Mittelohrs mit der m-rep Methode und einer Software namens Pablo zu erstellen. Dieses Ziel konnte erreicht werden und die Ergebnisse bestätigen, dass m-reps eine adäquate Methode für die Erstellung eines „Common Shape“ – Modells des Mittelohrs ist. Das Mittelohr setzt sich zusammen aus dem Trommelfell, der Ohrtrompete und der Paukenhöhle. Sie enthält die drei gelenkig verbundenen Gehörknöchelchen: Hammer, Amboss und Steigbügel. Diese empfangen Schallwellen, verstärken sie und senden sie weiter zu der Schnecke. Das Trommelfell ist eine dünne, ovalförmige Membran, die das Außenohr vom Mittelohr luftdicht abtrennt. Es hat eine effektive Schwingungsfläche von etwa 0,65 Quadratzentimetern, die durch auftreffende Schallwellen in Bewegung versetzt wird. Zur Erstellung der Modelle wurden 10 CT Bilder vom Mittelohr mit Amira 3.0 segmentiert. Als Grundlage für Erstellung des Modells wurde ein schon vorhandenes „Common Shape“ – Modell, das aus 15 Atomen (5x3) besteht, benutzt. Hauptkomponentenanalyse wurde eingesetzt um die Variationen des neuen „Common Shape“ – Modells

zu analysieren und quantifizieren. Die

Variationen der ersten fünf Hauptkomponenten sind in dieser Studie beschrieben. Es wurde unter Verwendung von Mathematica und MATLAB ein Vergleich zwischen dem ersten „Common Shape“ – Modell und einem zweitem (2.MM), welches unter gleiche Voraussetzungen vom einen Experten Benutzer erstellt wurde, durchgeführt. Obwohl die Erstellung der Modelle abhängig von manueller Interaktion der Benutzer ist, zeigen die Ergebnisse dieses Vergleichs, dass die zwei „Common Shape“ – Modelle fast identisch sind.

56

REFERENCES In the following list of references there are general references which are [1], [2], [3], [20], and [21], and the other ones are the specific references of this work. Books [1]

Joseph Gibaldi. MLA Handbook for Writers of Research Papers. 6th edition. The Modern Language Association of America. 2003

[2]

Merriam-Webster’s Medical Desk Dictionary. Merriam-Webster, Incorporated, Publishers. 1996

[3]

Thomas M. Lehmann, Meyer zu Bexten, editors. Handbuch der Medizinischen Informatik. Hanser. 2002

[4]

Gerhald J. Tortora, Sandra Reynolds Grabowski. Principles of Anatomy and Physiology. 10th ed. John Wiley & Sons, Inc. 2003

[5]

Herbert Lippert. Lehrbuch Anatomie. 6. Auflage. Urban & Fischer. 2003

[6]

Rafael C. Gonzalez, Richard E. Woods. Digital Image Processing. 2nd edition. Prentice Hall. 2002

[7]

Isaac N. Bankman ,editor. Handbook of Medical Imaging: Processing and Analysis. Academic Press. 2000

[8]

Milan Sonka, J.Michael Fitzpatrick, editors.Handbook of Medical Imaging: Volume 2 Medical Image Processing and Analysis. SPIE. 2000 Theses

[9]

Andrew Lewis Thal. “Deformable Solid Modeling via Medial Sampling and Displacement Subdivision”. Dissertation. University of North Carolina at Chapel Hill. Chapel Hill. 2004. < midag.cs.unc.edu >

[10] Karl D. Fritscher. “Development of a Software Framework for Preprocessing and Level-Set Segmentation of Medical Image Data”. M.Sc Thesis. IBIA at UMIT. Innsbruck. 2004. [11] Manjari I. Rao. “Analysis of a Locally Varying Intensity Template for Segmentation of Kidneys in CT Images”. M.Sc Thesis. University of North Carolina at Chapel Hill. Chapel Hill. 2003. < midag.cs.unc.edu > Papers [12] Klaus Engelke et al. Interactive 3D Editing Tools for Image Segmentation. Medical Image Analysis 8 (2004) 35-46 [13] P. Thomas Fletcher et al. Principal Geodesic Analysis for the Study of Nonlinear Statistics of Shape. IEEE Transactions on Medical Imaging (2004). < http://midag.cs.unc.edu > [14] P. Thomas Fletcher et al. Statistics of Shape via Principal Component Analysis on Lie Groups. CVPR Proceedings 1 (2003) 95-101 < midag.cs.unc.edu >

57

[15] Stephen M. Pizer et al. Deformable M-Reps for 3D Medical Image Segmentation. International Journal of Computer Vision 55 (2/3), 85-106, 2003. < midag.cs.unc.edu > [16] Stephan M. Pizer et al. Multiscale Medial Loci and Their Properties. International Journal of Computer Vision 55 (2/3), 155-179, 2003. [17] Stephan M. Pizer. Guest Editorial – Medial & Medical: A Good Match for Image Analysis. International Journal of Computer Vision 55 (2/3), 79-84,2003 [18] Karl D. Fritscher et al. Analyzing Inter-Individual Shape Variations of the Middle Ear Cavity by Developing a Common Shape Model Based on Medial Representation. International Congress Series 1268 (2004) 243-248. CARS 2004 [19] Tim McInerney et al. Deformable Organisms for Automatic Medical Image Analysis. IMIA Yearbook of Medical Informatics 04: Towards Clinical Bioinformatics.466-481. Schattauer.2004. [20] Martn Styner et al. Automatic and Robust Computation of 3D Medial Models Incorporating Object Variability. International Journal of Computer Vision 55 (2/3), 107-122,2003 [21] Lindsay I Smith. “A Tutorial on Principal Components Analysis”. February 26, 2002. < www.cs.otago.ac.nz/cosc453/student_tutorials/principal_components.pdf > Websites [22] LEO – Link Everything Online. < http://dict.leo.org > [23] Merriam – Webster Online < www.m-w.com > [24] Center for Imaging Science. Johns Hopkins University. Medical Imaging and Computational Anatomy < http://cis.jhu.edu > [25] Bartleby.com Great Books Online. < www.bartleby.com >. This website includes Gray’s Anatomy. [26] Instant Anatomy. Robert Whitaker. < www.instantanatomy.net > [27] MIDAG Medical Image Display and Analysis Group. < http://midag.cs.unc.edu >: Most of the papers and also Pablo User Guide by Gregg Tracton are on this website. [28] Amira. User’s Guide. < www.amiravis.com > [29] WolframResearch. < www.wolfram.com > [30] The MathWorks. < www.mathworks.com >

58

Short C.V.

Personal Data •

Name: Omar Alshafi

•

Email: [email protected]

•

Date of Birth: 24.10.1967 (officially 01.01.1966)

•

Nationality: Sudanese

•

Marital Status: Divorced

Education •

M.Sc. in Medical Informatics, University for Health Sciences, Medical Informatics and Technology (UMIT), Austria.

•

Diploma of Community Health and Tropical Medicine -"HealthCare in Developing Countries", University of Heidelberg, Germany, 2001.

•

Third State Examination of Medicine (= M.Sc equivalent), University of Freiburg, Germany, May 2000.

•

Bachelor of Medicine and Surgery, University of Cairo, December 1994.

Work Experience •

Practical Medical Year in Germany (19.10.98 26.09.99).

•

Medical Advisor at the Embassy of the Sudan in Cairo (01.04.96 31.08.98).

•

Visiting Pediatric Resident at New Children Hospital, Cairo University Hospitals (01.10.96 08.02.98).

•

Internship (Housemanship) at Cairo University Hospitals (01.03.95 29.02.96).

•

Teacher of English and Mathematics in Sudanese Intermediate Schools (two semesters).

59

Hiermit erkläre ich an Eides statt, die Arbeit selbstständig verfasst und keine anderen als die angegebenen Hilfsmittel verwendet zu haben.

I hereby declare to have completed this work independently and to have used no aids other than those mentioned. Hall in Tirol, --------------------------------------------------------------------------------------Omar Alshafi

60