Segmentation of Vessels in Peripheral CTA Datasets - Semantic Scholar

Segmentation of Vessels in Peripheral CTA Datasets Literature review and first tests of some approaches

Petr Felkel [email protected]

Segmentation of Vessels in Peripheral CTA Datasets 1 The aim of our work and of this report 2 Data description 2.1 Why CT when other techniques exist? 2.2 Problems by the CTA acquisition 2.3 BBA Datasets available 2.4 Vessel description 2.4.1 Anatomy of a stenotic vessel 2.4.2 Typical voxel densities around 2.4.3 Densities along a segmented vessel 2.5 Data in the dataset 008 3 Vessel Segmentation Methods Used by Others 3.1 Miloš Šrámek’s threshold-morphological method 3.2 Wave Vessel tracking Algorithm by Cornelia Zahlten 3.3 Data enhancement and vessel tracking by Frangi 3.3.1 Preprocessing by 3D filtering 3.3.2 Vessel tracking & border detection 3.4 Direct vessel tracking in 3D with vessel centerpoint localization 3.5 Real time vessel enhancement and detection 3.6 Live Wire, Live Wire on the Fly and Intelligent Scissors 3.7 Knowledge-based 2D approach from ESAT KU Leuven 4 Applicability of methods and tests 4.1 Notes about Miloš Šrámek’s threshold-morphological method 4.2 Tests of Vessel tracking by Wave Algorithm 4.3 Test of Frangi’s vessel enhancement algorithm 4.4 Notes about direct vessel tracking by Wink 4.5 Notes about real-time vessel enhancement 4.6 Tests of the modified Live wire 4.7 Notes about a knowledge base approach from KU Leuven 4.8 Ideas and interesting methods waiting for a study 4.8.1 References to other interesting approaches 4.8.2 Other ideas to think about 5 Visualization 6 Notices and Remaining Questions 7 Conclusions Acknowledgments References 1

1 2 3 3 3 5 6 6 6 8 10 15 16 17 18 18 19 20 21 23 25 26 26 26 28 29 29 30 31 31 31 32 33 33 34 34 35

1 The aim of our work and of this report The aim of our work is to develop a fast and robust method for CTA vessel and calcification segmentation, which requires minimum of user interaction or concentrates the interaction into minimal number of interventions. Selected vessel centerlines will be used for creation of sets of Curved Multi-Planar Reconstructions (CMPR) in 180° (rotated viewpoint), segmented vessels will be used for measurements of their diameter and other parameters. The aim of this report is a literature research on the topic of vessel segmentation and information about the preliminary tests of interesting methods. It is good to know at the beginning how far the others are, to take advantage of their ideas, to depict problematic steps of each method, and moreover, to avoid such approaches that do not yield robust vessel segmentation. And it is also good, to share the information with others. There are two major types of articles available on this topic – a medical one, which typically describes a set of real medical cases (real findings on a group of patients) without any detailed information about algorithms applied during the whole procedure of data processing from a raw form into presented images; and a technical one, which mainly deals with the detail description of data processing steps, often avoiding any information about what the physician thinks about the results and how useful they really are for a clinical praxis. I have found a lot of medical articles, but only [Ruehm00] has valuable information for me and is mentioned in Chapter References. This report gives an overview of methods dealing with different kinds of vessel segmentation, i.e., not only with peripheral vessels and not only for our modality (Computer-Assisted Tomography Angiography – CTA, as described later). The reasons are two, firstly, only very few articles about CTA exist (or have been found by me J), and secondly, the principles of 3D segmentation methods from different modalities can be adapted for our goal of CTA segmentation. I have already studied the following methods: • Thresholding-morphological method by Miloš Šrámek • Wave vessel-tracking by Cornelia Zahlten et al. • Vessel enhancement method by Frangi et al. • Direct vessel tracking by Wink et al. • Real time vessel enhancement by Poli et al. • Live Wire of Falcao, Barret & Mortensen • 2D knowledge based method by Cleynenbreugel and Smets et al. The methods waiting for studying include: • 3D knowledge based method by Vandermeulen • Hyperstack method by Vincken, Utrecht • Fuzzy segmentation of eye vessels by Tolias • Vessel segmentation from ultrasound by Šonka et al. The presented text has the following structure. In Chapter 2 I discuss available acquisition techniques and describe eight datasets available for testing with a detailed view on one of them. In Chapter 3 is the description of methods for vessel segmentation I have found up to now. Chapter 4 briefly discusses experiences and potentials of each method and digests my opinions about the applicability of the described methods for the task of peripheral vessels 2

CTA. Chapter 5 informs briefly about the visualization step, Chapter 6 collects the remaining notices and questions about some steps that were or still are not absolutely clear. Conclusions follow in Chapters 7.

2 Data description This chapter informs about the data acquisition method applied and about the datasets available now for testing purposes. One dataset is then described in a greater detail to give an imagination of how the vessels are represented in the dataset.

2.1 Why CT when other techniques exist? We use a Computer Tomography (CT) data since our goal is: - Noninvasive diagnostic method (x DSA) - Detection and visualization of calcification with better spatial resolution (x MRA) - Shortening of current 4 hours manual vessel delineation – at first by an interactive method, later ideally by a full automatic one - Measurements of vessel parameters Pros & Cons in more detail: •

•

•

Digital Subtraction Angiography (DSA) - invasive, expensive method with minor and major complication rates ranging from 0.17% to 7% [Ruehm00], of 2D nature, which implies also limitations for vessels parallel to the viewing direction [P] and limitations for detection and characterization of non-concentric stenoses [Ruehm00]. It also does not show calcification. Magnetic Resonance Angiography (MRA) – resolution 1.8x1.8x2.4mm or 0.9x0.9x1.2mm with zero interpolation in K-space (the reconstruction method done by the scanner itself), limited acquisition coverage 40–48 cm, 90 cm with a peripheral vascular coil, contraindications: pacemaker, claustrophobia, metallic foreign bodies, and pregnancy (it seems that this is a contraindication to all these three methods). In 98% MRA gives the same results as the DSA [Ruehm00]. It does not show calcification (except of the newest research in Mt. Sinai School of Medicine, NY – notice in a popular magazine). The main arguments for Computer Tomography Angiography (CTA) are its (currently) better spatial resolution, better visualization of calcifications (which is our main goal), and the availability of CTA compared to MRA (and some contraindications of MR in general – see the previous item).

2.2 Problems by the CTA acquisition In this section I tried to collect all information I knew about the details of the data acquisition. In fact, the data acquisition setting is not the task for me but rather for the radiologists. The major problems of the data acquisition can be classified into three classes; homogeneity of the contrast agent distribution, settings of the CT scanner and partial volume artifact. So called plateau-like concentration of a contrast agent is necessary for getting the constant value response inside the studied vessels during the scanning. It means the agent concentration stays as constant as possible in the whole vessel of interest, which can be achieved by an appropriate contrast agent injection protocol and by a proper scan delay setting. 3

In greater detail, the plateau-like concentration of a contrast agent is typically studied and achieved in abdominal vessels. It is much more complicated to achieve the homogeneous concentration distribution in peripheral vessels, since the physical characteristics of the small vessels is not known, and it becomes even more complicated if different amount of calcification, other types of stenoses and collateral flows exist. As an example of such a complicated concentration profile see Figure 5, where the profile in abdominal part is relatively constant (right), but it changes rapidly in the lower parts of the body (left). To important acquisition parameters which can influence the quality of datasets and in this way also the resulting segmentation belong the settings of a CT scanner (collimation, gantry tilt, maximal dose, etc...). Modification of these parameters can be tested after the properlyworking segmentation method exists to improve its reliability and accuracy. Limited resolution of the scanner and mainly the scanning of slices thicker than the size of scanned tissue-structures result in Partial Volume Artifact (PVA) on the tissue borders. If different tissues share one scanned voxel, the resulting scanned value is a weighted average of their values, which may be then misclassified as a different tissue.

What to test in the future to enhance the quality of the segmentation process and to achieve the best results? •

•

•

•

contrast media - injection protocol (timing, amount of the contrast agent) - different contrast agents (200 HU used at present or 400 HU). Would not be the vessel appearance more similar to a compact bone? different acquisition parameters settings - acquisition region size (inner-slice resolution) - gantry speed and collimation (projected slice spacing and thickness) - number of reconstructed slices (inter-slice resolution) - reconstruction filters and interpolation method phantom tests - calibrated length phantom (a twisted tube of known length with the distance markers visible in CT) for a precise length calibration along the vessels - branching phantom for a correct handling of branches - diameter phantom (a set of cylindrical objects of different diameters filled up by a contrast agent of the same concentration) to test, how linear the response of the scanner in relation to the changes of the vessel diameter is influence of artifacts (e.g., metal bodies create star like artifacts)

4

2.3 BBA Datasets available Table 1 contains the overview of eight BBA datasets available for testing purposes. The dataset of the highest resolution with voxel sizes 0.25x0.25x2 mm has the number 003b. First four datasets are shown in a 3D view in Figure 1. Dataset No.

No. of slices in the dataset

Slice Thickness/ Spacing* [mm]

Pixel size X=Y

001 002 003a 003b 004 005 006 007 008

444 783 553 169 551 1134 1070 988 1167

3/2 1.25 / 0.7 3/2 3/2 3/2 3/1 3/1 2/1 3/1

0.6484375 0.46289063 0.65429688 0.25 0.46875 0.7421875 0.7265625 0.50195313 0.7421875

Volume size (computed from No. of slices and thickness, and pixel sizes [mm x mm x mm] 332 x 332 x 888 237 x 237 x 548 335 x 335 x 1106 128 x 128 x 338 240 x 240 x 1102 380 x 380 x 1134 372 x 372 x 1070 257 x 257 x 1070 380 x 380 x 1166

Table 1 Available datasets and their parameters

001

002

003a

003b

004

Figure 1 First five datasets in a 3D view

*

Note about the acquisition/reconstruction parameters: This column represents the parameters of the slices in each dataset (e.g., overlapping slices of 3 mm thickness with mutual spacing of 1mm). These parameters were given as input parameters to the reconstruction process built-in in the CT scanner. The acquisition parameters (4x2.5 or 4x1) describe the configuration of the scanner. This spiral-CT scanner acquires simultaneously four projection datasets and allows to configure the detectors to scan 2.5 or 1 mm thick projection datasets. From these raw projection-data the transversal slices are reconstructed, with desired thickness and spacing. All the datasets in Table 1 have 4x2.5 mm, except from 007 with 4x1 mm.

5

2.4 Vessel description In this section I describe the anatomical structure of a stenotic vessel, the appearance of the vessel and surroundings tissue in ideal cross-section above the knee and discuss the density distribution along the segmented vessel from one of our CT datasets.

2.4.1 Anatomy of a stenotic vessel A typical outer vessel shape is circular or slightly elliptical. If the calcification is present, it is situated in the vessel, the inside diameter is irregular, very often far to be circular (see Figure 2 and the transversal profiles in Section 2.5). Also non-calcified stenoses exist, formed by soft tissue deposits, e.g., dead blood corpuscles. Calcification is a second evolutional step of such a stenosis. The vessel dimension can be significantly enlarged if aneurysm (a ball shaped object) is present.

Adventitia = fibrous and fatty surroundings External elastic lamina Media = smooth muscle cells and reticular collagen network (0.1-0.3 mm) Internal elastic lamina Intima = normal thin layer of plaque Plaque =calcification

Figure 2 Schematic cross-sectional anatomy of a diseased coronary vessel [Sonka et. al 98, p. 728]

2.4.2 Typical voxel densities around The stenotic vessel densities, measured by CT in Hounsfield Units (HU), should lie within these intervals [Schuster96]: • Adventitia = density for fat –65±10 HU. • Lamina and media = size comparable to the CT resolution (pixel size ~0.75mm), due to PVA separately invisible; modulates the voxel values of vessel blood/calcification. • Vessel blood o With a contrast agent should have 200, but has 150–260 HU, o without contrast agent should have 80±10 HU. • Plaque = typically more than one piece in the vessel, high densities (from 320 in aorta, to 1500 HU in smaller vessels). An example of the densities around a vessel above knee is in Figure 3. In this artificially color-coded image, the vessel is surrounded by the fat and muscles and can be separated from the bone very simply. But this is not the case of all vessels in the datasets, since the density values of blood in the vessels overlap the densities of low-density bone and marrow (see the histogram in Figure 4). 6

Figure 3 An example of voxel values in the cross section between knee and hip

Figure 4 Bone and blood vessel with overlapping densities (graph courtesy of Milos Sramek, Austrian Academy of Sciences)

Some smaller vessels go directly into the bone and supply the inner bone structures. This fact can complicate the vessel tracking approach.

7

2.4.3 Densities along a segmented vessel Voxel densities along two vessels from the dataset 008 (file 100_198.dat) were plot in the following two graphs. The vessel density distribution in a manually selected vessel in the left leg (see Figure 5) shows two large drop-downs in the density values, above the knee (slices 380–530) and in the upper part close to the hip (slices 685–741). These decreases of the contrast agent concentrations are caused by two stenoses, which drastically reduce the blood flow and therefore also the concentration of the contrast agent. The blood flow is spread into the small collateral vessels. Below the large stenosis above the knee, blood returns to the vessel and the concentration grows back. In fact, it is even higher, since the contrast agent cumulates its concentration here (and also the veins become to be visible, as the contrast agent flow continues). The peaks in lower slices reflect an erroneously selected voxels with calcification.

600

P ix el values

500

A verage values

400 300 200 100

961

921

881

841

801

761

721

681

641

601

561

521

481

441

401

361

321

281

241

201

161

121

81

41

0 1

Value[HU]

700

S lice nu mbe r - starts from th e foo t

Figure 5 Average and voxel density values along the left leg vessel of the dataset 008

8

A similar result from the second leg of the same dataset 008, traced by our tracking algorithm, has smaller variations of values (of 80HU) from two reasons: firstly, as the algorithm searches the most homogeneous path; secondly, the stenosis in this vessel is not too large to reduce the blood flow drastically (see Figure 6).

Figure 6 Another graph of voxel density values along the vessel (output from the tracking algorithm)

9

2.5 Data in the dataset 008 In this section I describe detailed information about the dataset 008 with a contrast agent, which has been taken as an example. I have measured vessel diameters and the densities of the blood, surroundings and calcification, and also some profiles of density values. The measurements were done in a tool called Osiris [Osiris] and are collected in Table 2. Letters L, R mean in this table the vessel in the left and right leg, represents an interval of values between a and b. Position

Abdomen Aorta abdominalis Before bifurcation After bifurcation Iliaca communis After leg bifurcation

Slice Vessel Number diameter [mm] 3 24.5–27.5 mm

148

19.3 mm

163

R: 12.6 mm L: 12.6-13.4!! Large calcification, eccentric!!! R: 5.2mm

413

Upper part of knee

714

Lower part of knee R stenosis, lot of smaller vessels L: After trifurcation

738 766

Vessel densities [HU] border 32,45,59

Surroundings [HU]

Calcification [HU]





muscle R: R: L: L: < 4, 85>!!!!

R: L:

R: R: 2 vessels L: 3.7, 4.5 mm L: L: 5 vessels R: 4.5, 5.2 mm R: R: L: 4.5 mm L: L: fat muscle L: 4.5 mm R: R: R: 4.5–5.2 mm L: L: R: too many small vessels

R:

L: 4.5–5.9mm Up to 8.2mm

L: not recognized

L: L:

Table 2 Detailed study of vessels in one dataset

10

L: not found??

Not recognized

Not recognized

Figure 7 Abdomen – aorta abdominalis

In abdomen before bifurcation (see Figure 7) the vessel lies near the spine, very close to the vertical axis of the image. Sometimes it touches the spine, i.e., the border between them is very narrow, up to 1-2 pixels; moreover, partial volume artifact (PVA) can reduce the gradient in the gap. The vessel has nearly a circular shape with diameter about 25mm and it is surrounded by a tissue with densities smaller than zero. The density values inside the vessel are relatively homogeneous and if the border does not touch any other tissue, they will oscillate near the value of 148. The cross-sectional profiles have peaks if calcification is present (Profile-3) or valleys if the muscle does not touch the vessel (Profile-1, 2, 4).

11

Figure 8 Zoomed vessel before bifurcation

Before bifurcation (see Figure 8) the outer diameter is approx 19mm. In this case, almost the whole vessel wall is calcified (except one small part). The profile has two peaks (Profile-1) or one peak and a valley in the non-calcified part (Profile-2).

12

Figure 9 Two vessels after bifurcation

After bifurcation (see Figure 9) the outer diameter of the vessel is about 12mm. We can see that a lot of calcification is present in this vessel, that its shape is far to be regular, and the inner vessel diameter is definitely not circular. The open question remains: Where should be the vessel centerline in the left vessel situated to get the best CMPR? The answer should be answered after the testing phase. Center in the blood lumen will cause eccentric movement of the vessel while rotating, center in the calcification may create a complicated view, which will look like a completely blocked vessel with blood flow at one of its borders. In the left vessel in Figure 9 the blood density varies from 128 to 313 (may be also an artifact of the scanner?) and the vessel is not surrounded only by the values less than zero, but it touches the muscle with a similar positive value (see also the Profile-7).

13

Figure 10 Vessel below the knee

Vessel near knee and below the knee has the size about 4.5–5.9 mm, surrounded very often by a muscle with densities and fat (see Figure 10), or very small collateral vessels are present (the other leg, which is not shown here). The blood densities in lower parts were significantly higher – see Table 2 and the graph in Figure 5.

14

3 Vessel Segmentation Methods Used by Others This chapter took the most of my effort and still the not studied approaches exist. I review here the knowledge obtained from the literature search on the topic of vessel segmentation. The motivation for doing the review was clear: to use what proved to be good and to avoid what is not good enough. Our goal is a robust method for vessel segmentation from CTA datasets. Since I have found only one technical article, which uses this modality for a vessel display, I have collected everything about segmentation of vessel objects (I have obtained until now), described the principles and tried to depict problematic steps of each method. The most of methods described in both types of articles, i.e., medical and technical ones, deal with DSA (digital subtraction angiography) and MRA (magnetic resonance angiography). The methods described here are: • Thresholding-morphological method Miloš Šrámek • Wave vessel tracking by Cornelia Zahlten et al. • Vessel enhancement method by Frangi et al. • Direct vessel tracking by Wink et al. • Real time vessel enhancement by Poli et al. • Live Wire of Falcao, Barrett & Mortensen • 2D knowledge based method by Cleynenbreugel and Smets et al. For details about the methods read this chapter, for discussion about their applicability in the task of 3D vessel segmentation see Chapter 4. Notices and some unanswered questions follow in Chapter 6.

15

3.1 Miloš Šrámek’s threshold-morphological method Origin: Austrian Academy of Sciences People: M. Šrámek (http://www.viskom.oeaw.ac.at/~milos/) Reference: personal communication Ideas in this chapter were not published anywhere. They result from the personal communication with Miloš Šrámek, who is a specialist for rendering and segmentation of 3D medical datasets. As already mentioned in Chapter 2.4, both tissues of interest, namely vessels and plaque calcifications, overlap in their CT density range with the bone tissue. Therefore, we need a technique capable of independent labeling of all three tissue types. We can follow, in general, two complementary strategies: 1. An indirect one, in which the bone tissue is identified and removed from the CT data by means of masking. In this case vessels and calcifications remain the tissues with the highest voxel density, which enables a straightforward application of subsequent processing and visualization techniques. 2. A direct one, in which the vessels and calcifications are segmented and labeled from the original data. Šrámek prefers the indirect approach since (i) bones are much thicker objects then vessels and, therefore, their segmentation is less sensitive to imaging artifacts (partial volume effect), and (ii) bone segmentation errors do not have so dramatic consequences as segmentation of vessels can (e.g., missing vessel segments). According to the nature of the CT angiography data, Šrámek recommends a thresholding, supplemented by morphologic operations and connected component labeling as the most promising segmentation technique. Of course, the tools must be developed to deal with such situations (rare but possible) when the basic simple operations are not powerful enough to accomplish the bone segmentation correctly. The data with bone tissue masked out can be further used for the segmentation of vessels and/or calcifications (thresholding, statistical techniques, region growing, vessel tracking) and for visualization (MIP, re-projection, direct volume rendering). Since now, after masking out the bone tissue, the vessels and calcifications have the highest density within the volume, these tasks are significantly simplified.

For discussion see Chapter 4.1.

16

3.2 Wave Vessel tracking Algorithm by Cornelia Zahlten Origin: Center for Complex Systems and Visualization, University of Bremen People: C. Zahlten, H. Jürgens, H.-O. Peitgen, C. J. G. Everetsz Area: CTA, MRA vessel tracking References: [Zahlten95a, 95b] The algorithm described Cornelia Zahlten in her PhD thesis [Zahlten95b]. She worked on LENA diagnostic system. She mentioned two approaches and used the second one: Hysteresis thresholding with skeletonization and symbolic graph [Gerig93, Szekely94]: 1. Object pixels selection by hysteresis thresholding: Histogram was divided into 3 parts - object, background, and overlapping part. Pixels from overlapping part were added to object only if they were connected to the object pixels. 2. Object skeletonization by erosions. 3. Distance transformation [Danielson*] to determine the diameter of the vessels. 4. Symbolic description via graph - bifurcations and end voxels = nodes, vessel segments = links. Region growing with simultaneous graph generations [Zahlten93, Zahlten95a] This approach has been developed by Zahlten. It is a region growing in waves approach, which is enriched by bifurcation detection and vessel graph generation. At first, a seed point is put interactively near the root of the vessel tree. Then, the wave goes through the object. The word wave means a connected list of voxels belonging to the vessel that have been added in the current step. Until now it is a normal flood fill with 26connectivity voxels, each voxel remembers where the wave has come from (26 binary labellist) and the wave direction correction can be applied for distorted vessels. The major difference to the classical region growing is in the bifurcation detection and simultaneous graph construction. If the voxels in the wave (newly added voxels in current step) are not mutually connected, the bifurcation/n-furcation is detected and an appropriate number of nodes (each for one branch) is created and inserted to the graph. The new wave parts are sorted according to the number of voxels and processed from the largest to the smallest one. That result in the construction of the vessel tree in top-to-bottom order. Each new branch becomes a new label, except for the largest one (75% of the current wave), which holds the current label. During the visualization step each label becomes a different color to show the hierarchy in the vessel tree. For a correct branch detection (to avoid over-branching in places with a high vessel curvature), the wave correction is necessary to assure the next wave step would be perpendicular to the actual vessel direction. Details in [Zahlten95a,b]. Zahlten discusses also a data preprocessing step – median filtering, which eliminates thin vessels, or a possibility to use an adaptive filtering by local region growing, which preserves thin connections. The adaptive filtering is done iteratively, max five times. The possible best filtering method, which eliminates differences in contrast agent distribution, is the (time consuming) combination of both filters, i.e., an application of the adaptive regional growing AFTER median filtering. For experiences and tests see Chapter 4.2

17

3.3 Data enhancement and vessel tracking by Frangi Origin: Image Sciences Institute, Utrecht University Hospital / Utrecht Medical Center People: A. F. Frangi, W. J. Niessen, R. M. Hoogeven, M. A. Viergever Area: DSA, vessel segmentation in abdomen References: [Frangi98, 99a, 99b, 00], similar approaches also in [Krissian98, 99], [Sato97a, 97b, 98, 00], [Marchal90], [Vandermeulen89] and [Netsch96]. The principle of this three stage algorithm is at first to do a preprocessing of the dataset by a 3D filter (vesselness operator ν), which enhances tubular structures; then to track the tubular structures in this 3D space by finding the path with a minimal cost, which goes through the centerline of the vessels; and finally to detect the vessel borders. The vesselness operator uses secondary derivatives for the detection of the principal object shape and works in a scale space [Lindeberg94, 96], i.e., at each voxel, a set of filterings is done with different sizes of the vesselness operator and the maximal response, which comes when the size of the operator matches the size of the vessel, depicts the vessel size. Scale-space filtering approach was used also by Netsch [Netsch96] for detection of spherical objects. The algorithm had been used for a two step vessel segmentation of carotid arteries. At first, I will describe the 3D filtering itself, then the principles of the method applied to find the centerlines and the vessel borders in the preprocessed dataset.

3.3.1 Preprocessing by 3D filtering This preprocessing step is applied to enhance 3D tubular structures in the dataset. It uses secondary derivatives combined via a special operator. The same principle but different formulation of an operator uses Sato et. al. [Sato00]. I will describe the operator used by Frangi et. al. The vesselness enhancement filter ν of Frangi et. al. [Frangi98] is based on the eigenvalues of a Hessian matrix (a matrix of partial secondary derivatives) evaluated in each pixel of the image in different scales σ. It has the following properties: • It enhances the tubular structures, • filters out blobs and plate-like structures, • works in the scale space (in the range between min and max scale, which are given by the size of searched vessels) . • but it - reduces the vessel diameter - has dropouts at bifurcations (detects them as to be plate-like) - may be too much sensitive to bone and calcification The filtering is done as follows: For each scale σ do: For each voxel do: 1. Compute eigenvalues of the Hessian matrix (the matrix of second partial derivatives) for the scale σ 2. Order and label the eigenvalues so that |λ1| ≤ |λ2| ≤ |λ3|. The respecting eigenvectors u1, u2 and u3 now point out the singular directions: a. u1 along the vessel b. u2 and u3 directions in orthogonal plane 18

3. For vessels (bright tubular structures in the darker environment) should: |λ1| ≈ 0, |λ1| ≤ |λ2|, λ2 ≈λ3 . The sign of λ2 and λ3 is important – minus for vessels brighter than background. 4. Compute the scalar value of the vesselness ν The vesselness ν(x, σ) operator for a given scale σ in voxel x: ν(x, σ) = 0 … if |λ2| > 0 or |λ3| > 0, 2 2 2 2 2 2 = [1 – exp(–RA / 2α )] exp(–RB / 2β ) [1 – exp(–S / 2c )] … otherwise. The operator consists of three parts normalized to have response in interval : • Anisotropy term RA= |λ2| / |λ3| should be close to 1 (cross-sect symmetry). • Blobiness term RB = |λ1| / sqrt(|λ2| |λ3|) should be small. • Degree of image content S = sqrt( (|λ1|)2 + (|λ2|)2 + (|λ3|)2 ) should be large. This is similar to trace of Hessian matrix Tr(H)=λ1 + λ2 + λ3=Ixx+Iyy+Izz = LoG(H) That means the parts have this behavior: For increasing RA the value in [] rises from 0 “ 1. For increasing RB the value of exp() falls from 1 “ 0. For increasing S the value in [] rises from 0 “ 1. Parameters α, β, c tune the sensitivity of the filter to deviations in RA, RB and S. Typical values used in the article [Frangi98] are α = β = 0.5 and c = ½ of the maximal Frobenius norm of the Hessian matrices, i.e., the length of the vector of lambdas (λ1, λ2, λ3). Other possibility is to use a normalization by the maximal intensity 0.25σ2Imax (information from the mail from Frangi). The resulting response of the filter in pixel x is computed as a maximal response over the scales ν(x) = max( ν(x, σ) ) for σ in < σmin, σmax >. The scale σ where ν reaches its maximum determines the size of the vessel.

3.3.2 Vessel tracking & border detection After filtering by a vesselness operator, the vessel centerline is detected – modeled by using B-spline curves, then the vessel wall is segmented – using a tensor product B-spline surface. Vessel centerline has to be connected and smooth. Therefore, it is constructed via energy minimization of a snake. Internal snake energy term provides a regularization term depending on the first and second derivatives of the curve. External snake energy term which forces the snake to the center of the vessel is computed via integration of the values of vesselness (vessel enhancement filter described above) along the vessel centerline. The vessel wall is computed similarly – like snake for surfaces. The external term takes into account knowledge about MR acquisition (see [Frangi99a, equation (9)] for details).

For experiences and first tests see Chapter 4.3 19

3.4 Direct vessel tracking in 3D with vessel centerpoint localization Origin: Image Sciences Institute, Utrecht University, The Nederlands People: O. Wink, W. J. Niessen, M. A. Viergever Area: CTA in abdominal aorta (AAA project), MRA References: [Wink00], [Verdonck95] This is a following work of the same group as Frangi et al. in Section 3.3. Very similar tracking approach called imaginary catheter was published as a work in progress also by Verdonck et al. [Verdonck95]. Wink et al. describes an interactive segmentation method, which works locally, without preprocessing of the whole dataset and needs about 10s per vessel. The user selects two starting points A and B in the thick part of a vessel and runs the tracking algorithm. The points give a possible direction of the vessel centerline a = B-A. The method estimates the position of the next candidate point C in this direction of vector a, in the distance |b| based on the current minimal vessel diameter dmin. Precisely |b| = αdmin. Then the precise position of a new point Cnew is computed. In the square surroundings of the candidate point C, which is perpendicular to the vessel axis direction a, they compute for each point the likelihood that it is a centerpoint of a vessel. This is done by casting of a fan of rays from each point in the square. The rays end at possible border of the vessel. Then, they detect the most possible vessel centerpoint Cnew in this square and store this new position as a new centerline end. It becomes the part of up-to-now detected vessel and a new candidate point is generated in just computed axis direction. The estimated point C need not to lie inside of the vessel - it is enough, when the vessel center lies in the square around it [Wink00, Fig. 8]. The size s of this square is computed as s=β|b|. The center likelihood is computed as follows: For each pair of rays in opposite directions the likelihood of being-a-center is computed as a ratio of the shorter to the longer of the distances to the vessel border. The border is detected as a falling gradient in the same direction as the direction of the ray. This has an advantage since by this approach we find the border of bright vessel or the END of calcification. Note: The authors also propose a possibility to force a search direction to the central vessel direction, to limit the curvature of a vessel by a coefficient and to search a whole tree of possible vessels and continue in the direction of the highest sum of center likelihoods (which is the most important for our task). They also propose the possible approach, which is based on exactly the same principle as our modification to the LW (see Chapter 4.6) —to define the START and END points and by the use of the dynamic programming or the tree search they propose to find the vessel centerline. The more points are defined, the more reduced is the data space. For experiences and first tests see Chapter 4.4.

20

3.5 Real time vessel enhancement and detection Origin: a) Department of Electrical Engineering, University of Florence, Italy, b) School of Computer Science, The University of Birmingham, UK, AI lab People: a) Poli, Valli, 1994 b) Poli, Cagnoni, since 1996 References: [Poli94, Poli96]. The following work with genetic algorithms optimization [Cagnoni97, Cagnoni99]. Area: DSA, real-time vessel segmentation The principle of this method is similar to the approach of Frangi et al. (see Chapter 3.3). Both use the Gaussian filters in different scales for vessel enhancement. The main points of Poli’s algorithm are: • Application of special directional filters (see Figure 11), • separation of the Gauss filter computation as much as possible => into atomic filter computations, to achieve a real-time response (convolution is a linear operation, so a convolution with a big kernel can be done by addition of the results of convolutions from two smaller filters, …), • application of a validation step to ignore the response to step edges – the filter response is accepted iff also two pixels in distance ±v perpendicularly to n are outside of the vessel, i.e., have lower values than a central pixel. Distance v should be greater then the largest structure of interest. This step removes negative values near the vessel borders and noise, but also thin vessel after branching from thick ones, • hysteresis thresholding of the validated images helps if the noise is too high.

Figure 11 Directional kernels for different half-length l and half-width ω [Poli96]. a) l=ω=σ,

b) l=σ, ω=2σ,

c)l=2σ, ω=σ,

21

d) l=2σ, ω=2σ

In greater details • 1D Gaussian is approximated by the binomial approximation, which is valid if 2σ2 and z are integers g(z) §VTUWπ)σ / 2^4s2 (4σ2 over z + 2σ2) , • all the filters are constructed from directional first derivatives of Gaussian => Gn(x) • these directional derivatives of Gaussian can be decomposed into Gn(x) = n.∇G(x) = n1.δG(x)/δx1 + n2.δG(x)/δx2, where n1 and n2 are components of n = [n1, n2], which is a vector orthogonal to vessel direction • the partial derivatives of Gaussian can be further approximated by means of properly shifted Gaussian kernels δG(x)/δx1 = 1/σ [G(x + (σ/2, 0)) – G(x – (σ/2, 0))] • for the direction vessel enhancement a more complicated filter is computed from the edge response derivatives of Gaussian filters (see also Figure 11): o subtraction of two derivative filters properly shifted (by ±ω) gets a filter which responses to vessels (bar-like structures) of given half-width ω in given direction n Bn(x,ω) = Gn(x+ωn) – Gn(x–ωn) o to increase the directional sensitivity of such a filter, a combination of 2N+1 kernels Bn(x,ω) is applied Bn (x, ω ) =

•

•

N 1 k B n (x + l t, ω ) , ∑ 2 N + 1 k =− N N

where t is a vector in vessel direction (orthogonal to n) to reduce the filter artifacts, the authors decompose the directional derivatives not only in two directions, but in four 2 2 2 2 n = {[1,0], [0,1], [ , ], [ ,– ]} 2 2 2 2 As a result, they take a maximum response over all n,l,ω ε (x) = max Bn (x, ω , l , v) * I (x) n ,ω ,l

• •

For their test image the best was σ = 1, ω = 2, l = 2, => seems that NO integration over set of scales is necessary ?!?!?! They recommend simultaneous application of two values of v: v1 = 1 and v2 > σ.

For discussion see Chapter 4.5

22

3.6 Live Wire, Live Wire on the Fly and Intelligent Scissors Origin: Faculty of Electrical Engineering, State Uni. Campinas, Brazil Medical Image Processing Group, Dept. of Radiology, Univ. Pennsylvania People: A. X. Falcao, J. Udupa, et.al. (LW), Mortensen/Barrett (Intelligent Scissors) Area: any object border segmentation References: Live-wire: [Falcao97a,b,c,98,00, Šonka98 p.162], Intelligent scissors: [Mortensen95, Bartrolí97] Principle The method was originally designed for interactive segmentation of boundaries in difficult images with faint signal of boundaries or even gaps in boundaries, for objects with similar boundary properties around them and regions distorted by noise. In general, it speed-ups the user interaction by interactive offering the boundary segments between the user defined starting point and current position of the cursor. After selection of the end-point of the segment, it becomes a new starting point and the whole process of boundary segment selection starts from this new point. The fundamental ideas of this method and the name live wire (LW) had been developed in cooperation of groups around Udupa and Barrett. But then (95/96), the development continued independently and Mortensen [Mortensen95] introduced the name intelligent scissors. The important principles used by Falcao et. al. [Falcao00] follow: • • • • • •

•

• •

The boundary is an oriented, closed and connected contour. (connected oriented Jordan curve – Jordan curve is a closed simple plain curve, i.e., not self-intersecting, topologically equivalent to a circle). It is formed from a subset of oriented edges such that the sum of the costs of these edges is minimal. The oriented pixel edge is defined between each pair of four-adjacent pixels, i.e., there are four possible edge orientations – up, down, left, right. The oriented pixel edge element which is a part of the boundary is called bel = boundary element Each bel b=(q,r) has its location (between pixels q and r) and orientation (such that q is always inside the boundary, i.e., to the left of b – a clockwise orientation of the boundary segment – [Falcao00, Fig. 2]). Each bel b is assigned a cost(b). More precisely, each bell is assigned a set of features, which values characterize the bel, but they are all converted into a single cost value cost(b). Details about the computation of the costs, about the training phase and also the explanation of the relation to intelligent scissors [Mortensen95] are in the article [Falcao98]. Some details about costs computation from [Udupa00, p.34]: o Scene intensity of pixel to the left from the bel (i.e., the value of the inner pixel, which is the smaller value of both) o Gradient at the bel o Cost = Inverted Gaussian function of both o Mean and covariances are set up via training Pixels and bels form nodes and arcs of a weighted directed graph The modified Dijkstra’s minimal cost path algorithm is used.

23

• • •

• • •

An algorithm computes a tree of shortest paths – from the starting node to all nodes in the graph in O(m+nC) time, where m is the number of arcs (bels), n the number of nodes (pixels) and C is the maximum cost assigned to any arc (bel) . The costs are stored as integers from the interval C ∈ [0, C] (not as real numbers) The bottleneck – a logarithmic complexity of the access of the priority queue – is overcome by a circular queue of pointers to C+1 buckets (bucket sort) – solution of [Dial-28] . This solution is possible as they use a cumulative cost (sum of the costs along the path) and the cost is stored “modulo C” in the circular queue. In each timestep, the lowest value in the queue is the current cost value and the maximal newly inserted value is equal to current cost plus C. Queue with C+1buckets is enough for the storage. An insert and vertex update in O(1), vertex with minimal cost search in O(C). Lists of vertices in buckets are double connected. Live wire searches all minimal cost paths to all pixels/nodes of the whole image.

Speeding-up the live wire: a) Live Wire on the Fly (LWOF) – principles described by [Falcao00, p.57] • LWOF works only in the region depicted by the cursor (The authors do not describe how to enlarge the current region if the user moves fast far from the current position). • LWOF finds exactly the same boundary, i.e., it computes exactly the same minimal cost paths, since all the boundary pixels are stored in the queue, and as the processing continues in each step with the lowest-cost element on the top of the queue. The cheapest path is traced and more expensive path regions remain deep in the queue. • LWOF follows this ideas: o Do not compute longer paths than current minimum path – i.e., quit computation of minimal paths after finding the minimal cost path. o In the already computed tree (region) just display the resulting boundary segment – the branch of the tree computed before. No need to start additional computation. o The larger tree of minimum cost paths contains completely the previously computed smaller tree. For longer paths simply apply the Dijkstra algorithm only for pixels which are new in the (larger) region. b) Enhancement of “Intelligent Scissors” • Vilanova-Bartrolí [Bartrolí97] has enhanced the method from [Mortensen95] into 3D. • She tested that it would be precise enough, if the recalculation is done in very small areas near the mouse cursor – sometimes a sub-optimal path is found, but the quality of segmentation results does not loose considerably. • She describes also an idea of training from the last t points of segmented contour [Mortensen95] – this approach works well for objects with consistent edge properties. LUT for converting of gradients into costs is created. • She describes also the construction of costs from gradient magnitude, gradient direction and from Laplacian zero crossings. Vilanova-Bartrolí also describes a distance correction of the filter mask for anisotropic volumes For experiences and tests see Chapter 4.6.

24

3.7 Knowledge-based 2D approach from ESAT KU Leuven Origin: ESAT KU Leuven, The Netherlands People: J. Van Cleynenbreugel, C. Smets (PhD), F. Fierens, P. Suetens, A. Oesterlinck, D. Vandermeulen Area: DSA References: [Cleynenbreugel87, Delaere90a,b,c, Smets90]. This chapter describes a hierarchical heuristic method for a knowledge-based vessel segmentation from projections in subtraction angiography (SA). The development was started by Cleynenbreugel et al. [Cleynenbreugel87], and continued in [Delaere90a,b,c, Smets90]. The method applies a rule-based system and segments the blood vessels via a multi-stage approach. The proposed multi-layered image representation [Smets90] has the following layers: 1) Pixels – with their position and gray value. They form the bottom level of the hierarchy, 2) Ridgepoint segments – are the high intensity line-structures after thinning and linking, 3) Blood vessel segments – are parts of a vessels detected as ribbon-like structures around extracted centerlines. A new blood vessel segment is created at each vessel crossing. 4) Blood vessels – are created by merging the blood vessel segments. Assumption of a treelike structure is utilized, models of crossings, connections and T-branching are applied. Intersections are labeled according to geometrical and topological knowledge. 5) List of anatomically labeled blood vessel segments is created by interpretation of vessel segments according to an anatomical model of coronary artery. The anatomical labeling may overrule the available geometrical knowledge in some cases. 6) 3D artery structure is reconstructed from the available multiple DSA projections An important aspect is the splitting of large amount of available heuristic knowledge into separate blocks, which simplifies integration of new heuristic models. Also the feedback strategy, which allows re-computation of missing attributes and the updating of interpretation labels makes the method more robust. For ideas about this approach see Chapter 4.7.

25

4 Applicability of methods and tests This chapter describes my experience with the algorithms described in Chapter 3 above and collects my opinions about applicability of described methods to the task of vessel segmentation in CT angiography datasets. Up to now, we have tested two algorithms: wave vessel tracking (3.2) and live wire (3.4). Therefore, about the other methods I mention here only opinions, which are not supported by precise algorithm evaluation L. The important part for the future decision about the tool for vessel segmentation in CTA datasets is also in the judgment between two principal approaches. Between an interactive tool with extremely speeded-up user interaction and a semi-automatic tool which requires absolute inevitable of the user interaction. The interactive tool strategy will concentrate our work on the computer supported user interaction methods and a specialized user interface design – in the extreme, on a relatively simple vessel editing tool which helps the user to define the vessel centerline with a minimum amount of clicking or even he can use the current methods enriched by a centerline points editing and by automatic centerline generation only in the simplest cases. The simple tool may be ready very fast but it has a limited use. The semiautomatic strategy is much more complex. It has to apply a knowledge-based approach and has to try to cover the maximum amount of different pathological cases. But the advantages for the radiologists, mainly saving of the interaction time, are so high that we will try to proceed in this direction.

4.1 Notes about Miloš Šrámek’s threshold-morphological method (Algorithm from Section 3.1) I have not tested this method and the only result I have seen was the idea behind that and one image. I think, that the principle of bone removal is very promising mainly for the visualization via MIP. For segmentation itself, false removal of vessel parts may appear and the following vessel segmentation algorithm will suffer from a problem of jumping over gaps in the data. I also think that the method as was described cannot handle the variation of intensities in the vessels automatically and demands an intensive user interaction. Due to thresholding it would also distort the vessel shape and that would affect the following measurements.

4.2 Tests of Vessel tracking by Wave Algorithm (Algorithm from Section 3.2) We have implemented a simplified version of this approach with an adaptive thresholding by means of bimodal histogram without a curvature correction. Before application of this algorithm, it is necessary to enhance the contrast between the vessel and the background by setting of a correct windowing.

26

I have tested the algorithm at the beginning of my bibliography search. I have used a two-step segmentation: I have masked out the vessel surroundings by interval selection and then did the vessel tracking itself. The masking of the volume had the same effect as the process of windowing. The algorithm was successful for the parts of the leg starting below the knee and ending at the lower part of the body if the windowing/masking isolated all the bridges to the surrounding tissues (The simplest dataset was the 003a which has higher resolution – see two right panels in Figure 12). The algorithm had problems in small vessels in lower part of the foot where it stopped very often as the vessel was no more homogeneous enough due to the noise and may be PVA (see Figure 12 left). Also if the vessel touched the bone in abdomen or if the star-like artifact was present it spread into the surrounding tissue or even filled up the whole 3D dataset. Another important disadvantage was a non-predictable erosion of the detected vessel caused by ignoring the voxels out of the windowing interval.

Figure 12 Datasets 001 (left) and 003a (right) and segmented vessels

If I think about the windowing, which is necessary to achieve any meaningful result, I suppose that it suppresses the adaptive thresholding to minimal influence and causes unpredictable erosion of the detected vessel. The detection of small vessels is then very hard (see limited length of vessels in the left panel in Figure 12). And the algorithm without branching corrections reduces itself back to the region growing. Also due to very low interactivity and many fails I realized not to continue in testing and not to use it for the project.

27

4.3 Test of Frangi’s vessel enhancement algorithm (Algorithm from Section 3.3) The algorithm has been developed for MRA vessel segmentation where no signal from bone is present, and especially for abdomen, where the vessels have a large diameter. As the bone is present in CTA datasets I think that the approach cannot be directly applied as is. But I think that the 3D filtering step, which enhances tubular structures of diameter from a given interval and even detects the vessel diameter, can be used in two following places: • In our modification of live wire vessel tracking (see Chapter 4.6 below) as a supporting feature helping in the detection of “being in the vessel”, or • as a post-processing step for precise positioning of detected path (e.g., the minimal path) to obtain a true centerline of the vessel. The promising results of tests in 2D are in Figure 13, as the filter enhances tubular structures and borders of circular structures. I need to do also the tests in 3D and also the tests how the brighter bone and calcified parts influence the response of the filter.

Figure 13 Testing image and a vesselness filter response

Pros & cons • For large diameters the vesselness method is precise enough (the case of carotid arteries) • For small vessels of diameter of 2-3 pixels (like in brain or foot), the Hessian is not computed in the correct centers of the vessels, but in the centers of sampled voxels – and the variation of anisotropy term RA= |λ2| / |λ3| is therefore too high – study of INRIA [Krissian98, Krissian99] • Frangi uses snakes for the vessel extraction from the vesselness data. It should prefer detection of smooth vessel centerlines, i.e., should work globally in a local neighborhood. This may be useful for jumps over bifurcations or calcifications.

28

Unanswered questions: Frangi used second partial derivatives of Gaussian in different scales. Up to now, I miss an exact description of how to construct a discrete kernel for given standard deviation σ. This is an important thing as masks for a given σ are one of the foundation stones of the scale space theory. • There are some examples of kernels, e.g., Laplacians of Gaussian (LoG) kernels in [Sonka00, (4.58) on p. 85] but they differ substantially in values in the diagonal direction, where the discrete approximation has significantly larger numbers. I use regularly sampled continuous derivatives functions for masks. • Other aspect how is the mask size is related to the standard deviation σ? The mask size has to be odd, of edge (3 to 6)* σ… From simple tests of approximation of a Gaussian filter results that the kernel size of 3*σ gives only 75%, size of 4*σ gives 91% and 6*σ gives 99% of the precise kernel values. This means the precise kernel has the sum of its values equal to 1, the others have 0.75, 0.91 or 0.99.

4.4 Notes about direct vessel tracking by Wink (Algorithm from Section 3.4) This method is the only one, in this report, which has been directly applied to CTA data. It seems to be very interesting, as it does not search the whole data volume but only the promising part, where the vessel should be located. It also handles calcifications in CTA with correct estimate of the OUTER border of calcifications, ignores perpendicularly outgoing vessels as they have a low contribution to the likelihood, selects one vessel in bifurcations (no wrong turnings or leaving of the vessel), and estimates also minimal vessel diameter. It is invariant to intensity, scale and rotation. On the other hand, the method was used for vessels with a large diameter in abdomen, which is not our case. Also the necessity of permanent user interaction and the start points selection may be a drawback in our large datasets. It needs a simultaneous visualization and a specialized user interface.

4.5 Notes about real-time vessel enhancement (Algorithm from Section 3.5) This algorithm differs from the Frangi’s vessel enhancement by application of different derivative kernels, with different width ω and length l. They also use integer arithmetic and a sophisticated decomposition of the kernels into addition of a large number of responses of only 2-pixel kernels, which results in a real-time vessel segmentation. Surprising is that Poli et al. does not use the kernels over the range of scales σ but only for σ = 1 !!! Also the best compromise values of σ = 1, ω = 2, l = 2, seems strange to me. The ideas of optimal computation of the convolutions are inspirative and should be have in mind if any filtering approach will be applied.

29

4.6 Tests of the modified Live wire (Algorithm from Section 3.6) Pros & cons + Combination of local and global information, speeds-up the interactive segmentation. – Must be supervised as it finds the shortest path, not the vessel. A correct window setting is necessary to stay in the vessel. – Finds a path inside the vessel. + Speeding up is possible by means of local neighborhood graph computations. [Falcao97a,b], or best-first search [Barrett96] . Falcao et. al. [Falcao00] describe also the memory requirements of this algorithm but the information in the article is unclear. They write: Dataset 50 slices 256x256 pixels each, 2 Bytes / pixel requires 26MB for pre-computed bel costs and 96MB total RAM. Typical C is 255 or 4095 (8 or 12 bits) An unanswered question remains how did they compute the 96MB of total memory they need, as I can estimate only about 6MB for storage of bells + 6MB for data… Bell array for one slice contains 2.n.(n-1) values, i.e., n.(n-1) for rows and the same for columns. [2x255x256 = 130 560 values / slice] 50 slice means 49x more inter-costs [6 397 440 values] => 4.26 Bytes/cost???? Data 2x50x256x256 = 50*131 072= 6 553 600 bytes

Our modification for vessel tracking: Our modification does not search the boundary but the path through a homogeneous region in a given window of values (see Figure 6 for an example of such a path). 1. Click on the start point in the vessel – the Abdomen Aorta and all vessel endpoints 2. Perform the shortest path search between the starting point and all the points in the image (graph), prefer homogeneity, jump to higher densities is ok (calc), jump out of the given window is highly penalized. Until now, the gradient is only between neighbors (difference). But it works very intuitively and FAST. Tested in 2D and 3D with promising results. 3. Draw the paths between start and all end points. At this point, any shortest path from the starting point to any other endpoint can be found interactively. Pros & cons of our modification + The algorithm detects vessels with a higher probability of success. – Non-optimized version needs a large amount of memory – now it is solved by swapping to disk, – a modification of any point inside the vessel means the setting of this point as a starting one and the complete re-computation of the shortest path tree, – the graph parameters are set – no update to the dataset via teaching phase is possible. Possible enhancements: • The speed and memory claims can be overcome by means of local heuristics an obviously also a special PC can be built, • Interactive CMPR in three major directions can be created for simultaneous displaying of the live wire (similar as frontal, sagittal and coronal views) • Ranking operators for limitation of PVA influence on the bone borders can be included. 30

4.7 Notes about a knowledge base approach from KU Leuven (Algorithm from Section 3.7) This approach has not been tested, that is why this chapter is called notes… • • •

Cleynenbreugel/Smets (SA) uses thin vessels detection via a maximal intensity detector, which is not applicable to CTA as we have the whole dataset, no 2D MIP of vessels. The approach seems to be a robust one, as it combines a bottom-up approach with a feedback based recalculations and re-labeling and applyes not only a geometric but also an anatomical knowledge. Our situation is more simple as we do not need to deal with overlapping and crossings of vessels caused by a 2D projection of a 3D object.

4.8 Ideas and interesting methods waiting for a study This chapter contains moreover a list of ideas and references to interesting approaches, than studied algorithms. Therefore, they are collected in a separated chapter.

4.8.1 References to other interesting approaches Simultaneous border detection A relatively complicated approach for precise vessel detection in DSA angiograms [Sonka94, Sonka99, pages 730–733]. It uses a 3D graph search method for 2D, applied in two cases: Œ Centerline detected interactively by an operator and L and R borders searched precisely, or Œ Some user-defined points define path inside the vessel, simultaneous border detection finds the approximation of borders (in 4x4 sub-sampled image) and final step detects the centerline in the middle between these L and R borders. o Other methods jump out to other bright vessels o The maximum brightness must even not be in the vessel (may be in case of stenosis?) o Šonka uses warped image – straightened vessel axis o + finds the smoothest and correct vessel centerlines in more cases o uses polyline approximation, no splines. At the end it is smoothed by filtering. Wall and plaque detection in ultrasound [Sonka99, pages 733–737] Works also with calcified vessels but has a better resolution, as the acquisition is performed from inside of the vessel of interest. Lung tree extraction [Sonka99, pages 738–744] Rule based system, which may be interesting also for vessel tracking. Fuzzy segmentation of eye vessels [Tolias98] A fuzzy C-means clustering algorithm for vessel segmentation in 2D retinal images. Segmentation using intrinsic shape information [Shiffman99]

31

At first, coherent meaningful regions are detected in volumetric datasets (salient regions), user than selects some meaningful regions from the salient ones, than the whole groups of regions are automatically searched within the volume of salient regions and finally, the regions in raw data are masked out.

4.8.2 Other ideas to think about Subdivide the dataset into parts and for different parts use a different approach • Seems as a promising principle as the data nature differs in abdomen and in foot. Simple by knee-joint & femur, complicated by the spine as they touch and the calcification merges with the bone (but the vessel is circular here and the vertebra has a shape of a parabolic arc) • But how many parts? Maybe click at the beginning in abdomen, abdomen bifurcation, knee, first part of trifurcation, second bifurcation, ankle, ends of the vessels Subtract contrast agent datasets and native datasets This simple approach (in fact very similar to standard DSA) would work well, if the patient were not move. But only fastening is not enough as for correct handling of tiny vessels also physiologic movements matter. Proper registration would be necessary but it would complicate this approach too much for a practical usage. For correctly registered images see an example in Figure 14, where the vessels form the brightest parts of the image.

Figure 14 Correctly registered slices and their subtraction

Transfer the data values before processing Before any segmentation transform the densities in the dataset by the function that enhances vessel values and suppresses the values of calcification. This should homogenize the vessel values for simpler processing

32

Use different segmentation methods for different parts of the volume Divide the vessels into parts and apply different methods for segmentation for each part (15 defined parts from [Ruehm00] may be too much). + different vessel surroundings of the vessels => different method can be more robust, – necessity to recognize (locate) the parts Other ideas • Use a rubber model of the vessel topology (tree) • Use anatomical knowledge about a normal vessel tree • Click to center, region grow to the borders • Use geometry – vessel is circular, except of the branching • Use position of the vessel in the volume (seems unusable, for great variations) • Match a model of bone. How many variations have the bones?

5 Visualization The data visualization should incorporate CMPR’s in ± 90° and a real-time 3D display of the detected vessels (ray-casting, MIP projection without bone with and without calcifications), possible color coded by the vessel or plaque diameter. I do not see any principal problem at this stage, if the segmentation of vessels and bone is available. This is mainly a task for a user interface programmer. There are two technical limitations to be overcome: 1. Limited number of slices. Our Mitsubishi VolumePro card [VolumePRO] can handle 450 slices of size 509x509pixels; the datasets have up to 783 slices. Application of two VolumePro cards in one PC may be up-to four- times slower, but has to be tested. 2. Limited operating memory. This can be a problem any time we need to handle large datasets with supporting data structures. Should not the memory consumption be reduced in principle, two brute force approaches can be used – a memory swapping or adding more RAM.

6 Notices and Remaining Questions I have wrote the discussion and questions into the whole Chapter 4. Here are the questions, which remain J. • • •

How to check the quality of segmentation? (Reference to current method or DSA?) How is the vessel diameter defined if it is not circular or it is filled by a calc? What should be detected as a vessel centerline for vessels with an extremely large or an extremely eccentric calcification, e.g., for the left vessel in Figure 9?

33

7 Conclusions I have summarized the information about angiography datasets acquired by a spiral CT scanner and about the appearance of vessels of interest in these datasets. Than I described the methods for segmentation of vessels in medical datasets found in literature and discussed their applicability to the task of CTA vessel segmentation. Studied methods were: • Thresholding-morphological method by Miloš Šrámek • Wave vessel-tracking by Cornelia Zahlten et al. • Vessel enhancement method by Frangi et al. • Direct vessel tracking by Wink et al. • Real time vessel enhancement by Poli et al. • Live Wire of Falcao, Barret & Mortensen • 2D knowledge based method by Cleynenbreugel and Smets et al. At this moment we are evaluating two segmentation methods: A vessel segmentation method based on combination of approaches of live wire and direct vessel tracking and a method for bone removal based on hysteresis thresholding.

Acknowledgments This work has been supported by the VrVis Center, Vienna, http://www.vrvis.at and by the company Tiani MedGraph, http://www.tiani.com, Vienna. I would also like to thank to the colleagues Rainer Wegenkittl, Edi Gröller, Dominik Fleischmann, Dominique Sandner and Martin ýapek for valuable discussions and comments that improved these report.

34

References [Bartrolí97]

[Barrett96]

A. Vilanova i Bartrolí. Interactive segmentation of medical images based on intelligent scissors. Projektarbeit in Fach Informatik, TU Wien, Betreuer Peter Hastreiter, April 1997. W. A. Barett and E. N. Mortensen. Interactive live-wire boundary detection. Medical Image Analysis, 1(4):331–341, 1996. Cited according to [Sonka00, p.94]

[Cagnoni97]

S. Cagnoni, Andrew B. Dobrzeniecki, Riccardo Poli, and J. C. Yanch. Segmentation of 3D medical images through genetically-optimized contour-tracking algorithms. Technical Report CSRP-97-28, University of Birmingham, School of Computer Science, December 1997. ftp://ftp.cs.bham.ac.uk/pub/tech-reports/1997/CSRP-97-28.ps.gz

[Cagnoni99]

S. Cagnoni, A.B. Dobrzeniecki, R. Poli, and J.C. Yanch. Genetic algorithmbased interactive segmentation of 3D medical images. Image and Vision Computing, 17(12):881–896, 1999. http://www.cs.bham.ac.uk/~rmp/papers/Cagnoni-IAVC1999.ps.gz J. van Cleynenbreugel, F. Fierens, C. Smets, P. Suetens, and A. Oosterlinck. Knowledge-based segmentation of subtraction angiograms. In Proceedings 10th international conference on information processing in medical imaging, pages 307–314, Utrecht, The Netherlands, June 22–26, 1987.

[Cleynenbreugel87]

[Danielson80] [Delaere90a]

[Delaere90b]

[Delaere90c]

[Falcao97a]

P. E. Danielson. Euclidean distance mapping. Computer Graphics and Image Processing, 14:227–248, 1980. Cited according to [Zahlten95b] D. Delaere, L. Maes, C. Smets, P. Suetens, A. Oosterlinck, and F. Van de Werf. A knowledge-based system for the fully automatic quantification of coronary stenotic lesions from two angiographic projections. In M. H. Loew, editor, Medical Imaging IV: Image Processing, volume 1233, pages 250–256, Newport Beach, California, USA, February 6–8, 1990. SPIE. D. Delaere, C. Smets, P. Suetens, A. Aubert, and F. Van de Werf. A knowledge-based system for the automatic quantification of stenotic lesions on angiograms. In Proceedings Conference on computers in cardiology, Chicago, USA, pages 317–320. IEEE 1991, September 23–26, 1990. http://bilbo.esat.kuleuven.ac.be/web-pages/people/Dominique.Delaere.html D. Delaere, C. Smets, P. Suetens, G. Marchal, and F. Van de Werf. A knowledge-based system for the 3 D reconstruction of blood vessels from two angiographic projections. In J Cornelis and S. Peeters, editors, Proceedings of North Sea Conference on Biomedical Engineering, 11 pages, Antwerp, Belgium, November 19–22, 1990. A. X. Falcao. Paradigmas de Segmentacao de Imagens Guiada pelo Usuario: Live Wire, Live-Lane e 3D-Live-Wire. PhD thesis, Institute of Computing - IC State University of Campinas - UNICAMP, 1997. In Portuguese. http://www.dcc.unicamp.br/~afalcao/

[Falcao97b]

A. X. Falcao. User-steered image boundary segmentation. Web page, 1997? http://www.dcc.unicamp.br/~afalcao/livewire.html

[Falcao97c]

A. X. Falcao. User steered image segmentation - live wire, pronto contour, pronto mask, 1997. http://www.dcc.unicamp.br/~afalcao/ author home 35

page, http://www.mmorph.com/prontomask/download/prontomask1.0.zip for Win, http://www.mmorph.com/prontomask/download/prontomask1.0linux.tar.gz for Linux, http://www.mmorph.com/prontomask/download/prontomask1.0-sunos.tar.gz for Solaris [Falcao98]

[Falcao00]

[Frangi98]

[Frangi99a]

[Frangi99b]

[Frangi00]

[Gerig93]

[Hong97]

[Krissian98]

[Krissian99]

A. X. Falcao, J. K. Udupa, S. Samarasekera, S. Sharma, B. E. Hirsch, and R. A. Lotufo. User-steered image segmentation. Graphical models and image processing, 60:233–260, 1998. A. X. Falcao, J. K. Udupa, and F. K. Miyazawa. An ultra-fast user-steered image segmentation paradigm: Live wire on the fly. IEEE Transaction on Medical Imaging, 19(1): 55–62, January 2000. A. F. Frangi, W. J. Niessen, K. L. Vincken, and M. A. Viergever. Multiscale vessel enhancement filtering. In W.M. Wells, A. Colchester, and S. Delp, editors, Medical Image Computing and Computer-Assisted Intervention MICCAI’98, volume 1496 of LNCS, pages 130–137. SpringerVerlag, Germany, 1998. http://www.cv.ruu.nl/people/alex/ A. F. Frangi, W. J. Niessen, R. M. Hoogeveen, Th. van Walsum, and M. A. Viergever. Quantitation of vessel morphology from 3D MRA .In C. Taylor and A. Colchester, editors, Medical Image Computing and ComputerAssisted Intervention - MICCAI’99, volume 1679 of Lecture Notes in Computer Science, pages 358–367. Springer Verlag, Berlin, Germany, 1999. http://www.cv.ruu.nl/people/alex/ A. F. Frangi, W. J. Niessen, R. M. Hoogeveen, Th. van Walsum, and M. A. Viergever. Model-based quantitation of 3D magnetic resonance angiographic images. IEEE Trans on Medical Imaging, 18(10): 946–956, 1999. http://www.cv.ruu.nl/people/alex/ A. F. Frangi, W. J. Niessen, P. J. Nederkoorn, O. E. H. Elgersma, and M.A. Viergever. Three-dimensional model-based stenosis quantification of the carotid arteries from contrast-enhance MR angiography. In Mathematical Methods in Biomedical Image Analysis 2000, 2000. In press. http://www.cv.ruu.nl/people/alex/ G. Gerig, Th. Koller, G. Szekely, Ch. Brechbuhler, and O. Kubler. Symbolic description of 3-D structures applied to cerebral vessel tree obtained from MR angiography volume data. Lecture Notes in Computer Science, 687:94– 111, 1993.Cited according to [Zahlten95b]. L. Hong, S. Muraki, A. Kaufman, D. Bartz, and T. He. Virtual voyage: interactive navigation in the human colon. In SIGGRAPH’97. International Conference on Computer Graphics and Interactive Techniques. Proceedings of the 24th annual conference on Computer graphics \& interactive techniques, pages 27–34, Los Angeles, CA USA, August 3–8, 1997. ACM. K. Krissian, G. Malandain, and N. Ayache. Model based multiscale detection and reconstruction of 3D vessels. Research Report RR-3442, INRIA, Sophia Antipolis, June 1998. http://www.inria.fr/RRRT/RR3442.html K. Krissian, G. Malandain, N. Ayache, R. Vaillant, and Y. Trousset. Model based detection of tubular structures in 3D images. Research Report RR3736, INRIA, Sophia Antipolis, July 1999. http://www.inria.fr/RRRT/RR3736.html 36

[Lindeberg94]

T. Lindeberg. Scale-space theory: A basic tool for analysing structures at different scales. J. Of Applied Statistics (Supplement on Advances in Applied Statistics: Statistics and Images: 2), 21(2): 224–270, 1994.

[Lindeberg96]

T. Lindeberg. Feature detection with automatic scale selection. Technical Report ISRN KTH NA/P–96/18–SE. Earlier version ISRN KTH NA/P– 94/04–SE, Department of Numerical Analysis and Computing Science, Royal Institute of Technology, S-100 44 Stockholm, Sweden, 1996.International Journal of Computer Vision, vol 30, number 2, pp. 77– 116, 1998. http://www.bion.kth.se/~tony/earlyvision.html and http://www.nada.kth.se/cvap/abstracts/cvap198.html G. Marchal, D. Vandermeulen, H. Bosmans, D. Delaere, P. Suetens, and A. Baert. A three-dimensional line filter for improved visualization of MRA .In Supplement to radiology, (76th Scientific Assembly and Annual meeting of RSNA), volume 177, and page 259. RSNA, November 25–30, 1990. Abstract.

[Marchal90]

[Mortensen95]

E. N. Mortensen and W. A. Barrett. Intelligent scissors for image composition. In SIGGRAPH’95. International Conference on Computer Graphics and Interactive Techniques. Proceedings of the 22nd annual ACM conference on Computer graphics, pages 191–198, Los Angeles, CA USA, August 6–11, 1995.

[Netsch96]

T. Netsch. Detection of microcalcification clusters in digital mammograms: A scale-space approach. In 3rd International workshop on digital mammography, Chicago, Illinois, U.S.A, June 1996. Abstract: http://www.radonc.uchicago.edu/IWDM/IWDMa4.html, The OSIRIS imaging software. http://www.genisis.ch/UIN.html

[Osiris] [P] [Poli94]

[Poli96]

[Ruehm00]

[Sato97a]

[Sato97b]

This reference is used for differentiation of my ideas and ideas of cited authors, if it is not clear from the text itself. R. Poli and G. Valli. Efficient enhancement and segmentation of blood vessels by integration of multiple directional filters. Technical Report 940701, Department of Electronic Engineering, University of Florence, Italy, July 1994. http://www.cs.bham.ac.uk/~rmp/htdocs/tr940701/tr940701.html R. Poli and G. Valli. An algorithm for real-time vessel enhancement and detection. Computer Methods and Programs in Biomedicine, 52:1–22, November 1996. http://www.cs.bham.ac.uk/ rmp/papers/PoliCMPB1996.ps.gz S. G. Ruehm, T. F. Hany, T. Pfammatter, E. Schneider, M. Ladd, and J. F. Debatin. Pelvic and lower extremity arterial imaging. American Journal o Roentgenology (AJR), 174:1127–1135, 2000. Y. Sato, S. Nakajima, H. Atsumi, T. Koller, G. Gerig, S. Yoshida, and R. Kikins. 3D multi-scale line filter for segmentation and visualization of curvilinear structures in medical images. In Proc. First Joint Conference of CVRMed II and MRCAS III, pages 213–222, Grenoble, France, 1997. http://splweb.bwh.harvard.edu:8000/pages/papers/yoshi/cr.html Y. Sato, C.-F. Westin, A. Bhalerao, S. Nakajima, N. Shiraga, S. Yoshida, G. Zientara, and R. Kikins. Tissue classification based on 3 D local intensity structure for volume rendering. In Proc. JAMIT Frontier ’97, Suita, Japan, pages 167–172, 1997. http://www.image.med.osaka37

u.ac.jp/~yoshi/paper/vr_jamit.pdf [Sato98]

[Sato00]

[Schuster96]

[Shiffman99]

[Smets90]

[Sonka94]

[Sonka99]

[Sonka00]

[SpaceVision] [Székely94]

Y. Sato, S. Nakajima, N. Shiraga, H. Atsumi, S. Yoshida, T. Koller, G. Gerig, and R. Kikins. 3D multi-scale line filter for segmentation and visualization of curvilinear structures in medical images. Medical Image Analysis, 2(2): 143–168, 1998. http://www.image.med.osakau.ac.jp/~yoshi/paper/linefilter.pdf Y. Sato, C.-F. Westin, A. Bhalerao, S. Nakajima, N. Shiraga, S. Tamura, and R. Kikins. Tissue classification based on 3 D local intensity structures for volume rendering. IEEE Transaction on Visualization and Computer Graphics, 6(2): 160–180, April/June 2000. E. Schuster. Bimedizinische Technik, chapter Bildgebung + Computertomographie, pages 19–23, 95–96.Institut für Medizinische Computerwisschenschaften, Uni-Wien, 1996. S. Shiffman. Segmentation of Medical Image Volumes Using Intrinsic Shape Information. PhD thesis, Stanford University, February 1999.Available as Technical Report CS-TR-99-1617. http://cstr.cs.cornell.edu/Dienst/UI/1.0/Display/stanford.cs/CS-TR-99-1617 or ftp://elib.stanford.edu/pub/reports/cs/tr/99/1617/ C. Smets. A knowledge-based System for the Automatic Interpretation of Blood vessels on Angiograms. PhD thesis, Katholieke Univesiteit Leuven Dept of Electrical Enginering – ESAT, Center for processing speech and images – PSI or Dept. of Pathofysiologie, Leuven University Press, 1990.Acta Biomedica Lovenaniensia 20. http://www.esat.kuleuven.ac.be/psi M. Šonka, M. D. Winniford, X. Zhang, and S. M. Collins. Lumen centerline detection in complex coronary angiograms. IEEE Transaction on Biomedical Engineering, 41(6): 520–528, June 1994. 0âRQND9+ODYiþDQG5%R\OHImage Processing, Analysis and Machine Vision. Published by PWS Publishing, Brooks/Cole, Pacific Grove, CA, 2 editions, 1999. M. Šonka and J. M. Fitzpatrick, editors. Handbook of Medical Imaging Volume 2, Medical Image Processing and Analysis, volume 2.SPIE, Belligham, WA, 2000. SpaceVision – a 3D medical diagnostic system. Tiani Medgraph, GmbH. http://www.tiani.com/Software/SPACEVISION/index.htm G. Székely, T. M. Koller, R. Kikinis, and G. Gerig. Structural description and combined 3D display for superior analysis of cerebral vascularity from MRA .In Visualization in Biomedical Computing, volume 2359, pages 272– 281, 1994.Cited according to [Zahlten95b].

[Tolias98]

Y. A. Tolias and S. M. Panas. A fussy vessel-tracking algorithm for retinal images based on fuzzy clustering. IEEE Transaction on Medical Imaging, 17(2): 263–273, April 1998. [Vandermeulen D. Vandermeulen, C. Smets, P. Suetens, G. Marchal, J. Gybels, and 89] A. J. Oosterlinck. Knowledge-based 3-d segmentation of blood vessels on a spatial sequence of MRI and ultrasound images. In S. J. Dwyer, R. G. Jost and R. H. Schneider, editor, Medical Imaging III: Image Processing, volume 1092 of Proc. SPIE, pages 142–148, 1989. [Verdonck95] B. Verdonck, I. Bloch, H. Maitre, D. Vandermeulen, P. Suetens and G. 38

Marchal. Blood Vessel Segmentation and Visualization in 3D MR and Spiral CT Angiography. Computer Assisted Radiology (CARS’95), pages 177–182, 1995. [VolumePRO] [Wink00]

[Zahlten93]

[Zahlten95a]

[Zahlten95b]

VolumePro500. PCI graphics card. Real Time Visualization division of Mitsubishi Electric, Sunnyvale, CA, USA. http://www.rtviz.com/ O. Wink, W.J. Niessen, and M.A. Viergever. Fast delineation and visualization of vessels in 3D angiographic images. IEEE Transactions on Medical Imaging, 19(4):337–346, April 2000. http://ieeexplore.ieee.org/iel5/42/18449/00848184.pdf, http://www.isi.uu.nl/Demos/demos.html C. Zahlten, H. Jürgens, H. O. Peitgen, C. J.G. Evertsz, R. Leppek, and K. J. Klose. Portal vein reconstruction based on topology. Radiology, 1993. http://www.cevis.uni-bremen.de/article-db/artikel/conni/EuropRadiol.ps.Z C. Zahlten. Reconstruction of branching blood vessels from CT-data. In B. Urban M. Göbel, H. Müller, editor, Visualization in Scientific Computing, pages 41–52. Springer-Verlag, Wien, 1995. http://www.cevis.unibremen.de/artdb/artikel/conni/Rostock.ps.Z C. Zahlten. Beiträge zur mathematischen Analyse medizinischer Bild- und Volumendatensätze. PhD thesis, Universität Bremen, http://www.informatik.uni-bremen.de/fb3/forschungsber.html, Uni-Bremen, centrum CeVis, 28.11.1995.Prof. Dr. H.-O. Peitgen, Prof. Dr. F. Nake.

39