Contrast-Enhanced IVUS Sequences. Sean M. O'Malley1, Morteza Naghavi2, and Ioannis A. Kakadiaris1. 1 Computational Biomedicine Lab, Dept. of Computer ...
Image-Based Frame Gating for Contrast-Enhanced IVUS Sequences Sean M. O’Malley1 , Morteza Naghavi2 , and Ioannis A. Kakadiaris1 1
Computational Biomedicine Lab, Dept. of Computer Science, Univ. of Houston 2 Association for Eradication of Heart Attack
Abstract. Numerous recent reports have noted the potential uses of contrast imaging in intravascular ultrasound (IVUS). In IVUS contrast studies, a particular region-of-interest in the vessel wall or surrounding tissues is monitored over time in order to assess the perfusion of a contrast agent introduced during imaging. However, as this procedure is often carried out in the coronary arteries, the resulting IVUS frame sequences suffer from a wide variety of motion artifacts which confound their analysis. As the majority of these artifacts are due to the activity of the heart, the most common method for reducing their effect is ECG gating. However, this gating method bears several known problems. We introduce an alternative, image-based method which addresses these problems by grouping frames from stationary-catheter IVUS sequences into sets of stabilized ensembles. Each ensemble contains frames captured when the catheter and vessel are similarly oriented. Our method requires only the ability to quantify pairwise differences in frame appearance and requires no secondary signals (e.g., ECG) or segmentation of the images.
1
Introduction
Contrast imaging is widely used in ultrasound as the basis for perfusion studies aimed at assessing blood flow through a region of vasculature or a particular organ. The contrast agents utilized in this context frequently consist of gaseous microbubbles contained in a stabilizing shell (diameter: 1-10 µm). These bubbles are designed to be efficient reflectors of incident ultrasound energy. Intravascular ultrasound (IVUS) provides cross-sectional images of the interior of blood vessels at a high resolution. While a number of methods for computer-aided analysis of IVUS sequences have been proposed over the last decade, IVUS perfusion methods are more recent and less developed. This is because IVUS has traditionally been used as a tool for studying vessel morphology, which does not generally require the use of contrast. However, contrastenhanced IVUS presents exciting opportunities for functional imaging [1]. Perfusion studies require that a particular anatomical region-of-interest be tracked over a period of time during which a contrast agent is introduced. In IVUS, while attempts are made to hold the imaging catheter (sensor) steady during recording, tracking is confounded by the inter-frame motion variability present due to imaging within the coronary arteries. In this paper, we propose a frame-gating technique that may be used to alleviate a wide variety of periodic
and non-periodic motion artifacts. Unlike previous efforts which either utilize ECG signals directly or attempt to mimic their performance through image analysis, we instead perform an appearance-based grouping of frames. In this way, unusual events (e.g., catheter slippage), common periodic effects (e.g., longitudinal catheter motion), and more subtle changes during recording (e.g., varying heartrate) may be implicitly accounted for. While the primary goal of our method is simply to extract a single stabilized subset of a longer frame sequence, by formulating the problem in terms of multidimensional scaling (MDS), a number of other useful operations may be performed. The MDS transform places points defined only by inter-point proximities into a metric space such that the proximities are preserved with minimal loss [2]. In our context, this allows the creation of a frame-similarity space which may be employed as a concise visual and numerical summary of an entire frame sequence. Clustering this space allows sets of frames with various similarity properties to be extracted efficiently. In addition, the method is self-calibrating in the sense that it need not be tuned to the grey-level, noise, or motion properties of the sequence at hand; e.g., we have applied it in an identical manner to 20- and 40-MHz IVUS data acquired in humans and swine with similar results. Note that as our goal is simply to obtain a sufficiently stable sequence, we do not require that the frames be captured at a specific fraction of the cardiac cycle. This paper is organized as follows. In Sec. 2, we review previous work in frame gating. In Sec. 3, we discuss the choice of similarity metric, creation of the frame-similarity space, and frame grouping. We summarize our results in Sec. 4.
2
Previous Work
The use of ECG signals for frame gating is ubiquitous in medical imaging as a means of stabilizing image sequences which suffer from cardiac motion artifacts. Given the original continuously-acquired frame sequence, gating selects a subset of frames to create a generally much shorter sequence. The principle behind this practice is that, if the data are always collected at a point in time where the heart is in a similar pose, they will exhibit greater visual consistency. A conceptual problem with ECG is that of choosing the most effective point in the signal (i.e., fraction of the R-R interval) at which to retain frames in order to obtain maximal inter-frame stability. In IVUS, the end-diastolic point (i.e., the R-wave itself, at 0%) is commonly chosen [3]. In principle the heart is relatively at rest at this point, though this choice is also a matter of convenience: selecting any fraction other than 0% is subject to decreased performance in the presence of heartrate variation due to the need to approximate where this point lies based on the time to the neighboring R-waves [4]. A further complication is that most ECG-based gating methods only capture one frame per cardiac cycle. For some applications (including our own) it would be desirable to retain more of the data by collecting multiple frames per cycle, as long as they are captured when the heart and catheter are similarly oriented. For pullback studies for which a corresponding ECG signal has not been acquired, a number of image-based methods have been developed which attempt
to derive a pseudo-ECG signal from the frame sequence itself [5–7]. However, even if accurate, these still suffer from the same drawbacks as ECG gating. It should be noted that some of the assumptions made by these methods are necessitated by their application to pullback sequences, where it is necessary to separate the effects of the pullback and of cardiac motion. For our application to stationary-catheter sequences, we make the assumption that all motion is unwanted; this allows the design of a more flexible gating tool.
3 3.1
Materials and Methods Contrast-Enhanced IVUS Protocol
We utilize IVUS sequences recorded in vivo in humans and swine. Each sequence consists of a number of pre-contrast frames, followed by a contrast injection proximally to the imaging catheter (which temporarily renders the lumen echo-opaque), followed by a number of post-contrast frames (Fig. 1). Each sequence is collected over a matter of minutes, the period of interest usually being from 600 to 900 frames long. For illustration purposes, figures presented in this and the following section represent sequences which have been decimated blindly (e.g., by retaining every 3rd frame). This has no effect on the behavior of the method, but renders the figures less crowded and easier to interpret.
(a)
(b)
(c)
Fig. 1. Representative frames from a typical contrast sequence (a) before, (b) during, and (c) after contrast injection.
3.2
Frame Gating
Our frame gating method follows three steps: (1) choosing a frame similarity or dissimilarity metric, (2) generating the frame-similarity space, and (3) clustering the space into stabilized ensembles of frames. Frame Similarity/Dissimilarity Metric. Given the series of frames under analysis, F1..n , we require that either a frame dissimilarity Φ(Fi , Fj ) or simˆ i , Fj ) function be chosen which is able to compare frames in our ilarity Φ(F
image sequence in a pairwise manner, returning a scalar. For simplicity, we will ˆ i , Fj ) ≡ Φ ˆi,j . In most cases, these use the notation Φ(Fi , Fj ) ≡ Φi,j and Φ(F functions may be one of the large number of existing registration metrics, e.g., mean-squares difference, normalized cross-correlation, mutual information, or an ultrasound-specific metric such as that proposed by Cohen and Dinstein [8]. Dissimilarity metrics increase in value with increasingly dissimilar images (e.g., mean-squares difference), whereas similarity metrics are maximal for identical images (e.g., normalized cross-correlation). If a dissimilarity measure is chosen, it must be adapted to the constraints Φi,i = 0 and Φi,j ≥ 0, while simˆi,i = 1 and 0 ≤ Φ ˆi,j ≤ 1. While in practice these ilarity measures must follow Φ contraints may not be followed naturally, metrics which do not conform may be adapted to do so simply by normalizing according to the range of values the metric attains over all frame pairs in the sequence. Creating the Frame-Similarity Space. In this section, we discuss the creation of a Euclidean frame-similarity space in which each frame in our n-frame sequence is represented as a single (though not necessarily unique) point and similar frames are spatially related. For consistency with prior literature on MDS, we adopt the notation of Seber [2] for the majority of this section. Individual entries of a matrix A are denoted ai,j . Vectors are columnar unless otherwise noted. Firstly, if a dissimilarity metric is used, the inter-frame dissimilarities are organized into the following symmetric matrix: Φ1,1 Φ1,2 · · · Φ1,n .. Φ2,1 . . D= (1) . .. .. . Φn,1 · · · Φn,n ˆ but requires ˆ is built similarly from Φ For a similarity metric, the matrix D subsequent conversion into a dissimilarity matrix D before proceeding. This can be q accomplished by letting each di,j = 1 − dˆi,j , or by Seber’s method, di,j = 1 − dˆi,j (recall that all dˆi,j ∈ [0, 1]). Indeed, there are many transˆ → D which may be applied. The important point to consider formations D is that the distances in D are what MDS tries to preserve in the subsequent transformation to the Euclidean space; hence, almost any conversion from similarities to dissimilarities that “makes sense” and conforms to the dissimilarity ˆ is transformed into a dissimilarity matrix D, constraints is suitable. Once D we no longer consider the similarity problem as a separate case. Next, let A be the matrix where ai,j = − 12 d2i,j and let Cn be the n × n centering matrix, Cn = In − n1 1n 1T n , where I is the identity, 1 is a vector of unit entries, and T indicates transpose. We let B represent the double-centered A: B = Cn ACn . We define λ1 ≥ λ2 ≥ . . . ≥ λn and v1 , . . . , vn to be the eigenvalues and associated eigenvectors of B and let¡√ p represent p of positive ¢ √ the number λ1 v1 , λ2 v2 , . . . , λp vp . Each eigenvalues. We now form the matrix Y = row of Y specifies a point in our p-dimensional frame-similarity space (i.e., the
ith row corresponds to the ith frame in the sequence). Similarly to methods used in principal component analysis, we may optionally reduce the dimensions of the space described by Y to fewer than p if needed to make subsequent computations less expensive. Essentially, this consists of removing one or more of the rightmost columns of Y; however, the details of this operation are beyond the scope of this paper (refer to [2] for further discussion). Clustering the Frame-Similarity Space. Given the set of p-dimensional points in the frame similarity space defined by Y, it remains to cluster these points into meaningful ensembles. These ensembles, in a general video-analysis sense, could be said to represent “events,” but in our context they typically represent common orientations of the catheter with respect to the vessel wall. Hence, some represent the stabilized frame sets we seek, eliminating the expected periodic motions of the heart, while outlying points and clusters may indicate the occurrence of unusual events such as the catheter being nudged. Almost any spatial3 clustering algorithm may be employed on the space at this point; common choices include hierarchical clustering and k-means. Note that for clustering purposes it is safe to make the assumption that our space is isotropic; that is, a hypersphere at a particular point in this space will contain an ensemble of frames which are similar according to a threshold determined by its radius. This is more relevant for some algorithms than others; e.g., k-means relies on clusters defined by centroids and would be invalid in an anisotropic space. For the experiments described here, we use randomly-initialized k-means with multiple runs to converge to a lowest-error clustering. In our experiments, a human operator selects k from a visualization of the clusterings associated with several k, the goal being to locate an ensemble which includes a number of frames which is reasonable for a particular analysis (e.g., 50 pre-contrast and 100 post-contrast frames) and excludes obvious outliers. However, many other selection methods with greater or lesser human interaction could be devised; e.g., finding the largest ensemble of frames within a given similarity threshold.
4
Results
For the following experiments, our frame comparison metric is normalized crosscorrelation, which returns values in the interval [−1, +1] with a value of 1 for identical frames. To convert this to a dissimilarity metric, values are clamped to the interval [0, 1] and subtracted from 1. A dissimilarity matrix and frame-similarity space for a 184-frame sequence are illustrated in Figs. 3(a-b). The semi-periodic nature of the motions being measured is apparent in the rippled appearance of the matrix. Clusterings of the space for k = 3 and k = 5 are shown in Figs. 3(d,f). These clusters represent stabilized frame sequences. The clusterings are also illustrated by dissimilarity matrices, Figs. 3(c,e), by reordering the original matrix, Fig. 3(a), such that the frame indices associated with each cluster are adjacent. 3
We note that while spectral clustering may seem an obvious choice when working with similarity matrices, its strength is in clustering connected components; here, on the other hand, we desire proximity-based grouping.
Taking a closer look at the similarity space, Fig. 3(b), we observe the presence of several outliers. The two furthest of these outliers are compared to two automatically-determined “typical” frames (i.e., those nearest the cluster centers) in Fig. 2. The outliers indicate the time at which contrast agent is injected; this event occurs only once during the sequence, following the protocol of Sec. 3.1. All of the outliers were grouped together when a sufficient number of clusters were allotted, Fig. 3(f).
5
Conclusions
We have introduced a novel frame gating method designed for the stationarycatheter sequences employed in IVUS perfusion imaging. Our implementation requires minimal manual guidance, consisting of: 1) selecting a cluster count for k-means, and 2) choosing a particular ensemble of frames from the resulting clusters. However, given application-specific criteria (e.g., minimum desired ensemble size), it would not be difficult to entirely automate this process. While we currently use this method routinely, it remains computationally expensive: the creation of a full (dis)similarity matrix requires quadratic time in the number of frames. We aim to reduce the time complexity to linear using nonmetric MDS [9], which would in principle allow the use of sparse matrices. Finally, we are currently developing an IVUS pullback gating method which, similarly to the current work, requires no ECG signal or frame segmentation. Acknowledgements: We would like to thank S. Carlier, N. Dib, and M. Vavuranakis for providing data for this study, and the other members of the Ultimate IVUS team for valuable assistance. This work was supported in part by NSF Grant IIS-0431144 and a NSF Graduate Research Fellowship (SMO).
References 1. Carlier, S., Kakadiaris, I.A., Dib, N., Vavuranakis, M., O’Malley, S.M., Gul, K., Hartley, C.J., Metcalfe, R.W., Mehran, R., Stefanadis, C., Falk, E., Stone, G., Leon, M., Naghavi, M.: Vasa vasorum imaging: A new window to the clinical detection of vulnerable atherosclerotic plaques. Curr Atheroscler Rep 7 (2005) 164–169 2. Seber, G.A.F.: Multidimensional scaling. In: Multivariate Observations. Wiley (1984) 3. Coskun, A.U., Yeghiazarians, Y., Kinlay, S., Clark, M.E., Ilegbusi, O.J., Wahle, A., Sonka, M., Popma, J.J.: Reproducibility of coronary lumen, plaque, and vessel wall reconstruction and of endothelial shear stress measurements in-vivo in humans. Catheter Cardio Inte 60 (2003) 67–78 4. Bruining, N., von Birgelen, C., de Feyter, P.J., Ligthart, J., Li, W., Serruys, P.W., Roelandt, J.R.T.C.: ECG-gated versus nongated three-dimensional intracoronary ultrasound analysis: Implications for volumetric measurements. Cathet Cardiovasc Diagn 43 (1998) 254–260 5. Nadkarni, S.K., Boughner, D., Fenster, A.: Image-based cardiac gating for threedimensional intravascular ultrasound imaging. Ultrasound Med Biol 31 (2005) 53–63
(a)
(b)
(c)
(d)
Fig. 2. Frames (a,b) are those nearest the cluster centroids for the first two clusters found with k = 3, Fig. 3(c,d). These represent two of the locations occupied by the imaging catheter most frequently over the course of the sequence. Panels (c,d) depict the frames represented by the two outliers nearest the bottom of Fig. 3(b). These were captured at the peak of contrast agent density in the bloodstream, visible as a cloud around the catheter.
6. de Winter, S.A., Hamers, R., Degertekin, M., Tanabe, K., Lemos, P.A., Serruys, P.W., Roelandt, J.R.T.C., Bruining, N.: Retrospective image-based gating of intracoronary ultrasound images for improved quantitative analysis: The Intelligate method. Catheter Cardio Inte 61 (2004) 84–94 7. Zhu, H., Oakeson, K.D., Friedman, M.H.: Retrieval of cardiac phase from IVUS sequences. In: Proc SPIE Medical Imaging: Ultrasonic Imaging and Signal Processing. Volume 5035. (2003) 135–146 8. Cohen, B., Dinstein, I.: New maximum likelihood motion estimation schemes for noisy ultrasound images. Pattern Recogn 35 (2002) 455–463 9. Tsogo, L., Masson, M.H., Bardot, A.: Multidimensional scaling methods for manyobject sets: A review. Multivar Behav Res 35 (2000) 307–319
0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 −0.25 −0.3 −0.35 −0.2
−0.15
−0.1
−0.05
(a)
0
0.05
0.1
0.15
0.2
0.05
0.1
0.15
0.2
0.05
0.1
0.15
0.2
(b) 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 −0.25 −0.3 −0.35 −0.2
Cluster 1 Cluster 2 Cluster 3 −0.15
−0.1
−0.05
(c)
0
(d) 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 −0.25 −0.3 −0.35 −0.2
(e)
Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 −0.15
−0.1
−0.05
0
(f)
Fig. 3. Analysis of a 184-frame sequence: (a,b) original 184 × 184 matrix D and 2D projection of frame-similarity space; (c,d) same, clustered with k = 3; and (e,f) clustered with k = 5. Dark points in the matrices indicate similar frames.
