Automatic Non-Rigid Temporal Alignment of IVUS

Automatic Non-Rigid Temporal Alignment of IVUS Sequences: Method and Quantitative Validation ∗ a,b c c , Simone Balocco , Xavier Carrillo , Josepa Mauri ,

a,b,

Marina Alberti

a,b

Petia Radeva

a Dept.

of Applied Mathematics and Analysis, University of Barcelona, 08007, Barcelona, Spain b Computer Vision Center, 08193 Bellaterra, Barcelona, Spain c University Hospital "Germans Trias i Pujol", 08916 Badalona, Spain.

Abstract Clinical studies on atherosclerosis regression/progression performed by Intravascular Ultrasound (IVUS) analysis would benet from accurate alignment of sequences of the same patient before and after clinical interventions and at follow-up. In this paper, a methodology for automatic alignment of IVUS sequences based on the Dynamic Time Warping (DTW) technique is proposed. The non-rigid alignment is adapted to the specic task by applying it to multidimensional signals describing the morphological content of the vessel. Moreover, DTW is embedded into a framework comprising a strategy to address partial overlapping between acquisitions and a term that regularizes non-physiological temporal compression/expansion of the sequences. Extensive validation is performed on both synthetic and

in-vivo

data. The

proposed method reaches an alignment error of approximately 0.43 mm for pairs of sequences acquired during the same intervention phase and 0.77 mm

Corresponding Author: Marina Alberti, Computer Vision Center, Edicio O, Campus UAB, 08193 Bellaterra (Cerdanyola), Barcelona, Spain; Email, [email protected]; Phone, 0034935811828 ∗

Preprint submitted to Ultrasound in Medicine and Biology

February 26, 2013

for pairs of sequences acquired at successive intervention stages.

Keywords:

Intravascular Ultrasound (IVUS), Dynamic Time Warping,

non-rigid alignment, sequence matching, partial overlapping strategy.

2

1

Introduction

2

Intravascular Ultrasound is a catheter-based imaging technique used for

3

diagnostic purposes and for planning and validation of Percutaneous Coro-

4

nary Intervention (PCI). IVUS sequences are acquired by dragging an ultra-

5

sound emitter carried by a catheter, at constant speed, from the distal to the

6

proximal position of a coronary vessel (pullback). Pullback alignment is re-

7

quired at several stages of the clinical pipeline. First of all, after performing

8

PCI, physicians need to assess the outcome of the intervention (i.e., evaluate

9

nal lumen dimensions and blood ow restoration, inspect stent placement

10

and side-branch occlusion by a deployed stent). Then, at follow-up, pullback

11

alignment is useful to monitor restenosis and the evolution of plaque composi-

12

tion. Currently, in plaque regression studies, the longitudinal correspondence

13

of coronary artery segments is manually determined by identifying common

14

landmarks, such as bifurcations (Nissen et al., 2006; Nakayama et al., 2010;

15

Shin et al., in press; Diletti et al., 2011; Kovarnik et al., 2012). Moreover,

16

a rigid correspondence of the segments adjacent to the landmarks is often

17

assumed (Nissen et al., 2006; Shin et al., in press).

18

Despite the constant speed of the catheter, the automatic alignment of

19

IVUS sequences is hampered by several obstacles.

20

subject to motion artifacts due to the catheter movement and the arterial

21

pulsation, such as the longitudinal swinging of the catheter and the roto-

22

translation of successive frames of the pullback. Moreover, the vessel cycli-

23

cally expands and contracts due to pressure changes during the heart cycle,

24

hence in dierent acquisitions diastole and systole could not correspond to

25

the same physical locations. The rotation of the probe with respect to the

3

IVUS acquisitions are

26

vessel can change and the catheter may follow dierent trajectories with

27

respect to the vessel walls, hence the imaged sections are not necessarily or-

28

thogonal to the vessel walls. The ultrasound beam may be reected by the

29

guidewire and may result in bright echoes and shadows in the IVUS images

30

(Ciompi et al., 2011).

31

orientation in dierent acquisitions. The catheter exibility causes non-rigid

32

deformations of the pullbacks. Moreover, the probe can rst remain stuck in

33

the vessel for some time and then accelerate. Since the pullbacks may have

34

dierent initial and nal spatial positions along the vessel, the overlapping

35

of corresponding vascular segments is frequently partial (see Figure 1(a)). In

36

fact, two corresponding vessel segments may have dierent overall lengths

37

(in terms of number of frames) in dierent acquisitions.

38

sel can undergo signicant morphological changes after the intervention (for

39

instance, stent deployment and post dilation change the lumen and vessel

40

area) or evolve at follow-up. As a consequence, a one-to-one correspondence

41

between frames of the two pullbacks cannot be found, making image-based

42

registration approaches inaccurate.

Guidewire artifacts can vary their appearance and

Finally, the ves-

43

Hence, in this study the IVUS alignment task is addressed as a feature-

44

based temporal alignment problem, in which the morphological content of the

45

artery is exploited.

46

scribed by temporal morphological signals, i.e., side-branch location, vessel,

47

lumen and plaque areas (see Figure 1(c)).

In the proposed approach the IVUS sequences are de-

48

In dierent applications, such as speech recognition, chromatography, ac-

49

tivity recognition and shape matching, several methods have been developed

50

for non-rigid signal alignment, like Dynamic Time Warping (DTW), Canon-

4

51

ical Time Warping (CTW) and Correlation Optimized Warping (COW)

52

(Sakoe and Chiba, 1978; Zhou and de la Torre, 2009; Nielsen et al., 1998).

53

DTW (Sakoe and Chiba, 1978) minimizes the Euclidean distance of corre-

54

sponding points of the signals. CTW (Zhou and de la Torre, 2009) extends

55

DTW by combining it with a Canonical Correlation Analysis step (CCA),

56

which provides a feature weighing mechanism and allows the alignment of sig-

57

nals with dierent dimensionality. The combined use of DTW and CCA can

58

improve the accuracy of the results, but may also yield worse performance

59

than the baseline DTW ( ), and the benets depend on the application.

60

CTW has higher computational complexity than DTW, due to the itera-

61

tive use of CCA required by the optimization process. COW (Nielsen et al.,

62

1998) is a piecewise data alignment method.

63

by dividing them into segments and allowing limited changes in segment

64

lengths. The nal segment lengths are selected so as to optimize the overall

65

correlation between the sequences. The problem is solved as a segment-wise

66

correlation optimization by means of dynamic programming. The solution

67

space is dened by two parameters: the initial segment length

68

maximum segment length increase or decrease,

69

dierent methods have been proposed for matching of symbolic sequences,

70

i.e., strings of characters. Some examples are the techniques for solving the

71

Longest Common Subsequence (LCSS) and the Approximate String Match-

72

ing (ASM) problems, and the Smith-Waterman algorithm (Das, 2001; Sellers,

73

1980; Smith and Waterman, 1981). Given a query and a target strings, LCSS

74

(Das, 2001; Vlachos et al., 2003) determines their longest common subse-

75

quence, i.e., nds subsequences of the query and target that best correspond

?

5

Two sequences are aligned

Slack .

Seg

and the

On the other hand,

76

to each other. The distance between the two sequences is computed based on

77

the ratio between the length of their longest common subsequence and the

78

length of the whole sequence. The aim of ASM methods is to identify the

79

subsequence of a text most similar to a given pattern, i.e., whose Levenstein

80

Distance to the pattern is minimal (Sellers, 1980; Navarro, 2001).

81

stein Distance measures the dierence between two strings as the minimum

82

number of character insertions, deletions and substitutions needed to make

83

them equal.

84

for identication of common molecular subsequences (Smith and Waterman,

85

1981), is a local alignment algorithm that matches two sequences by using

86

dynamic programming. Smith-Waterman nds similar subsequences without

87

exhaustive search.

Leven-

Finally, the Smith-Waterman algorithm, originally proposed

88

Although the automatic alignment of IVUS sequences is useful for clinical

89

research studies which compare the vessel over time, it is interesting to note

90

that it was only rst addressed in a recent study presented by our group

91

in (Alberti et al., 2012a), where a feature-based non-rigid temporal align-

92

ment is proposed. In order to address the non-rigid correspondence between

93

frames, a DTW-based framework is adapted to the specic clinical task. The

94

DTW alignment technique is applied to multidimensional morphological sig-

95

nals which describe the morphological content of the sequence. To tackle the

96

partial overlapping problem, the DTW algorithm is integrated into a Sliding

97

Window approach. Moreover, a regularization term is introduced to penalize

98

signicant dierences in the global temporal expansion/compression of IVUS

99

sequences.

100

The Sliding Window approach has two main limitations.

6

First, corre-

101

sponding IVUS subsequences must have the same length in terms of number

102

of frames.

103

quences in order to identify the optimal corresponding subsequences, result-

104

ing in high computational cost.

Second, DTW is iteratively applied to dierent pairs of subse-

105

In the present paper, the approach in (Alberti et al., 2012a) is improved

106

and completed by proposing a novel solution for handling partial overlap-

107

ping.

108

constraint forcing the selected matching segments to be of the same length,

109

and it reduces the computational complexity with respect to the Sliding

110

Window strategy.

111

approaches, taking advantage of the most suitable characteristics of both

112

techniques while overcoming their limitations. In fact, on one hand, DTW

113

uses the Euclidean distance as a dissimilarity measure, which is adequate

114

to continuous sequences, while ASM uses the Levenstein distance, originally

115

proposed for strings.

116

meric sequences, since a threshold is required to determine when two numeric

117

values are equal, and performance will heavily depend on the threshold set-

118

ting.

119

matching between the whole sequences, while ASM can handle partial over-

120

lapping of sequences through an ad-hoc initialization strategy. EPS exploits

121

an initialization inspired by ASM to tackle partial overlapping, but using the

122

Euclidean distance as a dissimilarity measure, like DTW. In the EPS strat-

123

egy, rstly the extremes of the corresponding subsequences are identied on

124

the two sequences. Then, the selected matching subsequences are aligned by

125

means of DTW.

The Extremes of Path Search (EPS) strategy overcomes the rigid

The proposed solution combines the DTW and ASM

Levenstein distance is not directly applicable to nu-

On the other hand, DTW has the constraint of computing a global

7

126

An extensive validation of our IVUS alignment framework is performed,

in-vivo

127

both on synthetic data and on two

128

consisting of 42 total IVUS pullbacks acquired from 21 dierent patients.

129

data-sets of dierent complexity,

The main contributions of our study are the following:

a thoroughly

130

automatic workow for IVUS sequence alignment is presented, which is po-

131

tentially applicable to other image modalities. To this aim, a DTW-based

132

approach is specically tailored for the clinical task. Moreover, a novel strat-

133

egy for multidimensional sequence alignment, robust to partial overlapping,

134

is proposed, which can be used in a wider range of alignment problems.

135

An exhaustive

136

sets, which have been created in order to achieve a reliable validation of our

137

method and to reect its clinical application, respectively. Finally, a novel

138

description of IVUS sequences is provided in terms of morphological signals.

139

Method for IVUS Sequences Alignment

140

Multidimensional Proles Framework

in-vivo

validation is performed on two dierent IVUS data-

141

In this paper, a pair of corresponding IVUS sequences is described by

142

temporal morphological proles (i.e., signals describing the evolution of mor-

143

phological measurements along the vessel) and dened as a pair of time series

144

X ∈ Rm×nx

145

morphological proles used in this study are listed in the following para-

146

graphs.

147

Gating Preprocessing

and

Y ∈ Rm×ny ,

of length

nx , ny

and dimensionality

m.

The

148

The heart beating generates undesired artifacts in IVUS acquisitions, dis-

149

turbing the computation of the morphological measurements. A rst artifact

8

150

consists of repetitive oscillations of the catheter (swinging eect) along the

151

axis of the vessel, resulting in possible multiple sampling of the same ves-

152

sel positions (Gatta et al., 2010). A second problem is represented by the

153

vessel pulsation: due to pressure changes during the heart cycle, the vessel

154

cyclically expands and contracts. As a consequence, the appearance of the

155

cross-sectional images can signicantly change depending on the heart cycle

156

phase.

157

In order to obtain a unique reconstruction for the transversal sections

158

of the artery, and to limit the morphological variations due to vessel pul-

159

sation, one possible solution is the selection of the frames belonging to the

160

same phase of the cardiac cycle. The compensation of both artifacts can be

161

addressed by using a gating technique, either by exploiting the electrocar-

162

diogram (ECG) signal, if available (Bruining et al., 1996), or by image-based

163

gating algorithms (O'Malley et al., 2008; Gatta et al., 2010), which have

164

the advantage of being applicable also in case of arrhythmia. In this study,

165

an image-based gating technique is applied (Gatta et al., 2010) to select

166

the frames belonging to the end-diastolic phase. The gated images provide

167

coherent morphological measures, since in end-diastole the arterial tissues

168

are subject to the same blood pressure. Moreover, the gating preprocessing

169

compensates for the longitudinal oscillations of the catheter caused by heart

170

beating and ensures that the frames are consecutive in the direction of the

171

catheter movement.

172

Prole extraction

173

In this study, the following morphological measurements are proposed:

174

(1) vessel area, dened as the area inside the media-adventitia border, (2)

9

175

lumen area, (3) area of calcied plaque, (4) area of bro-lipidic plaque, (5)

176

angular extension of vascular bifurcations. Such signals can be manually or

177

automatically extracted. In this paper, the proles are automatically com-

178

puted using state-of-the-art algorithms, as follows: the vessel area is com-

179

puted by means of the method proposed in (Ciompi et al., 2012), while the

180

lumen area is computed by exploiting the method proposed in (Balocco et al.,

181

2011). In the computation of the tissue areas, three classes of plaque can be

182

discriminated using (Ciompi et al., 2010): calcied, brotic and lipidic. The

183

necrotic core has not been added to this tissue characterization, consider-

184

ing that there is no agreement on the denition of necrotic core and on the

185

reliability of its assessment in IVUS (Thim et al., 2010; Nair et al., 2002,

186

2007; Sathyanarayana et al., 2009; Stone et al., 2011; Wykrzykowska et al.,

187

2012) and that the morphological signals considered in this study are pro-

188

viding sucient information for the addressed problem. Additionally, since

189

in the analyzed data-sets the lipidic samples are few, inhomogeneous and

190

scattered into the brotic area (no lipidic pools are present), the proles of

191

lipidic plaque would not be reliable and would contain outliers. Therefore,

192

the brotic and lipidic areas are combined into a single region, to obtain a

193

consistent representation of the vessel morphology. Finally, a method for the

194

automatic detection of the position and the angular extension of vascular

195

branching in IVUS is applied (Alberti et al., 2012b).

196

The chosen morphological proles are invariant to frame rotation, thus

197

making the method independent of the catheter torsion. The use of multiple

198

features is aimed to increase the robustness with respect to 1-D alignment, by

199

capturing dierent aspects of the vessel morphology, in particular increasing

10

200

the robustness to modications due to surgical intervention.

201

An accurate prole extraction is important for the proposed IVUS align-

202

ment method. It might be noticed that the framework is independent of the

203

technique employed for the measurements. Other methods could be used to

204

extract the morphological proles, and the framework could potentially be

205

extended by using a dierent set of morphological proles.

206

alignment method can rely on generic segmentation algorithms for signal ex-

207

traction, for the sake of completeness the performance and computational

208

time of the algorithms used in the experiments are reported in Tables 1 and

209

2.

210

IVUS Alignment Framework

211

The DTW algorithm

212

Although the

The proposed signal alignment framework is based on the DTW tech-

X = [x1 , x2 , . . . xnx ]

213

nique. To align two sequences

214

DTW builds a matrix

215

sure between

216

formulation,

217

multidimensional alignment the distance

d(nx ×ny ) ,

where

d(i, j)

and

represents a dissimilarity mea-

X(i) and Y (j) (Sakoe and Chiba, 1978). d(i, j)

[ ] Y = y1 , y2 , . . . yny

In the classical DTW

is computed as the Euclidean distance. In the case of a

d(i, j)

is:

v u m u ∑ ( )2 d(i, j) = t xidim − yjidim , i

(1)

idim=1 218

where

idim is one of the dimensions of X

and

Y.

The dierent dimensions

219

of the sequences are normalized independently of each other. Successively,

220

the Minimum Cumulative Distance (MCD) matrix

221

namic programming as follows:

11

D

is computed by dy-

D (i, j) = d (i, j) + min(D (i − 1, j) ,

(2)

D (i − 1, j − 1) , D (i, j − 1)) . The rst row and the rst column of the MCD matrix

222 223

D

are initialized

with cumulative values as follows:

   D(i, 1) = D(i − 1, 1) + d(i, 1)

(3)

  D(1, j) = D(1, j − 1) + d(1, j) , where

224

i ∈ {1, 2, . . . , nx }

225

matrix represents the

226

between

X

and

and

j ∈ {1, 2, . . . , ny }.

matching cost,

The last element of the

i.e., the minimal cumulative distance

Y:

Φ (X, Y ) = D(nx , ny ). Finally, the algorithm nds the

227

X

of

229

computing a

230

top-left cell

231

cells), as illustrated in Figure 2.

232

Regularization Cost (RC)

233

and

Y

228

warping path

on a common time axis,

backtracking

(1, 1)

of

D

(4)

(a mapping of the time axes

wp = ⟨[i(k), j(k)] |k = 1, . . . , K⟩)

(a path from the bottom-right cell

(nx , ny )

by

to the

by following the minimum values of the neighboring

In this study, a regularization term is introduced in the DTW alignment

band constraint

234

framework. Such regularization strategy is inspired by the

235

used in the classical version of DTW (Sakoe and Chiba, 1978), where the

236

warping path

237

the diagonal of the dissimilarity matrix.

238

ization is applied to non-diagonal transitions in the

is guided, by limiting its acceptable domain to a band around

12

Similarly, in this study, a penal-

warping path

(one-to-

239

many correspondences among frames) (Holmes, 1988; Ramaker et al., 2003),

240

to avoid an excessive presence of horizontal and vertical transitions, which

241

would represent non-physiological temporal compression/expansion of the

242

two IVUS sequences.

243

path

increases.

With respect to (Sakoe and Chiba, 1978), which sets the

244

band constraint

to a constant width, thus xing a threshold, in the proposed

245

solution the regularization is directly integrated into the dynamic program-

246

ming computation: non-diagonal transitions of the path are uniformly given

247

a higher cost (

248

MCD leading to the entry

As a result, the smoothness of the output

regularization cost, (i, j)

RC) in computing the MCD matrix. The

is calculated as:

D(i, j) = d(i, j) + min{(D(i − 1, j) + C, D(i − 1, j − 1), D(i, j − 1) + C} , 249

where the parameter

C

warping

(5)

represents the direction penalty. The value of

N

C

250

is tuned by cross-folding. The data-set is divided into

subsets (denoted

251

as folds), and for each fold the value of the parameter is optimized over the

252

other (N -1) folds.

253

fold.

254

when the proles are aected by noise corruption.

255

Partial Overlapping Strategy

Then, the optimized value is applied to the considered

The proposed regularization is aimed to reduce the alignment error

256

The typical limitation of the DTW approach is the computation of a

257

global matching between the whole sequences, forcing the extremes to corre-

258

spond in the

259

1978). A potential problem arises when the two sequences partially overlap,

260

i.e., when a sequence matches only to a subsequence of the other sequence.

warping path (boundary condition constraint) (Sakoe and Chiba,

13

261

This is the typical condition for IVUS pullbacks, due to the possible varia-

262

tion in the starting and nal positions of the probe along the vessel during

263

acquisition (see Figure 1). Two strategies are possible to tackle this issue:

264

Sliding Window (SW) Approach.

265

in (Alberti et al., 2012a), with the goal of increasing robustness to partial

266

overlapping. The solution consists in the integration of the DTW alignment

267

algorithm into a Sliding Window (SW) approach. The two sequences

268

Y

269

the alignment between the overlapping subsequences is identied by means

270

of DTW. The optimal sliding iteration is selected by minimizing a

271

cost :

A rst strategy has been recently proposed

X

and

are iteratively slided one along the other (see Figure 3) and for each step

ΦN ORM (Xiter , Yiter ) = Φ (Xiter , Yiter ) /liter , Xiter

Yiter

(6)

are the overlapping subsequences and liter is the over-

272

where

273

lapped length, i.e., the length of the overlapping window at the sliding it-

274

eration

275

strained to have the same length on

276

computational cost, the number of iterations

277

the subsequence of the shortest sequence which remains outside the window

278

to a maximum length,

279

Extremes of Path Search (EPS).

280

tial overlapping is proposed. Extremes of Path Search (EPS) integrates an

281

initialization inspired by the ASM techniques (Sellers, 1980; Navarro, 2001).

282

The ASM problem has been originally posed for discrete string matching and

iter.

and

matching

It can be observed that the overlapping subsequences are con-

X

and

Y.

In order to decrease the

Niter

is restricted by limiting

welast . In this paper, a new strategy to handle par-

14

283

consists in identifying the subsequence of a text which is most similar to a

284

given pattern string (as well as the starting position and the extension of the

285

subsequence). In the dynamic programming solution to ASM (Sellers, 1980),

286

the rst row of the dynamic programming matrix (Equation 3), correspond-

287

ing to the text, is initialized with zeros so that the pattern can start with

288

zero error at any position in the text. Following the same idea, in the proposed EPS technique, the rst row

289 290

and column of the MCD matrix

D

(Equation 3) are initialized as follows:

   D(i, 0) = 0, i ∈ {1, 2, . . . , nx }   D(0, j) = 0,

j ∈ {1, 2, . . . , ny } .

In the classical DTW approach, the

291

warping path )

(7)

end of match

(i.e., the nal point of

corresponds to the last element of the

D

the

293

which represents the

294

the two matching sequences are partially overlapped, we allow the

295

match

296

of the matrix

297

diagonal distance

matching cost Φ(X, Y ) (Equation 4).

matrix

(nx , ny ),

292

Since in our case

end of

to be selected as the minimum value between last row and last column

DN ORM , L

obtained by normalizing the MCD matrix

D

by the

(see Figure 4(b)):

ΦEP S (X, Y ) = argmin(DN ORM (i, ny ), i,j

(8)

DN ORM (nx , j)), 298

where

i ∈ {1, 2, . . . , nx }

and

j ∈ {1, 2, . . . , ny }.

In the general case of

299

IVUS images, one of the pullbacks is not completely contained in the other,

300

but there is a mutual overlap of two sequences.

301

lowed to match to only a subsequence of the other, and there is no dis-

15

Both sequences are al-

302

tinction between the roles of the two sequences, such as the text and pat-

303

tern roles in ASM. Consequently, the search of the

304

be repeated twice, rst assessing the nal frames

305

between

X

and

Y,

then inverting the signals (X

(

end of match

xf x , y f y

)

has to

of the match

′

= [xnx , xnx −1 , . . . , x1 ], ( ) initial frames xix , yiy of

306

[ ] Y ′ = yny , yny −1 , . . . , y1 )

307

the match, as illustrated in the block diagram in Figure 4(a). Finally, the

308

warping path

309

quences

310

by applying the DTW algorithm, as shown in Figure 4(c), between the initial

311

(

xix , yiy

and searching for the

(non-linear alignment) between the selected matching subse-

[ ] Xsub = [xix , xix +1 , . . . , xfx ] and Ysub = yiy , yiy +1 , . . . , yfy is obtained

)

(

and the nal element

xfx , yfy

)

of the match.

312

EPS results in a more compact strategy for handling the partial over-

313

lapping problem with respect to the SW approach, because it is directly

314

embedded into the DTW technique.

315

frames of a correspondence need to be computed only once.

316

that the computational complexity of the DTW algorithm is

317

computational complexity of EPS is

318

it is

319

X

320

Experimental Results

321

Materials

O ((nx + ny )nx ny ),

and

Y

where

nx

Additionally, the initial and ending

O (3nx ny ),

and

ny

Hence, given

O (nx ny ),

the

while for the SW approach

are the number of gated frames of

(around 100 images).

in-vivo

322

A set of IVUS sequences consisting of 42

323

coronary arteries has been used in this study.

324

acquired from 21 patients in the Hospital  Germans Trias i Pujol , Badalona

325

(Spain) by means of iLab IVUS Imaging System (Boston Scientic, Natick,

16

pullbacks from human

The sequences have been

326

MA, US). Sequences have been recorded with constant pullback (0.5 mm/sec)

327

using a catheter with 40 MHz central frequency Atlantis SR 40 Pro (Boston

328

Scientic, Natick, MA, US), at a sampling rate of 30 frames/sec. Only gated

329

frames belonging to the same phase of the cardiac cycle have been selected

330

at preprocessing, using the method proposed in (Gatta et al., 2010).

331

the acquisitions have been performed strictly following the clinical protocol

332

approved by the hospital ethical committee and informed consent for the

333

study has been obtained from all patients.

All

334

The clinical data have been randomly chosen without any exclusion crite-

335

ria from the hospital database. The study population is composed of patients

336

ranging in age from 32 to 82 (median 70); there are 3 diabetic patients. In

337

particular, 37 of the 42 sequences contain a stent, resulting in 20 of the 21

338

pullback pairs containing stent. In some patients stent is present from pre-

339

vious interventions, while in others it has been deployed during PCI. The 42

340

analyzed sequences constitute two data-sets aimed at dierent purposes:

341

ˆ Data-set A is specically used for the validation of our method only, and

342

it consists of 8 pairs of corresponding IVUS pullbacks (16 sequences)

343

acquired at the same stage of the percutaneous intervention (i.e. an-

344

gioplasty, stent deployment, and/or stent post-dilatation), either before

345

or afterward. Since there are no morphological changes due to inter-

346

vention, a high number of manually annotated ground-truth landmarks

347

could be dened. To this aim, the presence of morphological structures,

348

such as small calcications and external vessels, and the initial and end

349

positions of deployed stent have been considered.

350

ˆ Data-set B

reects the clinical application of our study, and it contains

17

351

13 pairs of corresponding pullbacks (26 sequences), all characterized

352

by signicant morphological changes due to percutaneous intervention.

353

Following the same validation strategy employed in (Shin et al., in

354

press; Diletti et al., 2011), only bifurcation locations (initial and end

355

positions) are used as ground-truth. Indeed side branches are the only

356

immutable landmarks, since lumen and media size, stent and plaque

357

might vary due to surgical artery dilatation or stent deployment.

358

Manual annotations have been performed by an expert clinician. Finally, the

359

in-vivo

360

ing 12.2 landmarks per sequence) and 60 side-branch locations in Data-set B

361

(4.6 landmarks per sequence).

362

Methodological Comparison

ground-truth consists of a total 98 landmarks in Data-set A (averag-

363

The performance of the proposed approach is compared to two other

364

state-of-the-art techniques, CTW (Zhou and de la Torre, 2009) and COW

365

(Nielsen et al., 1998) and to the algorithm proposed in (Alberti et al., 2012a).

366

To ensure a fair comparison, all the alignment algorithms (DTW, CTW, and

367

COW) will benet from the robustness improvements proposed in (Alberti

368

et al., 2012a): (1) adapting the original framework (Sakoe and Chiba, 1978;

369

Zhou and de la Torre, 2009; Nielsen et al., 1998) to multidimensional signals,

370

using the same weight for the dierent features (2) integrating the alignment

371

algorithms into the partial overlapping strategy and (3) applying the path

372

regularization term RC. Regarding (2), the EPS approach has been speci-

373

cally designed for the DTW technique, hence the other alignment algorithms

374

will employ the SW strategy. A list of all possible combinations is reported

375

in the rst column of Table 3.

18

376

In order to validate the method, the performance of the automatic align-

alignment error E , dened as the distance

377

ment is evaluated in terms of the

378

between the ground-truth reference and the output

379

puted as the average error for all the ground-truth points and is expressed in

380

number of gated frames. The evaluation is performed using both synthetic

381

data with applied controlled distortion and

382

Experiments on Synthetic Data

383

Synthetic Morphological Signals

384

Pairs of sequences

(X, Y )

in-vivo

in-vivo

morphological proles extracted from

386

ure 5 summarizes the applied types of distortion:

data.

pullbacks. The scheme in Fig-

1. Amplitude distortion: additive zero-mean random noise is applied to

w1

388

the morphological proles. The noise amplitude

389

percentage of the mean value of the signal (Figure 5(a)).

390 391 392

is computed as a

2. Partial overlapping: a portion of the original sequence is selected, whose length is a percentage

w2

of the initial prole (Figure 5(b)).

3. Temporal distortion: a temporal expansion/compression generates ver-

393

tical or horizontal transitions in the

394

spondences between the frames of

395

in which we randomly introduce:

396

is com-

are synthetically generated by modifying the

385

387

warping path. E

X

warping path, and

Y.

i.e., multiple corre-

We distinguish three cases,

(a) the same number (w3 ) of multiple correspondences from

X

to

Y

397

and vice-versa (Figure 5(c)). A randomly generated time transfor-

398

mation matrix for time warping is used,

399

Torre, 2009).

M

is initialized as

19

M,

M = In ,

as in (Zhou and de la where

n

is the length

400

of the matching portions of the signals. Then,

401

are randomly chosen and replicated and

402

chosen and deleted.

w3

w3

columns of

M

columns are randomly

403

(b)

w4 additional multiple correspondences from X to Y

(Figure 5(d)).

404

(c)

w5 additional multiple correspondences from Y

(Figure 5(e)).

405

The parameters

406

Their default values and ranges, which represent average

407

and realistic variations, respectively, are suggested by a medical expert and

408

empirically measured on the whole ground-truth:

409

(75 ± 25)%, w3 = 60 (0 − 120) frames, w4 = 5 (0 − 20) frames, w5 = 0 (0 − 20)

410

frames.

w1 , w2 , w3 , w4 , w5

to

X

model the signal distortion simulation.

in-vivo

w1 = (100 ± 100)%, w2 =

The tuning of the parameters of the alignment methods,

411

welast

413

40

414

parameters are estimated by exhaustive search:

415

[6, Seg − 9], welast ∈ [0, 35]

416

DTW and CTW are fully automatic, while COW requires the setting of the

417

initial segment length

418

is performed by minimizing the mean value of

E

Seg , Slack ,

412

and

C,

conditions

over

Nexp =

experiments, setting the distortion parameters to default values.

and

C ∈ [0, 0.1].

The

Seg ∈ [16, 30], Slack ∈

It is worth noticing that both

Seg and the maximum segment length variation Slack .

A rst synthetic experiment focuses on assessing the robustness of the

419

framework to variations in the number of morphological features.

420

ond experiment evaluates the robustness to each of the previously described

421

simulated distortions.

422

Multidimensional Alignment

423 424

A sec-

In order to evaluate the robustness of the framework as a function of the number of morphological proles,

E

is computed by varying the number of

20

m in the range [1, 5].

425

acquired signals

426

by setting

427

all the possible feature combinations are tested and then the error is com-

428

puted as the average by repeating the test

429

Table 3 shows that

430

particularly signicant when more than one signal are considered, conrm-

431

ing the interest of a multidimensional extension of the method. Similarly, all

432

the methods improve their robustness when combined with RC and partial

433

overlapping strategies (SW or EPS). As observed by comparing the last two

434

columns of Table 3, when the number of features is high (more than four)

435

the

436

formances and they are both superior with respect to

437

COW-SW-RC.

438

RC

439

Robustness to Signal Noise and Distortion

440

w1 , w2 , w3 , w4 , w5

DTW-SW-RC

E

Pairs of synthetic signals are generated

to default values. For each value of

Nexp = 40

decreases at the increase of

and the

DTW-EPS-RC

m.

m ∈ [1, 5],

times. As expected,

The error reduction is

approaches have comparable per-

CTW-SW-RC

On the other hand, for a low number of features

and

DTW-SW-

is the most robust approach.

In the second set of experiments, the robustness of the framework to noise

E

441

and distortion is assessed. Figure 6 shows

442

parameters

443

chosen range, the others are set to default values. In general, COW shows

444

the highest error, indicating that a segment-wise alignment is the least suited

445

for the IVUS pullback alignment. The performance of CTW is comparable to

446

DTW, but the latter is computationally less expensive, since CTW requires

447

the iterative use of CCA. It is worth noticing that, in this study, the two

448

sequences have the same dimension (i.e., the same number of morphological

449

proles), therefore the advantage given by CTW of aligning signals with

w1...5 .

as a function of the distortion

When one of the distortion parameters is varied in the

21

450

dierent dimensionality is not relevant in this application.

451

experiment demonstrates that the partial overlapping strategy is eective,

452

since CTW and COW are robust to partial overlapping only when integrated

453

in the SW framework. Similarly, DTW improves its robustness only when

454

combined with the SW or EPS strategies (see Figure 6(b)). The synthetic

455

experiments show that both versions of DTW (embedded into the SW and

456

EPS strategies) are the most performant among state-of-the-art algorithms.

457

Regarding the partial overlapping solution, as observed in Figure 6(d) and

458

(e), EPS is advantageous over SW in case of temporal distortion of dierent

459

intensity in the two pullbacks, since it is extremely robust to variations of

460

and

461

additive noise

462

slightly superior (Figure 6(a)).

463

Experiments on In-Vivo Data

464

w5 .

The

Moreover, this

w4

The performances of SW and EPS are similar with respect to the

w1 ,

in-vivo

and only for a large amount of noise the SW strategy is

validation is performed on Data-set A, used to reliably vali-

465

date our method, and on Data-set B, created to reect the clinical applica-

466

tion. The parameter tuning is performed by means of

467

(LOPO) cross-validation technique in each data-set, over

468

ing to dierent patients (pullback pairs). LOPO can be considered as a spe-

469

cial case of

470

one patient. Further details about LOPO technique can be found in (Ciompi

471

et al., 2011). For each fold, one of the pullback pairs is iteratively used as

472

test set and the parameters are optimized by minimizing

473

pairs.

N -fold

Leave-One-Patient-Out N

folds correspond-

cross-validation, where each fold contains the data from

22

E

over all the other

474

Quantitative Results Table 4 reports the

475

in-vivo

results of the compared approaches. Similarly

476

to what observed in the synthetic experiments, DTW can be selected among

477

state-of-the-art alignment techniques for reasons of superior performance and

478

lower computational cost. The statistical signicance of the results is eval-

479

uated according to the Wilcoxon side-ranks test (Demsar, 2006). At a sig-

480

nicance level

481

distributions are equal can be rejected if

z≤1.96,

482

Wilcoxon statistics.

DTW-EPS-RC

483

approaches are comparable in both data-sets.

484

RC

485

variability, assessed by using side-branch locations labeled by a second physi-

486

cian, in Data-set B. The direct comparison between the two approaches can

487

be appreciated in Figure 7, where the

488

of

489

tively, are superimposed to ground-truth annotations (the crosses).

490

be noticed that errors of the

491

extremes of the

492

left corner of the image), while in the

493

path

494

corner). The incorrect solution is due to the rigid constraint of the SW tech-

495

nique, which forces the matching windows selected on each pullback to be

496

of the same length, in terms of number of frames.

497

putational cost of

498

Indeed in

and

α = 0.05,

The results of the

DTW-SW-RC

DTW-SW-RC

the null hypothesis that the mean values of the two

z

where

is the value of the

and

DTW-SW-RC

Moreover, both

DTW-EPS-

reach performances comparable to the inter-observer

DTW-EPS-RC

(Figure 7(a)) and

warping path

warping paths,

DTW-SW-RC

computed by means

(Figure 7(b)), respecIt can

alignment are located at the

(where the path becomes vertical in the top-

DTW-EPS-RC

alignment the

warping

is smoother and closer to the manual annotation (cross in the top-left

DTW-EPS-RC

DTW-SW-RC,

Additionally, the com-

is lower with respect to

DTW-SW-RC.

the DTW algorithm is iteratively applied

23

N iter

499 500

times, whereas in

DTW-EPS-RC, only three times.

The quantitative results illustrated in these Sections (both on synthetic

in-vivo

501

and

data) show that the algorithms based on the DTW approach

502

are superior to other state-of-the-art methods. Although comparable perfor-

503

mances are obtained by averaging the results of

504

EPS -RC,

505

strategy because of its robustness at the boundary of the matching (Figure

506

7) and for the lower computational cost, hence the most appropriate solution

507

for the IVUS alignment task.

DTW -SW -RC

and

DTW -

it can be stated that EPS is the best suited partially overlapping

508

The performance of the chosen method can be further appreciated in

509

Figure 8, illustrating the results of the alignment on several pullback pairs

510

from Data-set A (rst row) and B (second row). The mean value for

511

0.85

512

can be estimated as approximately

513

Qualitative Results

514

gated frames in Data-set A and

1.53

0.43

E

is

gated frames in Data-set B, which

and

0.77

mm, respectively.

The images in Figure 9 illustrate several examples of frame-to-frame cor-

DTW-EPS-RC.

515

respondences identied by

516

the interest of the proposed

517

results are compared to frames identied by considering a

518

dence between pullbacks. In Data-set A, the rigid matching between frames

519

is simulated by estimating a

520

ground-truth landmarks coordinates. Since in Data-set B the landmarks are

521

limited to few side-branch positions, the linear tting is performed on the

522

non-linear warping path.

523

non-linear

In order to qualitatively assess

alignment method, the automatic

linear warping path

linear

correspon-

as linear regression of the

Figure 9 qualitatively compares the matching results on both Data-sets

24

524

A and B. The rst column of Figure 9 reports, for each pair of pullbacks (P1

525

and P2), a warping matrix in which the ground-truth, the

526

path

527

indicated by a green horizontal line. The selected frame of P1 (second col-

528

umn), is compared versus the frames of P2 which are identied by

529

(third column) and by

530

can be qualitatively observed in Figure 9, the

531

rectly identies corresponding frames in both data-sets. A higher similarity

532

between the columns two and three can be visually assessed. In particular,

533

the presence of the same calcications (rows 3 and 5), bifurcations (rows 1, 6

534

and 7), external vessels (row 4), and a similar shape of the vessel structures

535

(rows 2 and 8) can be recognized.

536

Discussion

and the

linear

alignment are depicted.

linear

non-linear warping

The analyzed frame on P1 is

non-linear

(fourth column) alignment, respectively. As it

DTW -EPS -RC

technique cor-

537

It must be noticed that in some challenging cases in Data-set B, the

538

morphological changes induced by stent deployment make it impossible to

539

visually compare the corresponding frames extracted from the pullbacks P1

540

(pre-operative) and P2 (after stent deployment), even for an expert physician.

541

For instance, such ambiguity can be observed in the frames of Figure 10,

542

where the changes in vessel appearance caused by stent placement prevent

543

from estimating which of the two frames of P2 (third and fourth columns)

544

is the most similar to the corresponding frame of P1 (second column).

545

we observe the position along the pullback of the two analyzed frames of P1

546

(rst column), we can notice that the rst frame lies close to the manual

547

landmarks (rst row), while the second frame is located in a segment where

25

If

548

the landmarks are less dense (second row). In the rst case, it is reasonable

549

to believe that the

550

is extremely dicult to assess if the

551

algorithm is real or if it is an artifact produced by the algorithm. This issue

552

cannot be directly addressed, because of the lack of visual or morphological

553

landmarks. However, it can be noticed that the non-straight path obtained

554

between frames 45-80 and 80-100 of P1, approximately, corresponds to the

555

portion of the vessel where the stent has been implanted (red region in Figure

556

10(rst column)), hence it is presumable that the

557

actually induced by stent deployment.

non-linear

warping is correct, while in the second case it

non-linear

Moreover, in order to assess if the

558

deformation computed by the

non-linearity

non-linear

behavior is

in Data-set B is excessive

non-linearity in both Data-sets A

559

or lies in an acceptable range, the amount of

560

and B (N LA and

561

amount of

non-linearity (N LA ) is estimated as the average distance between

562

the

linear

tting and the ground truth landmarks. Since in Data-set B the

563

number of manual annotations is too low,

564

distance between the

565

amount of

566

N LB = 2.4 ± 1

567

of the

568

measurement

569

a higher impact on the catheter path.

570

that

571

necessary for pullback alignment

N LB , respectively) can be assessed.

non-linearity

tting and the

is estimated as

is computed as the average

DTW-EPS-RC warping path.

N LA = 1.79 ± 1

N LA ,

The

gated frames and

gated frames. As expected, in Data-set B the

warping path

N LA

linear

N LB

To do so, the reference

non-linearity

is slightly (but not excessively) higher than the reference since the vessel dilations and the stent deployment have However, it is interesting to note

is non-negligible, demonstrating that a

.

26

non-linear

alignment is

572

Conclusion

573

In this paper, we presented a fully automatic framework for the temporal

574

alignment of IVUS acquisitions of the same vessel, before and after percu-

575

taneous intervention. This goal is reached by identifying a continuous non-

576

rigid frame-to-frame correspondence between the two pullbacks. The IVUS

577

sequences are described by means of multiple temporal proles representing

578

their morphological content. The DTW technique for non-rigid alignment is

579

embedded into a multidimensional framework, specically developed for ad-

580

dressing the challenges of IVUS alignment. The proposed solution includes

581

a robust strategy to handle the partial overlapping problem and a regular-

582

ization term to avoid a non-physiological temporal compression/expansion

583

of the two sequences and to compensate for possible noise in the acquired

584

signals. With respect to (Alberti et al., 2012a), a novel strategy for partial

585

overlapping is presented.

586

An exhaustive validation is performed, both on synthetic data and on two

587

in-vivo

588

vessel without any morphological change, while the other contains pre-post

589

intervention cases. Our approach reaches an average alignment error of ap-

590

proximately

591

and

592

solutions with respect to the baseline DTW alignment. Qualitative

593

results illustrate the interest of the proposed

594

the clinical value of the method. The presented framework is robust to mor-

595

phological changes induced by stent deployment and post dilation and is

596

invariant to rotations of the probe and to the catheter or imaging system

data-sets, one of which consists of multiple acquisitions of the same

in-vivo

0.43

and

0.77

mm on the two data-sets, respectively. Synthetic

results show the robustness increase obtained by the proposed

27

non-linear

in-vivo

alignment and show

597

employed. Moreover, given the extracted morphological proles, the average

598

computational time is less than 0.2 seconds per sequence pair, making the

599

application suitable for intra-operative procedures.

600

implemented in MATLAB and it has been executed on a PC equipped with

601

an Intel Core 2 Duo 2.13 GHz processor and 4 GB RAM. It is worth noticing

602

that, when fully automatic segmentation methods are employed, the morpho-

603

logical signals can be extracted in a rst moment (in an oine segmentation),

604

for instance after acquiring an IVUS sequence, while the proposed method

605

for sequence alignment can be run in almost real-time, for instance when

606

comparing two IVUS pullbacks.

The method has been

607

Although satisfactory performances have already been obtained, the pro-

608

posed framework can be extended in future studies, investigating the use of

609

dierent weights for the morphological features.

610

image-based measurements (for instance, entropy) could also complete the

611

set of proles.

612

studies on atherosclerotic plaque regression/progression in order to automat-

613

ically align cases acquired at dierent times and can provide a robust tool for

614

plaque evolution follow-up. Moreover, future work will be addressed towards

615

intra-modality alignments, for instance between IVUS and angiographic or

616

Optical Coherence Tomography data.

617

Acknowledgements

It is worth noticing that

The developed framework can be applied in large clinical

618

This work has been supported in part by the projects TIN2009-14404-

619

C02, Lumen Border Detection (Research activities based on pre-determined,

620

prioritized list of IVUS-Related research topics of interest of Boston Sci-

28

621

entic), La Marató de TV3 082131, CONSOLIDER INGENIO CSD 2007-

622

00018, AIB2010SE-00210 and SGR00696.

29

623

References

624

Alberti M, Balocco S, Carrillo X, Mauri J, Radeva P. Automatic non-rigid

625

temporal alignment of ivus sequences. In: MICCAI, 2012a.

626

Alberti M, Balocco S, Gatta C, Ciompi F, Pujol O, Silva J, Carrillo X,

627

Radeva P. Automatic bifurcation detection in coronary ivus sequences.

628

IEEE Trans. Biomed. Engineering, 2012b;594:10221031.

629

Balocco S, Gatta C, Ciompi F, Pujol O, Carrillo X, Mauri J, Radeva P.

630

Combining growcut and temporal correlation for ivus lumen segmentation.

631

In: IbPRIA, 2011. pp. 556563.

632

Bruining N, von Birgelen C, Prati F, Mallus M, Li W, Hoed WD, Patijn

633

M, de Feyter P, Serruys P, Roelandt J. Dynamic three-dimensional recon-

634

struction of icus images based on an ecg-gated pull-back device. In: EMBS,

635

1996.

636

Ciompi F, Gatta C, Pujol O, Rodriguez-Leor O, Mauri J, Radeva P. Recon-

637

struction and Analysis of Intravascular Ultrasound Sequences. Bentham

638

Science, Ch. 16, 2011. pp. 231250.

639

Ciompi F, Pujol O, Gatta C, Alberti M, Balocco S, Carrillo X, Mauri-Ferre

640

J, Radeva P. Holimab:

a holistic approach for media-adventitia border

641

detection in intravascular ultrasound. Med Image Anal, 2012;16:10851100.

642

Ciompi F, Pujol O, Gatta C, Leor OR, Ferre JM, Radeva P. Fusing in-vitro

643

and in.vivo intravascular ultrasound data for plaque characterization. Int

644

J Cardiovasc Imaging, 2010;26:763779.

30

645

Das S. Filters, wrappers and a boosting-based hybrid for feature selection.

646

In: Proceedings of the Eighteenth International Conference on Machine

647

Learning, 2001. pp. 7481.

648 649

Demsar J. Statistical comparisons of classiers over multiple data sets. Journal of Machine Learning Research, 2006;7:130.

650

Diletti R, Garcia-Garcia HM, Gomez-Lara J, Brugaletta S, Wykrzykowska

651

JJ, van Ditzhuijzen N, van Geuns RJ, Regar E, Ambrosio G, Serruys PW.

652

Assessment of coronary atherosclerosis progression and regression at bi-

653

furcations using combined IVUS and OCT. JACC Cardiovasc Imaging,

654

2011;4:774780.

655

Gatta C, Balocco S, Ciompi F, Hemetsberger R, Rodriguez-Leor O, Radeva

656

P. Real-time gating of ivus sequences based on motion blur analysis:

657

Method and quantitative validation. In: MICCAI, 2010. pp. 5967.

658

Holmes JN. Speech Synthesis and Recognition. Taylor & Francis, Inc., 1988.

659

Kovarnik T, Wahle A, Downe RW, Sonka M. Intravascular Ultrasound. In-

660

Tech, Ch. IVUS Role in Studies Assessing Atherosclerosis Development,

661

2012. pp. 5368.

662

Nair A, Kuban BD, Tuzcu EM, Shoenhagen P, Nissen SE, Vince DG. Coro-

663

nary plaque classication with intravascular ultrasound radiofrequency

664

data analysis. Circulation, 2002;106:22002206.

665

Nair A, Margolis MP, Kuban BD, Vince DG. Automated coronary plaque

666

characterisation with intravascular ultrasound backscatter: ex vivo valida-

667

tion. EuroIntervention, 2007;3:113120.

31

668

Nakayama

T,

Komiyama

N,

Yokoyama

M,

Namikawa

S,

Kuroda

N,

669

Kobayashi Y, Komuro I. Pioglitazone induces regression of coronary

670

atherosclerotic plaques in patients with type 2 diabetes mellitus or im-

671

paired glucose tolerance: a randomized prospective study using intravas-

672

cular ultrasound. Int J Cardiol, 2010;138:157165.

673 674

Navarro G. A guided tour to approximate string matching. ACM Comput. Surv., 2001;33:3188.

675

Nielsen NPV, Carstensen JM, Smedsgaard J. Aligning of single and multiple

676

wavelength chromatographic proles for chemometric data analysis using

677

correlation optimised warping. J Chromatogr, 1998;805:1735.

678

Nissen SE, Nicholls SJ, Sipahi I, et al. Eect of very high-intensity statin ther-

679

apy on regression of coronary atherosclerosis: The asteroid trial. JAMA:

680

The Journal of the American Medical Association, 2006;29513:15561565.

681

O'Malley SM, Granada JF, Carlier S, Naghavi M, Kakadiaris IA. Image-

682

based gating of intravascular ultrasound pullback sequences. IEEE Trans.

683

Inf. Technol. Biomed., 2008;12:299306.

684

Ramaker

H,

van

685

time

686

2003;498:133153.

warping

Sprang

of

ENM,

spectroscopic

Westerhuis batch

JA,

data.

Smilde

Analytica

AK.

Dynamic

Chimica

Acta,

687

Sakoe H, Chiba S. Dynamic programming algorithm optimization for spoken

688

word recognition. IEEE Trans Acoust Speech Signal Process, 1978;26:43

689

49.

32

690

Sathyanarayana

S,

Carlier

S,

Li

W,

Thomas

L.

Characterization

of

691

atheroscelrotic plaque by spectral similarity of radiofrequency intravas-

692

cular ultrasound signals. EuroIntervention, 2009;5:133139.

693 694

Sellers P. The theory and computation of evolutionary distances: Pattern recognition. J Algorithm, 1980;1:359373.

695

Shin ES, Garcia-Garcia HM, Okamura T, Serruys PW. Eect of statins on

696

coronary bifurcation atherosclerosis: an intravascular ultrasound virtual

697

histology study. Int J Cardiovasc Imaging, in press.

698 699

Smith TF, Waterman MS. Identication of common molecular subsequences. J Mol Biol, 1981;147:195197.

700

Stone GW, Maehara A, Lansky AJ, de Bruyne B, Cristea E, Mintz GS,

701

Mehran R, McPherson J, Farhat N, Marso SP, Parise H, Templin B, White

702

R, Zhang Z, Serruys PW and Investigators P. A prospective natural-history

703

study of coronary atherosclerosis. N Engl J Med, 2011;364:226235.

704

Thim T, Hagensen MK, Wallace-Bradley D, Granada JF, Kaluza GL, Drouet

705

L, Paaske WP, Botker HE, Falk E. Unreliable assessment of necrotic core

706

by virtual histology intravascular ultrasound in porcine coronary artery

707

disease. Circ Cardiovasc Imaging, 2010;3:384391.

708

Vlachos M, Hadjieleftheriou M, Gunopoulos D, Keogh E. Indexing multi-

709

dimensional time-series with support for multiple distance measures. In:

710

Proceedings of ACM SIGKDD, 2003. pp. 216225.

711

Wykrzykowska JJ, Mintz GS, Garcia-Garcia HM, Maehara A, Fahy M, Xu k,

712

Inguez A, Fajadet J, Lansky A, Templin B, Zhang Z, de Bruyne B, Weisz

33

713

G, Serruys PW, Stone GW. Longitudinal distribution of plaque burden and

714

necrotic core-rich plaques in nonculprit lesions of patients presenting with

715

acute coronary syndromes. JACC Cardiovasc Imaging, 2012;5:S1018.

716 717

Zhou F, de la Torre F. Canonical time warping for alignment of human behavior. In: NIPS, 2009. pp. 22862294.

34

718

Figure Captions

719

Figure 1:

Pair of IVUS sequences of the same vessel: (a) longitudinal views,

720

(b) corresponding frames and (c) temporal signals describing the pull-

721

backs.

722 723

724

Figure 2:

Example of MCD matrix

of length

Figure 3:

D

and

warping path

nx , ny .

Three examples of sliding positions for an idealized couple of se-

(X, Y )

725

quences

726

Yiter

727

overlapping window is indicated by diagonal traits.

728

Figure 4:

for two sequences

(continuous lines).

At each iteration

are the overlapping subsequences of

X

Y,

and

iter, Xiter

respectively. The

(a) General scheme of the EPS approach, (b) detailed scheme of

729

end of match

730

by the DTW algorithm, shown superimposed on the MCD matrix

731

Figure 5:

and

search and (c) example of output

Idealized pairs of sequences

warping path

obtained

D.

(X, Y ) (continuous lines), and frame-

732

to-frame correspondences (dotted lines), before and after the distortion

733

simulation: (a) amplitude distortion, (b) partial overlapping, (c), (d)

734

and (e) temporal distortions.

735

Figure 6: E

736

lapping

737

Figure 7:

as a function of (a) amplitude distortion

w2 ,

(c), (d) and (e) temporal distortions

Ground-truth and automatic

warping path

738

pair in Data-set B, computed (a) by

739

EPS-RC.

740

smaller the error.

w1 ,

(b) partial over-

w3 , w4

for an

DTW -SW -RC

and

w5 .

in-vivo pullback

and (b) by

DTW-

Note that the closer the dotted line is to the crosses, the

35

741

Figure 8:

Ground-truth and automatic

warping path

for

in-vivo

pullback

742

pairs in Data-set A (rst row) and B (second row), computed by

743

EPS-RC.

744

smaller the error.

745

Figure 9:

DTW-

Note that the closer the dotted line is to the crosses, the

Examples of frame-to-frame correspondences. A matrix report-

warping path

and the

linear

746

ing the ground-truth, the automatic

align-

747

ment (rst column), a frame on P1 (second column), the corresponding

748

frames on P2 identied by automatic alignment (third column) and by

749

linear

750

Figure 10:

tting (fourth column) are shown.

Frame-to-frame correspondences in a vessel segment pre-post

751

stent deployment. A matrix reporting the ground-truth, the automatic

752

warping path,

753

ment (rst column), a frame on P1 (second column), the corresponding

754

frames on P2 identied by automatic alignment (third column) and by

755

linear

the

linear

alignment and the extension of the stent seg-

tting (fourth column) are shown.

36

756

Tables

757

Table 1:

Performance of the applied bifurcation detection (Alberti et al.,

758

2012b) and plaque characterization (Ciompi et al., 2010) methods in the

759

original publications, in terms of the following classication parameters:

760

accuracy (

761

ratio (

762

used implementation (

A), sensitivity (S ), precision (P ), specicity (K ), false alarm

FAR )

and F-Measure (

computational time per frame in the

sec/frame ).

Bif urcation

F ibrotic plaque

Lipidic plaque

Calcif ied plaque

A (%)

95.21 ± 2.28

87.09 ± 0.33

87.09 ± 0.33

87.09 ± 0.33

S (%)

80.66 ± 14.90

90.66 ± 0.43

58.46 ± 1.61

99.37 ± 0.22

P (%)

94.60 ± 2.65

73.85 ± 0.72

92.59 ± 0.33

95.82 ± 0.26

K (%)

93.38 ± 2.29

85.86 ± 0.55

98.56 ± 0.09

96.32 ± 0.24

F AR (%)

4.62 ± 2.29

F (%)

86.35 ± 9.28 97

97

97

763

T ime (sec/f rame)

764

765

F );

Table 2:

50

Performance of the applied media segmentation (Ciompi et al.,

766

2012) and lumen segmentation (Balocco et al., 2011) methods in the

767

original publications, in terms of the following error measures: mean

768

radial distance (

769

radial distance (

770

(

771

computational time per frame in the used implementation (

mrd ),

MRD ),

maximum radial distance (

SgnMrd ),

Hausdor distance (

HD ),

signed mean

mean area error

mae ), Jaccard measure (JM ) and percentage of area dierence (PAD ); sec/frame ).

37

Media

(stent

Lumen

(bifur-

frames)

cation frames)

0.17 ± 0.08

0.18 ± 0.07

0.31 ± 0.12

M RD (mm)

0.551 ± 0.381

0.57 ± 0.24

0.62 ± 0.19

0.12 ± 0.44

11

11

11

−0.020 ± 0.344

HD (mm)

0.538 ± 0.381

mae (mm2 )

0.594 ± 0.674

JM (%)

87.5 ± 9.7

P AD (%)

10.1 ± 12.4

T ime (sec/f rame)

Table 3:

102

Quantitative results (

MEAN ±STD ) for E (number of gated frames)

775

on synthetic data, as a function of

776

tures), where 1 gated frame

777

Lumen

0.211 ± 0.182

772

774

(all

frames)

mrd (mm)

SgnM rd (mm)

773

Lumen

≈

m

(number of morphological fea-

0.5 mm.

m=1

m=2

m=3

m=4

m=5

CT W

9.36 ± 9.15

4.24 ± 4.28

2.52 ± 2.03

1.87 ± 1.18

1.52 ± 0.63

CT W − SW

6.56 ± 9.33

2.53 ± 3.15

1.74 ± 1.95

1.23 ± 0.63

1.12 ± 0.49

CT W − SW − RC

5.5 ± 8.32

2.23 ± 2.33

1.55 ± 1.05

1.21 ± 0.59

0.98 ± 0.38

COW

8.01 ± 3.75

9.21 ± 4.16

9.51 ± 4.36

9.78 ± 4.42

9.45 ± 4.16

COW − SW

4.36 ± 5.44

2.4 ± 1.31

2.13 ± 0.94

2.09 ± 0.85

2.16 ± 0.88

COW − SW − RC

4.05 ± 4.95

2.33 ± 1.22

2.09 ± 0.93

2.02 ± 0.89

1.98 ± 1.04

DT W

9.03 ± 9.45

4.05 ± 4.16

2.4 ± 2.21

1.6 ± 1.02

1.26 ± 0.58

DT W − SW

7.54 ± 10.64

3.25 ± 5.28

1.87 ± 2.79

1.2 ± 1.09

0.96 ± 0.41

DT W − SW − RC

3.75 ± 5.8

1.79 ± 1.37

1.24 ± 0.77

0.96 ± 0.42

0.8 ± 0.29

DT W − EP S − RC

7.04 ± 10.02

2.67 ± 3.97

1.51 ± 2.01

1.04 ± 1.24

0.76 ± 0.29

38

778

779 780

Table 4:

Quantitative

in-vivo results (MEAN ±STD ) for E (number of gated

frames) on Data-sets A and B, where 1 gated frame

Data − set A

Data − set B

CT W

2.75 ± 4.11

2.33 ± 1.99

CT W − SW

1.33 ± 1.11

2.37 ± 2.92

CT W − SW − RC

1.08 ± 0.72

1.56 ± 1.27

COW

6.94 ± 8.19

5.92 ± 3.92

COW − SW

2.15 ± 0.98

1.88 ± 0.98

COW − SW + −RC

1.83 ± 0.79

2.08 ± 0.8

DT W

2.63 ± 3.72

2.07 ± 2.38

DT W − SW

1.19 ± 0.62

2.17 ± 2.72

DT W − SW − RC

1.21 ± 0.55

1.47 ± 1.02

DT W − EP S − RC

0.85 ± 0.37

1.53 ± 0.92

781

782

inter − observer

1.2 ± 1.41

39

≈

0.5 mm.