Comparing Unet training with three different datasets to correct CBCT

Comparing Unet training with three different datasets to correct CBCT images for prostate radiotherapy dose calculations Guillaume Landry1 , David Hansen2 , Florian Kamp3 , Minglun Li3 , Ben Hoyle4,5 , Jochen Weller4,5,6 , Katia Parodi1 , Claus Belka3,7 , and Christopher Kurz1,3 1

Department of Medical Physics, Fakult¨at f¨ ur Physik, Ludwig-Maximilians-Universit¨at M¨ unchen (LMU Munich), Garching, Germany 2 Gradient Software, Aarhus, Denmark 3 Department of Radiation Oncology, University Hospital, LMU Munich, Munich, Germany 4 Universit¨ ats-Sternwarte, Fakult¨at f¨ ur Physik, Ludwig-Maximilians Universit¨at M¨ unchen, M¨ unchen, Germany 5 Excellence Cluster Universe, Garching, Germany 6 Max Planck Institute for Extraterrestrial Physics, Garching, Germany 7 German Cancer Consortium (DKTK), Munich, Germany E-mail: [email protected] Abstract. Image intensity correction is crucial to enable cone beam computed tomography (CBCT) based radiotherapy dose calculations. This study evaluated three different deep learning based correction methods using a U-shaped convolutional neural network architecture (Unet) in terms of their photon and proton dose calculation accuracy. CT and CBCT imaging data of 42 prostate cancer patients were included. For target ground truth data generation, a CBCT correction method based on CT to CBCT deformable image registration (DIR) was used. The method yields a deformed CT called (i) virtual CT (vCT) which is used to generate (ii) corrected CBCT projections allowing the reconstruction of (iii) a final corrected CBCT image. The single Unet architecture was trained using these three different datasets: (Unet1) raw and corrected CBCT projections, (Unet2) raw CBCT and vCT image slices and (Unet3) raw and reference corrected CBCT image slices. Volumetric arc therapy (VMAT) and proton pencil beam scanning (PBS) single field uniform dose (SFUD) plans were optimized on the reference corrected image and recalculated on the obtained Unet-corrected CBCT images. The mean error (ME) and mean absolute error (MAE) for Unet1/2/3 were -1/2/3 Hounsfield units (HU) and 48/88/56 HU. The 1% dose difference pass rates were better than 98.4% for VMAT for 8 test patients not seen during training, with little difference between Unets. Gamma evaluation results were even better. For protons a gamma evaluation was employed to account for small range shifts, and 2%/2mm pass rates for Unet1/2/3 were better than 85%/89% and 91%. A 3 mm range difference threshold was established. Only for Unet3 the 5th and 95th percentiles of the range difference distributions over all fields, test patients and dose profiles were within this threshold.

CBCT Unet comparison

2

A single Unet architecture was successfully trained using both CBCT projections and CBCT image slices. Since the results of the other Unets were poorer than Unet3, we conclude that training using corrected CBCT image slices as target data is optimal for PBS SFUD proton dose calculations, while for VMAT all Unets provided sufficient accuracy.

Submitted to: Phys. Med. Biol.

CBCT Unet comparison

3

1. Introduction

5

10

15

20

25

30

35

40

In modern image-guided external beam radiotherapy, cone-beam CT (CBCT) imaging plays an integral role for accurate patient alignment. While CBCT systems have been widely adopted in photon radiotherapy installations, they have recently also appeared at proton beam facilities (Veiga et al. 2016, Hua et al. 2017, Stock et al. 2017, Landry & Hua 2018). Although the acquired CBCT images can be used for accurate 3D anatomy based patient positioning, the image quality is typically not sufficient to perform accurate dose calculations and to infer the applied daily dose or even adapt the treatment plan (Kurz et al. 2015). The latter would particularly be desirable for anatomical sites which undergo considerable inter-fractional anatomical changes in order to fully exploit the potential of modern highly conformal treatment techniques, such as volumetric modulated arc photon therapy (VMAT) or pencil beam scanning (PBS) proton therapy, which allows for intensity modulated proton therapy (IMPT). In the literature, a wide range of techniques have been proposed to correct for CBCT imaging artifacts related to, among others, scatter detection (Siewerdsen & Jaffray 2001), beam hardening (Thing et al. 2016) or scatter glare (Poludniowski et al. 2011). These techniques include simple look-up table based approaches (Kurz et al. 2015), the use of planning CT (pCT) to CBCT deformable image registration (DIR) (Landry et al. 2014, Landry et al. 2015, Veiga et al. 2014, Veiga et al. 2015, Wang et al. 2016, Veiga et al. 2016) and Monte-Carlo (MC) based methods for scatter estimation (Mainegra-Hing & Kawrakow 2010, Thing et al. 2016, Z¨ollner et al. 2017). Several of these methods have been shown to allow for accurate photon (Ding et al. 2007, Fotina et al. 2012, Veiga et al. 2014) or even proton dose (Landry et al. 2014, Landry et al. 2015, Veiga et al. 2015, Veiga et al. 2016) calculation in the scope of specific applications and treatment sites. Still, most of these methods have their own particular limitations: DIR based methods, generating a so-called virtual CT (vCT) (Peroni et al. 2012), were shown suitable for proton therapy of the head and neck region, but failed for the pelvic region due to the more pronounced and complex anatomical changes from fraction to fraction (Kurz et al. 2016). In such scenarios, improved results were attained with methods that only rely on the generated vCT as prior for projection based intensity correction (Niu et al. 2010, Niu et al. 2012, Park et al. 2015, Kurz et al. 2016). DIR inaccuracies only slightly affect the yielded corrected CBCTs image. However, these methods typically generate corrected CBCT images on a time scale of minutes, which is not acceptable if aiming at using CBCT images for daily pre-treatment adaptation. For the same reason, accurate, but slow MC based methods are currently not a preferred solution. A promising recent approach for fast (sub-second per image slice) and accurate CBCT image correction is based on deep convolutional neural networks (CNN), which have been shown to yield impressive results for various image-to-image translation tasks (Han 2017, Wolterink et al. 2017). One of the most commonly used architectures, a U-shaped CNN (Unet) (Ronneberger et al. 2015), has also been applied to CBCT

CBCT Unet comparison

45

50

55

60

65

4

intensity correction in a few recent publications. In (Kida et al. 2018) a Unet was used for translating the CBCT to a pCT equivalent image by training the network using the uncorrected CBCT images and corresponding vCTs as input. In (Maier et al. 2018a, Maier et al. 2018b), a Unet was trained for projection based image correction using simulated CBCT data on the basis of diagnostic pCT images of the head and neck region. In particular, the network was trained to estimate the MC simulated scatter contributions in the simulated CBCT forward projections in order to perform a projection based CBCT correction. (Maier et al. 2018c) applied a similar approach to industrial x-ray imaging. Neither of the studies on medical imaging investigated the accuracy of dose calculation. A similar projection based approach was investigated by (Hansen et al. 2018) for a group of prostate cancer patients. The latter study also investigated CBCT based dose calculation accuracy, which was found high (100% for a 2% dose difference test) for VMAT, but still limited for IMPT due to the high sensitivity of proton ranges on the CT numbers. All three studies concluded that intensity correction can be performed on a sub-second scale. However, it is unclear whether training the Unet on projections or reconstructed images is optimal in the context of photon and the more challenging case of proton dose calculation. For this work, the Unet architecture from (Hansen et al. 2018) was used as a starting point to investigate dose calculation accuracy. So far, all publications have chosen a certain Unet architecture and a single given set of training data pairs, i.e., raw CBCT and vCT (Kida et al. 2018) or raw projection and corrected projection (Maier et al. 2018a, Hansen et al. 2018). In this work, we have attempted to see whether the same Unet architecture could be trained using the alternative types of training data and to identify which training approach is optimal for dose calculation. Thus, a single Unet architecture was trained using as input and ground truth target data for training: (Unet1) Raw and corrected CBCT projections, as in (Maier et al. 2018a, Hansen et al. 2018)

70

(Unet2) Original CBCT and vCT images, as in (Kida et al. 2018) (Unet3) Original and CBCT images corrected with the methods from (Kurz et al. 2016), which is an original approach from this work

75

80

The specific Unet architecture was taken from (Hansen et al. 2018), under the assumption that the network would learn, via the use of a residual architecture, which specific components of the network can be set to identity for a certain training dataset. Thus the only difference between projection- and image-based Unets, besides the nature of the input data, was the data augmentation used during training, which were adopted from the original publications (Kida et al. 2018, Hansen et al. 2018). Accuracy of the retrieved corrected CBCTs was evaluated in terms of Hounsfield unit (HU) accuracy as well as in terms of photon and proton dose calculation accuracy for the pelvic region.

CBCT Unet comparison

5

2. Materials and methods 2.1. Patient data

85

90

95

100

CT and CBCT imaging data of 42 prostate cancer patients treated with VMAT were used in this study. CT images, originally used for treatment planning, were acquired with a Toshiba Acquilion LB CT scanner (Canon Medical Systems, Japan) with an image grid of 1.074 mm×1.074 mm×3.0 mm. The CBCT images were acquired in treatment position with the XVI system (version 5.0.2) of a Synergy medical linear accelerator (Elekta, Sweden). For this study, we selected cases where CBCT images were acquired with the socalled 20×20 protocol, which uses exposures of 20 ms at an x-ray tube current of 20 mA per projection. The protocol made use of a bow-tie filter, had a superior-inferior field of view of 20 cm and a shifted detector panel (M position, for larger lateral field of view). The number of projections per scan was between 346 and 357. This protocol ensured that all parts of the flat panel detector were below their saturation point, which ensures accurate patient outline reconstruction. CBCT projections were reconstructed using the GPU FDK implementation of RTK (Reconstruction ToolKit, (Rit et al. 2014)) on a 1.0 mm×1.0 mm×1.0 mm grid (CBCTorg ). The CTs of all patients were delineated by a trained physician as part of the treatment procedure. A CTV-to-PTV margin of 7 mm, following clinical protocol, was used to account for positioning inaccuracies and inter-fractional anatomical changes. 2.2. Ground truth target data for training

105

The CBCT images were corrected for typical CBCT artefacts and converted to pCT equivalent HU numbers by using the method of (Park et al. 2015). Briefly, the method makes use of a vCT obtained by DIR of the pCT to a CBCT image. The vCT is used to generate idealized projections which allow estimation and correction of non-idealities in the CBCT projections.

/I 0)

FDK

ICBCTcor = CF × ICBCTorg − ISCA

DIR

DRR

−ln(I/I0 )

I0 · e−DRR

FDK

Figure 1. Flow chart illustrating how CBCTcor projections and images are obtained from CBCTorg projections and images. FDK, DIR and DRR stand for CBCT image recontruction, CT to CBCT deformable registration and digitally reconstructed radiographs respectively. The operations −ln(I/I0 ) and I0 e−DRR , where I0 is the open field signal and I the measured projection intensity, are used to alternate between measured intensity and the projections required for FDK reconstruction. f is a smoothing operation and CF is a scaling factor, while ISCA are the low frequency deviations between the vCT projection (IvCT ) and the measured projections (ICBCTorg ) (Park et al. 2015).

ISCA = f (CF × ICBCTorg − IvCT)

(I −ln

CBCT Unet comparison 6

7

CBCT Unet comparison

110

This approach has been extensively evaluated in other reports in terms of HU as well as photon and proton therapy dose calculations (Park et al. 2015, Kurz et al. 2016, Z¨ollner et al. 2017), and will not be described in detail here. For a detailed description pertaining to this work please see (Hansen et al. 2018). The process yields three datasets which will be used in this work: (i) corrected CBCT projections (ii) vCT (iii) corrected CBCT image, CBCTcor , reconstructed from the corrected projections

115

Figure 1 illustrates the steps required to obtain CBCTcor , and figure 2 shows example slices of the original CBCT (CBCTorg ), vCT and CBCTcor for two patients.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 2. Example dataset used in this study for patient 26 (a)(c)(e) and patient 9 (b)(d)(f), both from the test dataset.

CBCT Unet comparison

8

2.3. Data partitioning for training

120

The patient data were split into three groups: training (27), validation (7) and testing (8) using the common 60/20/20 partitioning approach. The training data were used to train the network, while the validation data were used to decide when to stop training. The independent testing data were used for the evaluations described later in section 2.7. 2.4. Unet and training

125

130

135

140

145

150

We made use of the Unet architecture introduced in (Hansen et al. 2018) for CBCT projection correction. The building blocks of the Unet are illustrated in figure 3, and the Unet itself in figure 4. Three training approaches were considered in this work and will be referred to as Unet1, Unet2 and Unet3. Besides the Unet architecture, implemented in pytorch (Paszke et al. 2017), the loss function (mean squared error), optimizer (Adam) (Kingma & Ba 2015), GPU (Nvidia Quadro P5000) and batch size (8) were common to all approaches. For all cases the training was stopped when the loss function of the validation data began increasing. Figure 4 illustrates the datasets used to train the three Unets, with detailed explanations below. 2.4.1. Unet1 Unet1 followed (Hansen et al. 2018), where the raw CBCT projections were converted to the log-domain, using scan specific flood-field images for the log transform, which was done with RTK. For Unet1 the input and target data for training were the original and the corrected projections. 9472 projection pairs were used for training and 2451 for validation. All projections were zero padded to a size of 512×512 (from 510×510) and training was done in 2D on a projection-per-projection basis with a batch size of 8. The data augmentation was the same as in (Hansen et al. 2018), using mixup (Zhang et al. 2018), where two unrelated projections are linearly combined with a weighting factor sampled from a uniform distribution. The learning rate was set to 3×10−4 . 2.4.2. Unet2 Unet2 followed the training presented in (Kida et al. 2018), where the vCT is used to train the Unet to convert the original CBCT images. The input and target data were thus pairs of CBCTorg and vCT image slices. Following DIR, the deformed pCT image was re-sampled onto the CBCT image grid. Similarly to (Kida et al. 2018), a binary mask was used to set regions outside the patient’s outline to the values of air prior to training. Only the cylindrical part of the field-of-view was retained, i.e., the conical ends on the cranial and the caudal side were cropped, preserving the central 200 slices, thus yielding 5400 image pairs for training and 1400 for validation. The images were also zero padded to a size of 512×512 (from 410×410). Data augmentation was similar to (Kida et al. 2018); a sequence of random left-right flips, position shifts and HU shifts were applied. The position shifts were sampled from

9

CBCT Unet comparison Input Convolution 3x3 Input

Input

PReLU

Convolution 3x3, 1→8 channels

Convolution 3x3

PReLU Convolution 3x3, 8→1 channels

+

PReLU

PReLU

(a) convert

(b) residual

Input

Bilinear upsampling

Input1

Input2

Convolution 2x2, stride 2

Convolution 3x3

Concatenation

PReLU

PReLU

Convolution 1x1

Res block

Res block

PReLU

(c) down

(d) up

(e) mix

Figure 3. (a)-(e) Unet building blocks based on parametric rectified linear units (PReLU) and (b) residual blocks (Res block).

160

[-5,5] mm at integer values to avoid interpolating the images, which would introduce smoothing. HU shifts were sampled from [-20,20] HU. While shifts and flips were applied to both input and target data, the HU shifts were only applied to the input to simulate variability in CBCT acquisitions. For Unet2 it was necessary to reduce the learning rate to 2×10−4 to avoid spikes in the validation loss which compromised the convergence of the training.

165

2.4.3. Unet3 For Unet3 the vCT target data of Unet2 were replaced by CBCTcor image slices, while the input remained CBCTorg image slices. The same binary masks, axial field of view cropping, padding and data augmentation were employed. The learning rate was set to 3×10−4 as for Unet1, since the spikes seen with Unet2 were not observed for Unet3.

155

170

Unet3mixup As a short additional study, a variant of Unet3 was trained with the mixup data augmentation used for Unet1 instead of flips, shifts and HU shifts. However, contrary to the uniform distribution of linear interpolation weights used for Unet1, the Beta distribution (Beta(α, α)) from the original mixup paper (Zhang et al. 2018) was employed with α = 0.2. This means that the interpolation weights were more frequently sampled near 0 and 1 than 0.5. Unet3mixup results will be presented in an abridged fashion, with the main focus placed on Unet3. 2.5. Corrected images

175

The final trained models were applied to the training, validation and test datasets. For Unet1 the original projections were corrected and a subsequent FDK reconstruction yielded CBCTUnet1 . For Unet2 and Unet3 the models were applied directly on the masked CBCTorg , yielding CBCTUnet2 and CBCTUnet3 . 2.6. Dose calculation

180

CBCTcor images from the test dataset were imported to a research version of a commercial treatment planning system (TPS, RayStation version 4.99, RaySearch,

10

CBCT Unet comparison Input

1

Convert block

+

1

Output

L2 norm

Target

Convolution 1x1

1

Res block

Mix block 8

Up block

8

Unet1

Down block

Mix block 16

Up block

16

Down block

Mix block 32

Up block

32

Down block

Mix block 64

Up block

64

Unet2

Down block

Mix block 128

128

Up block

Unet3

Down block

256

Figure 4. The Unet architecture and the data used for training Unet1, Unet2 and Unet3. The numbers next to the arrows indicate the number of channels.

185

190

195

Sweden) along with the delineated structures obtained from the pCT to CBCT DIR. No further corrections were applied to the delineations, since the goal of this study was not the evaluation of the accuracy of DIR-generated contours. All plans aimed at a median CTV dose of 74 Gy in 37 fractions. All plans had a CTV V95% of 100% and PTV V95% above 98%. For the OARs, in particular the bladder and the rectum, plans aimed at respecting the upper dose thresholds given in the QUANTEC report (Marks et al. 2010). VMAT plans were optimized using one full arc and the Elekta Synergy beam model implemented in RayStation. A 3.0 mm×3.0 mm×3.0 mm dose grid was used for dose calculation with the collapsed-cone algorithm. Proton PBS plans were optimized using two opposing beams at 90◦ and 270◦ gantry angle (i.e., horizontally from the left and right of the patient). Single field uniform dose (SFUD) optimization was used, ensuring homogeneous dose distributions for each field. The implemented IBA Dedicated beam model and a 3.0 mm×3.0 mm×3.0 mm dose grid were used during pencil-beam dose calculation. Two additional PBS plans using only one of the two fields above were optimized on a 1.0 mm×1.0 mm×3.0 mm dose grid to investigate proton range accuracy. The 3.0 mm slice thickness was used to reduce memory consumption since the range was never measured in that direction. 2.7. Data evaluation

200

The mean error (ME) and mean absolute error (MAE) in HU of CBCTUnet1 , CBCTUnet2 and CBCTUnet3 compared to CBCTcor was computed for all patients.

CBCT Unet comparison

205

210

11

The dose distributions of all plans were recomputed on the corresponding CBCTUnet1 , CBCTUnet2 and CBCTUnet3 . For each plan, the VMAT and PBS dose distributions were compared to the one obtained using CBCTcor based on a gamma evaluation focusing on doses above 50% of the prescription. Criteria of 1%/1 mm and 2%/2 mm for VMAT and 2%/2 mm and 3%/3 mm for PBS plans were applied. The amount of voxels passing the gamma evaluation, as well as the amount of voxels showing percentage dose differences lower than 1% and 2% (VMAT) and 2% and 3% (PBS) were determined. Looser criteria were chosen for PBS given the more stringent nature of proton dose calculation. For PBS single field plans, range differences were calculated in beam’s-eye-view for the 90◦ and 270◦ beams by comparing the location of the distal 80% isodose on CBCTcor and CBCTUnet1/2/3 along individual dose profiles, yielding 2D distributions of range shifts for each field. 3. Results

215

220

225

3.1. Network training and timing Figure 5 shows the loss function for the training (left column) and validation data (right column) per network training iteration for each Unet. The training was stopped at iterations 140, 95 and 140 for Unet1, Unet2 and Unet3 respectively, according to the increase of the loss function of the validation dataset. The loss functions for Unet2 and Unet3 showed similar values for both training and validation datasets. The loss function values of Unet1 were not directly comparable to those of Unet2 and Unet3 given the different nature of the input images and data augmentation. Training times were 14.5 hours for Unet1, 6 hours for Unet2 and 9 hours for Unet3. The average time to correct an input projection for Unet1 was 12.5 ms, corresponding to 4.4 s for a 350 projections complete scan. For Unet2/3 the time to correct an image slice was 11 ms, with an entire volumetric image (264 slices) requiring 2.9 s. For comparison, image reconstruction required approximately 5 s on the same GPU, including read and write operations. 3.2. Image analysis

230

235

Figure 6 presents the results of applying the trained networks to patient 9 (chosen arbitrarily) of the test dataset (see Figure 2(b)), as well as the difference of the resulting CBCTUnet1/2/3 with respect to CBCTcor (see Figure 2(f)). Both CBCTUnet1 and CBCTUnet3 exhibited a similar noise behavior as CBCTcor , while CBCTUnet2 was noticeably smoother than the other CBCT images and the vCT. CBCTUnet2 did not exhibit the higher intensities observed for pixels at the patient’s skin for CBCTUnet1 and CBCTUnet3 , which had also appeared in CBCTcor . For the three Unets, areas of HU under- and over-estimation were observed in the difference images, however, systematic shifts were not discernible.

12

CBCT Unet comparison

(a)

(b)

(c)

(d)

(e)

(f)

Figure 5. Value of the loss function for (a),(c),(e) the training and (b),(d),(f) validation datasets for (a),(b) Unet1, (c),(d) Unet2 and (e),(f) Unet3. A smoothed curved was added to the validation loss functions for display purposes.

240

245

Figure 7 reports quantitative results in terms of the ME in HU with respect to CBCTcor per patient for the three Unets and CBCTorg for the training, validation and testing datasets. CBCTUnet1 showed the overall lowest ME for the training dataset, while CBCTUnet2 and CBCTUnet3 showed a systematic deviation of up to 10 HU and 5 HU, respectively. For the three Unets, the validation datasets exhibited ME limited to ±15 HU with no clear systematic behavior. Unet1 showed the best ME results for the test dataset, with ME contained within ±7 HU. Unet2 and Unet3 showed ME of up to

13

CBCT Unet comparison

(a)

(b)

(c)

(d)

(e)

(f)

Figure 6. (a),(c),(e) Corrected images from the three Unets for a slice containing the PTV for test dataset patient 9. (b),(d),(f) Difference images between the Unet results and the reference corrected CBCT images used for training. (a) CBCTUnet1 shows the patient table since it was reconstructed from corrected projections.

250

17 HU and 12 HU respectively. Figure 8 is similar to Figure 7 but shows the MAE in HU. CBCTUnet1 and CBCTUnet3 showed the overall lowest MAE, while CBCTUnet2 had markedly higher MAE. For the three Unets, the MAE was lower than for CBCTorg . Table 1 reports the ME and MAE averaged over all patients and over the test dataset for the three Unets. In terms of average ME the three Unets showed similar, near 0 HU results. The average MAE were higher for CBCTUnet2 , suggesting that the noise in CBCTcor and CBCTUnet1 as well as CBCTUnet3 is correlated. Table 1 additionally reports results for the Unet3mixup variant. The results were

14

CBCT Unet comparison

ME CBCTcor - CBCTorg

15 10

ME in HU

ME in HU

60 40 20

ME CBCT cor - CBCT Unet1

20

Train Val Test

5 0 5 10

0

15

0 1 3 4 5 6 9 14 16 17 18 19 20 22 23 24 25 26 28 30 31 32 34 35 36 37 38 39 40 41 43 44 45 46 48 50 51 52 53 54 56 57

20

Patient number

Train Val Test 0 1 3 4 5 6 9 14 16 17 18 19 20 22 23 24 25 26 28 30 31 32 34 35 36 37 38 39 40 41 43 44 45 46 48 50 51 52 53 54 56 57

80

Patient number

(a)

ME CBCT cor - CBCT Unet2

15

10

10

ME in HU

15

5 0 5

15 20

5 0 5 10

Train Val Test

15

0 1 3 4 5 6 9 14 16 17 18 19 20 22 23 24 25 26 28 30 31 32 34 35 36 37 38 39 40 41 43 44 45 46 48 50 51 52 53 54 56 57

10

Patient number

(c)

ME CBCT cor - CBCT Unet3

20

20

Train Val Test 0 1 3 4 5 6 9 14 16 17 18 19 20 22 23 24 25 26 28 30 31 32 34 35 36 37 38 39 40 41 43 44 45 46 48 50 51 52 53 54 56 57

20

ME in HU

(b)

Patient number

(d)

Figure 7. Mean error (ME) per patient for the comparison of CBCTcor and (a) CBCTorg , (b) CBCTUnet1 , (c) CBCTUnet2 and (d) CBCTUnet3 . The data are labeled as belonging to the training, validation and testing datasets.

255

similar to those of Unet3 in terms of ME and MAE and per patient analysis yielded results similar to those shown in Figure 7(d) (not shown).

Table 1. Average ME and MAE over all patients and over the patients of the test dataset. All values in HU. The numbers in square brackets indicate the minimum and maximum over all patients.

CBCTorg Unet1 Unet2 Unet3 Unet3mixup ME all patients 32 [7,54] -1 [-9,15] 3 [-14,17] 2 [-11,12] 1 [-11,15] test patients 30 [11,54] -2 [-7,6] 2 [-10,16] -1 [-11,12] 1 [-10,15] MAE all patients 105 [88,124] 48 [36,62] 88 [71,109] 56 [43,69] 55 [41,69] test patients 104 [91,119] 51 [43,62] 88 [75,105] 58 [49,69] 58 [48,69]

15

CBCT Unet comparison 160

MAE CBCTcor - CBCTorg Train Val Test

140

MAE in HU

MAE in HU

120 100 80 60 40 20

Patient number

(b)

Train Val Test

MAE in HU

MAE in HU

100 80 60 40 20 0 1 3 4 5 6 9 14 16 17 18 19 20 22 23 24 25 26 28 30 31 32 34 35 36 37 38 39 40 41 43 44 45 46 48 50 51 52 53 54 56 57

0

Patient number (c)

90 80 70 60 50 40 30 20 10 0

MAE CBCTcor - CBCTUnet3 Train Val Test

0 1 3 4 5 6 9 14 16 17 18 19 20 22 23 24 25 26 28 30 31 32 34 35 36 37 38 39 40 41 43 44 45 46 48 50 51 52 53 54 56 57

MAE CBCTcor - CBCTUnet2 120

Train Val Test

Patient number

(a)

140

MAE CBCTcor - CBCTUnet1

0 1 3 4 5 6 9 14 16 17 18 19 20 22 23 24 25 26 28 30 31 32 34 35 36 37 38 39 40 41 43 44 45 46 48 50 51 52 53 54 56 57

0 1 3 4 5 6 9 14 16 17 18 19 20 22 23 24 25 26 28 30 31 32 34 35 36 37 38 39 40 41 43 44 45 46 48 50 51 52 53 54 56 57

0

90 80 70 60 50 40 30 20 10 0

Patient number (d)

Figure 8. Mean absolute error (MAE) per patient for the comparison of CBCTcor and (a) CBCTorg , (b) CBCTUnet1 , (c) CBCTUnet2 and (d) CBCTUnet3 . The data are labeled as belonging to the training, validation and testing datasets. Notice the different vertical scales.

3.3. Dose analysis

260

265

270

For the VMAT plans, the 2%/2mm gamma evaluation and 2% dose difference pass rates were better than 99.5% for all Unets and patients. Evaluation with a 1% dose difference criterion yielded passing rates better than 99.5% for all test patients and Unets as well, except for patient 26 and Unet3 where the pass rate was 98.4%. Given these figures, additional VMAT dose calculation accuracy results will not be shown. Figure 9 illustrates the dose calculation accuracy for the PBS SFUD plan for patient 9, where dose recalculations on CBCTUnet1/2/3 are shown along with the one from the original plan. Differences in the dose distributions recalculated on CBCTUnet1/2/3 and CBCTcor were mostly outside the PTV, which was consistently covered, and attributed to proton range shifts. The patterns of over- and under-dosage caused by range shifts were similar for CBCTUnet2 and CBCTUnet3 , while CBCTUnet1 showed a distinct pattern. Figure 10 presents the gamma evaluation pass rates for the three Unets and all test patients, as well as the percentage dose difference pass rates, for the PBS SFUD plan.

16

CBCT Unet comparison

(a)

(b)

(c)

(d)

(e)

(f)

(g)

Figure 9. (a) Dose distribution from the PBS SFUD plan optimized on CBCTcor and recalculated on (b) CBCTUnet1 , (d) CBCTUnet2 and (f) CBCTUnet3 and (c),(e),(g) the difference to the dose from CBCTcor , respectively

17

CBCT Unet comparison

100

percent difference pass rate/%

280

gamma pass rate/%

275

We observed that Unet3 yielded the highest pass rates for both evaluations, and was the only network for which the 2%/2mm gamma evaluation yielded pass rates above 90% for all patients. For the percent difference pass rate, Unet3 showed results better than 85% for the 3% criterion. Unet1 showed the lowest gamma and percent difference pass rates, while Unet2 exhibited intermediate results. Figures 11(a) and 11(b) show the SFUD proton range difference distributions per gantry angle, patient and Unet. Results for both gantry angles were comparable. The best overall performance was again achieved by Unet3 where only a few dose profiles showed range errors larger than ±3 mm. The worst results were obtained for Unet1, where for patients 19 and 44 the median values of the range difference distributions for the 270◦ gantry angle were above 3 mm. The results for all patients and both gantry angles are summarized in Figure 11(c), where it can be seen that only for Unet3 the 5th and 95th percentiles are contained within ±3 mm.

95 90 85 80

2%/2mm 3%/3mm

t1

Une

t2

Une

3

t Une

100 95 90 85 80 75 70 65 60

2% 3%

t1

Une

(a)

Une

t2

t3 Une

(b)

Figure 10. PBS dose distribution evaluation for the three Unet trainings. (a) Gamma pass rates for the testing patients presented as box-plots for the 3%/3 mm and 2%/2 mm criteria considering voxels with at least 50% of the prescribed dose. The individual pass rates are additionally plotted as closed symbols. (b) Percent dose difference pass rates for 2% and 3% thresholds. Boxes indicate the median as well as the 25th and 75th percentiles, with whiskers extending to the minimum and maximum. n.s.

n.s.

18

CBCT Unet comparison

range difference/mm

4

G90

Unet1 Unet2 Unet3

2 0 2 n.s. 4

Pat1

Pat9

9

Pat1

6

Pat2

2

Pat3

2 Pat3

Pat3

Pat3

8

Pat4

4

Pat5

1

8

4 Pat4

Pat5

(a)

range difference/mm

4

G270

Unet1 Unet2 Unet3

2 0

n.s.

2 4

Pat1

Pat9

9 Pat1

6 Pat2

1

(b)

range difference/mm

4 2 0 2 4

t1 Une

2

t Une

t3 Une

(c) Figure 11. Proton range evaluation for the results of the three Unet trainings. (a),(b) box-plots presenting the range difference per patient for the (a) 90◦ gantry angle (G90) and (b) 270◦ gantry angle (G270). (c) Overlaid box- and violin-plots presenting the range difference distributions for each Unet training over all patients and beam angles. The violin-plot presents the density distribution. (a)-(c) The boxes indicate the median as well as the 25th and 75th percentiles. The whiskers extent to the 5th and 95th percentiles.

CBCT Unet comparison

19

4. Discussion 285

290

295

300

305

310

315

320

The general feasibility of utilizing a dedicated Unet architecture for CBCT intensity correction using either projection or image based training, as previously demonstrated in (Maier et al. 2018a, Maier et al. 2018c, Hansen et al. 2018) and (Kida et al. 2018), was confirmed in this study. Our results, however, demonstrate that it was feasible to train the same Unet architecture on both CBCT projections and CBCT images. This supports our hypothesis that during training the Unet may learn which network components are relevant for either tasks, possibly through the use of a residual architecture. For the training patient cohort, projection based training yielded less biased results than image based training (see Figure 7). This bias was, however, limited to about 5 HU (Figures 7(c),(d)), and thus clearly lower than the ME observed for the testing set with the three Unets, which did not exhibit any bias. Unet3mixup showed a similar bias (not shown for brevity), thus we do not expect that data augmentation was the cause of the slight training data bias observed for Unet2 and Unet3 when compared to Unet1. While Unet1 and Unet3 yielded corrected CBCT images of similar appearance, Unet2 was noticeably smoother (see Figure 6). The appearance of CBCTUnet2 was similar to the improved CBCT of (Kida et al. 2018) (see Figure 4 in that publication, i-CBCT). We attribute this to (i) much lower noise levels of the vCT compared to the CBCTcor (see Figure 2) and (ii) residual DIR errors leading to slight mis-alignments between CBCTorg and the vCT. Thus CBCTUnet2 has higher MAE than the results obtained with the other Unets (see Table 1) when comparing to the noisy CBCTcor as reference. This is also due to the fact that for the results of Unet1 and Unet3, the noise is correlated during training since CBCTcor is obtained from the same projections as CBCTorg , where a smooth correction for low frequency mismatches was subtracted (Park et al. 2015, Kurz et al. 2016, Z¨ollner et al. 2017). The CBCT images used in this study were acquired with a lower dose than those from (Hansen et al. 2018). Our use of the 20×20 protocol ensured that the CBCT images did not suffer from detector saturation artifacts. These typically entail an erosion of the patient outline, since short arc lengths through thin sections of the patient lead to a fluence saturating the detector pixels corresponding to that ray line. This may contribute to the better performance of Unet1 (see Figure 7(b)) when compared to (Hansen et al. 2018) where ME of up to ± 40 HU were observed. The use of this protocol however caused higher noise levels in the CBCTorg and CBCTcor images used in this study compared to (Hansen et al. 2018). The 20×20 protocol also had lower and a narrower range of ME (7 to 54 HU, see Table 1) compared to (Hansen et al. 2018) (100 HU to 200 HU), potentially reducing the magnitude of the required corrections. Another reason for the improved performance of Unet1 might have been the use of a slightly larger projection training set size (9472 vs. 7323). The VMAT dose calculation accuracy reported in this work was very good for the three Unets we considered, with all test patients exhibiting 1% dose difference pass rates better than 98.4%. This result is an improvement from (Hansen et al. 2018) which

CBCT Unet comparison 325

330

335

340

345

350

355

360

365

20

had 1% dose difference pass rates as low as 47%, and lower than 90% for 3 out 8 test patients. We again attribute this to the use of the 20×20 protocol. Thus all three Unet training approaches investigated in this work would be adequate for CBCT correction in the context of VMAT with this protocol. The dose calculation accuracy results for the PBS SFUD plans were generally lower than those for VMAT, owing to the more sensitive nature of proton dose calculations to CT number errors. Due to the SFUD nature of our plans, dose differences were confined to the distal edges of the PTV for each field, where proton range shifts caused considerable differences at the steep dose gradients (see Figure 9). In the PTV, where SFUD optimization yields a homogeneous dose for each field, the dose distribution was not perturbed. The use of SFUD optimization led to better 2% dose difference pass rates than in (Hansen et al. 2018) (full IMPT was used), where pass rates as low as 15% were observed, as opposed to 68% for Unet1 in this work. For Unet3 the lowest pass rate was 79%. By using a gamma evaluation, which is more robust to dose shifts, 2%/2mm pass rates for Unet3 were better than 90%. These results agree with the more robust nature of SFUD PBS treatment plan optimization compared to full IMPT. For Unet1 and Unet2, pass rates were overall smaller, which could not be anticipated from the minor differences in terms of mean HU error. This is probably due to the fact that deviations in intensity between CBCTcor and the CBCTs obtained from the Unets are not spatially equally distributed. Thus, when analyzing the HU errors within the whole body contour, over- and underestimation of HU values might cancel out, but still impact the results in terms of dose calculation accuracy, which is only being performed on a sub-part of the image. For proton dose calculations based on x-ray CT images, a range uncertainty of 3% is typically assumed (Yang et al. 2012). For our prostate cases, with ranges of approximately 20 cm to the distal edge of the PTV, the range uncertainty would thus be 6 mm. Hence, a threshold of 3 mm defining acceptable proton range accuracy was used in this work. An additional 3 mm uncertainty would yield a cumulative uncertainty of 6.7 mm, which is well below the range shifts expected from gross anatomical changes. Using this 3 mm threshold for range shifts, we can thus state that Unet3 provides sufficiently accurate PBS dose calculation when using SFUD optimization, since the 5th and 95th range difference percentiles were within 3 mm (see Figure 11(c)). This could not be said of the results of Unet1 and Unet2, which had 95th percentiles of 4.2 mm and 3.2 mm respectively. The evaluations performed in this study rely on CBCTcor as reference image. The accuracy of CBCTcor for proton dose calculation has been previously investigated and found sufficient for proton therapy (Park et al. 2015, Kurz et al. 2016). Since Unet3 was trained using CBCTcor as target data, it might have been anticipated that it yielded images which have the highest dosimetric agreement to it. Any residual error in CBCTcor will also likely be present in the results of Unet3, and potentially bias the evaluations of the results of Unet1 and Unet2. However, it is interesting to observe that the training process was able to mitigate some of the limitations of CBCTcor stemming

21

CBCT Unet comparison

(a)

(b)

(c)

(d)

(e)

(f)

Figure 12. Example case (patient 9) of a bladder image slice where DIR failed to produce an accurate vCT. (a) CBCTorg , (b) vCT, (c) CBCTcor , (d) CBCTUnet1 , (e) CBCTUnet2 and (f) CBCTUnet3 .

370

375

380

385

from imperfect vCT generation. In Figure 12 we can observe a case where DIR failed to generate a correct bladder on the vCT. The intensity difference between the bladder and the neighboring fatty tissue is partially transferred to the CBCTcor , where it appears as shadows in the bladder. These shadows appear much reduced in CBCTUnet3 , and somewhat reduced in CBCTUnet1/2 . The Unet3 training approach employed in this paper may also be applicable to training data generated with alternative CBCT correction methods. One could foresee the use of MC based scatter correction of CBCT images such as in (Thing et al. 2016). Finally, the time required to correct either the raw projections or images was comparable for Unet1/2/3. Scans with more projections would take longer to correct with Unet1, but a doubling of the number of projections would still yield correction times lower than 10 s. Furthermore, it may be possible to correct the projections as they are acquired, avoiding the correction overhead and effectively making Unet1 faster (reconstruction only) than Unet2/3 (reconstruction + image correction) in terms of time required to yield the final image. All three approaches would allow image correction in an online adaptive radiotherapy workflow or in a dose-guided positioning approach used in (Hofmaier et al. 2017), which relied on CBCTcor . Such a workflow would rely on four main components which should be as fast as possible: CBCT image correction, delineation, dose calculation and optimization.

CBCT Unet comparison

22

5. Conclusion

390

395

A given Unet architecture was successfully trained using both CBCT projections and CBCT image slices. Whether using pairs of raw and corrected projections, raw CBCT image slices and vCT slices or raw CBCT image slices and CBCTcor image slices, VMAT dose calculation accuracy was excellent, with 1% dose difference pass rates better than 98.4% for the 8 test patients used in this study. Unet3, trained using CBCTcor image slices as target data, additionally achieved 5th and 95th percentiles of range differences within ±3 mm for all test patients and beam angles when compared to the CBCTcor . This translated to a PBS dose calculation accuracy using SFUD optimization with 2%/2mm gamma pass rates better than 90%. Since the results of the other Unets were slightly poorer, we conclude that training using CBCTcor image slices as target data was optimal for PBS SFUD proton dose calculations on this dataset, while for VMAT all Unets provide excellent accuracy. Acknowledgements

400

405

410

415

420

This work was sponsored by the German Research Foundation (DFG) Cluster of Excellence Munich Center for Advanced Photonics (MAP). Christopher Kurz was supported by the German Cancer Aid (Deutsche Krebshilfe). One of the GPUs used for this research was donated by the NVIDIA Corporation. We would like to thank Nadine Spahr, Christoph Brachmann and Florian Weiler for providing the CT-to-CBCT DIR used in this study. We thank Erik Traneus and colleagues from Raysearch Laboratories, as well as Marco Pinto, for support on the research version of the TPS used in this study. We thank Jan Hofmaier for his help on implementing the reference CBCT shading correction method. There are no conflicts of interest. References Ding, G. X., Duggan, D. M., Coffey, C. W., Deeley, M., Hallahan, D. E., Cmelak, A. & Malcolm, A. (2007). A study on adaptive IMRT treatment planning using kV cone-beam CT, Radiotherapy and Oncology 85(1): 116–125. Fotina, I., Hopfgartner, J., Stock, M., Steininger, T., L¨ utgendorf-Caucig, C. & Georg, D. (2012). Feasibility of CBCT-based dose calculation: comparative analysis of HU adjustment techniques, Radiotherapy and Oncology 104(2): 249–256. Han, X. (2017). MR-based synthetic CT generation using a deep convolutional neural network method, Medical Physics 44(4): 1408–1419. Hansen, D., Landry, G., Kamp, F., Li, M., Belka, C., Parodi, K. & Kurz, C. (2018). ScatterNet: a convolutional neural network for cone-beam CT intensity correction, Medical Physics in press. Hofmaier, J., Haehnle, J., Kurz, C., Landry, G., Maihoefer, C., Sch¨ uttrumpf, L., S¨ uss, P., Teichert, K., S¨ ohn, M., Spahr, N., Brachmann, C., Weiler, F., Thieke, C., K¨ ufer, K.-H., Belka, C., Parodi, K. & Kamp, F. (2017). Multi-criterial patient positioning based on dose recalculation on scattercorrected CBCT images, Radiotherapy and Oncology 125(3): 464 – 469.

CBCT Unet comparison 425

430

435

440

445

450

455

460

465

470

475

23

Hua, C., Yao, W., Kidani, T., Tomida, K., Ozawa, S., Nishimura, T., Fujisawa, T., Shinagawa, R. & Merchant, T. E. (2017). A robotic C-arm cone beam CT system for image-guided proton therapy: design and performance, The British Journal of Radiology 90(1079): 20170266. Kida, S., Nakamoto, T., Nakano, M., Nawa, K., Haga, A., Kotoku, J., Yamashita, H. & Nakagawa, K. (2018). Cone Beam Computed Tomography Image Quality Improvement Using a Deep Convolutional Neural Network, Cureus 10(4). Kingma, D. P. & Ba, J. (2015). Adam: A Method for Stochastic Optimization, International Conference on Learning Representations (ICLR). Kurz, C., Dedes, G., Resch, A., Reiner, M., Ganswindt, U., Nijhuis, R., Thieke, C., Belka, C., Parodi, K. & Landry, G. (2015). Comparing cone-beam CT intensity correction methods for dose recalculation in adaptive intensity-modulated photon and proton therapy for head and neck cancer, Acta Oncologica 54(9): 1651–1657. PMID: 26198654. Kurz, C., Kamp, F., Park, Y., Z¨ ollner, C., Rit, S., Hansen, D., Podesta, M., Sharp, G. C., Li, M., Reiner, M., Hofmaier, J., Neppl, S., Thieke, C., Nijhuis, R., Ganswindt, U., Belka, C., Winey, B. A., Parodi, K. & Landry, G. (2016). Investigating deformable image registration and scatter correction for CBCT-based dose calculation in adaptive IMPT, Medical Physics 43(10): 5635– 5646. Landry, G., Dedes, G., Z¨ ollner, C., Handrack, J., Janssens, G., de Xivry, J. O., Reiner, M., Paganelli, C., Riboldi, M., Kamp, F. et al. (2014). Phantom based evaluation of CT to CBCT image registration for proton therapy dose recalculation, Physics in Medicine & Biology 60(2): 595. Landry, G. & Hua, C.-h. (2018). Current state and future applications of radiological image guidance for particle therapy, Medical physics 45(11): e1086–e1095. Landry, G., Nijhuis, R., Dedes, G., Handrack, J., Thieke, C., Janssens, G., Orban de Xivry, J., Reiner, M., Kamp, F., Wilkens, J. J. et al. (2015). Investigating CT to CBCT image registration for head and neck proton therapy as a tool for daily dose recalculation, Medical physics 42(3): 1354– 1366. Maier, J., Berker, Y., Sawall, S. & Kachelrieß, M. (2018a). Deep scatter estimation (DSE): feasibility of using a deep convolutional neural network for real-time x-ray scatter prediction in cone-beam CT, Medical Imaging 2018: Physics of Medical Imaging, Vol. 10573, International Society for Optics and Photonics, p. 105731L. Maier, J., Eulig, E., V¨ oth, T., Knaup, M., Kuntz, J., Sawall, S. & Kachelrieß, M. (2018b). RealTime Scatter Estimation for Medical CT using the Deep Scatter Estimation (DSE): Method and Robustness Analysis with Respect to Different Anatomies, Dose Levels, Tube Voltages and Data Truncation, Medical Physics . Maier, J., Sawall, S., Knaup, M. & Kachelrieß, M. (2018c). Deep Scatter Estimation (DSE): Accurate Real-Time Scatter Estimation for X-Ray CT Using a Deep Convolutional Neural Network, Journal of Nondestructive Evaluation 37(3): 57. Mainegra-Hing, E. & Kawrakow, I. (2010). Variance reduction techniques for fast Monte Carlo CBCT scatter correction calculations, Physics in Medicine & Biology 55(16): 4495. Marks, L. B., Yorke, E. D., Jackson, A., Ten Haken, R. K., Constine, L. S., Eisbruch, A., Bentzen, S. M., Nam, J. & Deasy, J. O. (2010). Use of normal tissue complication probability models in the clinic, International Journal of Radiation Oncology* Biology* Physics 76(3): S10–S19. Niu, T., Al-Basheer, A. & Zhu, L. (2012). Quantitative cone-beam CT imaging in radiation therapy using planning CT as a prior: First patient studies, Medical Physics 39(4): 1991–2000. Niu, T., Sun, M., Star-Lack, J., Gao, H., Fan, Q. & Zhu, L. (2010). Shading correction for on-board cone-beam CT in radiation therapy using planning MDCT images, Medical Physics 37(10). Park, Y.-K., Sharp, G. C., Phillips, J. & Winey, B. A. (2015). Proton dose calculation on scattercorrected CBCT image: Feasibility study for adaptive proton therapy, Medical Physics 42(8). Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L. & Lerer, A. (2017). Automatic differentiation in pytorch, NIPS 2017 Workshop. Peroni, M., Ciardo, D., Spadea, M. F., Riboldi, M., Comi, S., Alterio, D., Baroni, G. & Orecchia,

CBCT Unet comparison

480

485

490

495

500

505

510

515

520

24

R. (2012). Automatic segmentation and online virtualCT in head-and-neck adaptive radiation therapy, International Journal of Radiation Oncology* Biology* Physics 84(3): e427–e433. Poludniowski, G., Evans, P., Kavanagh, A. & Webb, S. (2011). Removal and effects of scatter-glare in cone-beam CT with an amorphous-silicon flat-panel detector, Physics in Medicine & Biology 56(6): 1837. Rit, S., Oliva, M. V., Brousmiche, S., Labarbe, R., Sarrut, D. & Sharp, G. C. (2014). The Reconstruction Toolkit (RTK), an open-source cone-beam CT reconstruction toolkit based on the Insight Toolkit (ITK), Journal of Physics: Conference Series 489(1): 012079. Ronneberger, O., Fischer, P. & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation, International Conference on Medical image computing and computer-assisted intervention, Springer, pp. 234–241. Siewerdsen, J. H. & Jaffray, D. A. (2001). Cone-beam computed tomography with a flat-panel imager: Magnitude and effects of x-ray scatter, Medical Physics 28(2). Stock, M., Georg, D., Ableitinger, A., Zechner, A., Utz, A., Mumot, M., Kragl, G., Hopfgartner, J., Gora, J., B¨ ohlen, T. et al. (2017). The technological basis for adaptive ion beam therapy at MedAustron: Status and outlook, Zeitschrift f¨ ur Medizinische Physik . Thing, R. S., Bernchou, U., Mainegra-Hing, E., Hansen, O. & Brink, C. (2016). Hounsfield unit recovery in clinical cone beam CT images of the thorax acquired for image guided radiation therapy, Physics in Medicine & Biology 61(15): 5781. Veiga, C., Alshaikhi, J., Amos, R., Louren¸co, A. M., Modat, M., Ourselin, S., Royle, G. & McClelland, J. R. (2015). Cone-Beam Computed Tomography and Deformable Registration-Based “Dose of the Day” Calculations for Adaptive Proton Therapy, International Journal of Particle Therapy 2(2): 404–414. Veiga, C., Janssens, G., Teng, C.-L., Baudier, T., Hotoiu, L., McClelland, J. R., Royle, G., Lin, L., Yin, L., Metz, J., Solberg, T. D., Tochner, Z., Simone, C. B., McDonough, J. & Teo, B.-K. K. (2016). First clinical investigation of CBCT and deformable registration for adaptive proton therapy of lung cancer, International Journal of Radiation Oncology*Biology*Physics pp. –. Veiga, C., McClelland, J., Moinuddin, S., Louren¸co, A., Ricketts, K., Annkah, J., Modat, M., Ourselin, S., D’Souza, D. & Royle, G. (2014). Toward adaptive radiotherapy for head and neck patients: Feasibility study on using CT-to-CBCT deformable registration for “dose of the day” calculations, Medical Physics 41(3): 1–12. Wang, P., Yin, L., Zhang, Y., Kirk, M., Song, G., Ahn, P. H., Lin, A., Gee, J., Dolney, D., Solberg, T. D. et al. (2016). Quantitative assessme nt of anatomical change using a virtual proton depth radiograph for adaptive head and neck proton therapy, Journal of applied clinical medical physics 17(2): 427–440. Wolterink, J. M., Leiner, T., Viergever, M. A. & Iˇsgum, I. (2017). Generative Adversarial Networks for Noise Reduction in Low-Dose CT, IEEE Transactions on Medical Imaging 36(12): 2536–2545. Yang, M., Zhu, X. R., Park, P. C., Titt, U., Mohan, R., Virshup, G., Clayton, J. E. & Dong, L. (2012). Comprehensive analysis of proton range uncertainties related to patient stopping-power-ratio estimation using the stoichiometric calibration, Physics in Medicine & Biology 57(13): 4095. Zhang, H., Cisse, M., Dauphin, Y. N. & Lopez-Paz, D. (2018). mixup: Beyond Empirical Risk Minimization, International Conference on Learning Representations International Conference on Learning Representations (ICLR). Z¨ ollner, C., Rit, S., Kurz, C., Vilches-Freixas, G., Kamp, F., Dedes, G., Belka, C., Parodi, K. & Landry, G. (2017). Decomposing a prior-CT-based cone-beam CT projection correction algorithm into scatter and beam hardening components, Physics and Imaging in Radiation Oncology 3: 49 – 52.