MICCAI/ EMPIRE10 Lung Registration Challenge 3D Slicer Non-rigid Registration

MICCAI/ EMPIRE10 Lung Registration Challenge 3D Slicer Non-rigid Registration Dominik S. Meier 1,, Lidwien Veugen 2, Raul S Jose 1, James V. Miller 3, Ron Kikinis 1 1 Brigham & Women's Hospital, Dept. of Radiology, Boston, MA; U.S.A. 2 Eindhoven University of Technology, Dept. of Biomedical Eng., Eindhoven, Netherlands 3 Visualization and Computer Vision Laboratory, GE Research, Niskayuna, NY, U.S.A. Abstract. Combinations of three different registration modules within the 3DSlicer software were applied, seeking a fast and fully automated registration without any preprocessing or the use of masks. Ease of use and speed were key criteria for the choice of methods, i.e. desired was a pipeline with less than one hour total processing time and no preprocessing requirements. Employed were Affine, BSpline and DemonWarp algorithms. BSpline registration consisted of three cycles with increasing degrees of freedom (DOF) and cubic grids of 7,12 and 17 grid-points, respectively. A second pipeline invoking the BRAINSDemonWarp routine on the result of the first method was also applied, to test if high-dof warping could improve the results. Average lung boundary error achieved was 0.03% (rank 19) by the BSpline method and 0.05% (rank 26) by the DemonWarp. The use of the DemonWarp incurred significant singularity penalties. Achievable result accuracy was limited by the necessary fusion of multiple deformation fields into a single solution, which necessitates roundoff errors at each stage and prevents subvoxel accuracy. All images were processed in a fully automated fashion and with a single set of parameters, hence results are not representative for using the 3DSlicer registration modules in a semi-automated fashion, with interactive tuning of parameters and the use of masks. Keywords: 3DSlicer, BSpline, Demons, BRAINSDemonWarp Introduction The open source software 3DSlicer (www.slicer.org) [1] contains a large set of registration algorithms, including automated and interactive, intensity-, surface- or landmark-based. Three different registration tools were used and their result transforms combined into two pipelines. All three registration modules are based on registration transforms, similarity metrics and optimizers implemented from the Insight Toolkit (ITK) [2]. The rationale for the methods used in this registration challenge may differ somewhat from other algorithm compared in this challenge: we sought to test the feasibility of applying the 3DSlicer registration modules from a clinician researcher perspective. Hence the method of choice needed to be fully automated with little or no

preprocessing. Ease of use and overall processing time were important considerations along with achievable accuracy and robustness. As a benchmark we sought overall processing time of less than 1 hour. Two pipelines were tested: the first employed a stepwise approach of four consecutive registrations with increasing degrees of freedom (DOF). The second method applied an additional high-dof warping algorithm based on optical flow concepts, starting from the position obtained by method 1. Both methods were fully automated, ran all images with a single set of settings and did not use any masks. Neither the similarity metric, nor the optimizer were tailored in any form to the geometry or contrast or image content encountered in the images. Methods Pipeline The overall pipeline of the two methods is shown in Figure 1. Affine (12 DOF) BSpline1 (~10 2 DOF) BSpline2 (~10 3 DOF) BSpline3 (~10 4 DOF) DemonWarp (~10 5 DOF) Method 1 Method 2 Figure 1: Overview of two registration pipelines tested within the 3DSlicer software package. Method 1 comprised initialization with an affine transform, followed by 3 consecutive nonrigid BSpline warps of increasing DOF. The second method was a high-dof warping based on the Demons algorithm, starting from the output of method 1. Fast Affine Initial alignment was obtained with the "Fast Affine Registration" module in 3DSlicer (www.slicer.org) based on ITK algorithm implemented from [3]. Input images were the original fixed and moving images, as provided, without any preprocessing or masking. Mutual information was the cost function criterion for all registrations. Parameters used are listed in Table 1. Result of this step was a 12 DOF linear affine transform, used as starting pose for the subsequent BSpline registration. Fast Nonrigid BSpline The "Fast Nonrigid BSpline" module in 3DSlicer is built on the ITK BSpline algorithm, based in turn on [4]. Details on the algorithm can be found in the ITK Users Guide [2]. The method places a 3D grid of evenly spaced control points over the image, with three translational DOF per point. Cost function was mutual information; no masking or intensity normalizations were applied. Input images were the original images, as provided, along with the affine transform result from step one

as starting pose. Result of each BSpline registration was a resampled/warped moving image and a vector deformation field, saved as a 4D volume. Three consecutive rounds of BSpline registrations were applied, using the registered image of the previous as moving input for the next. The parameters used at each iteration are listed in Table 1. Step samples iterations grid size DOF hist.bin initial pose 1.Affine 150,000 2000-12 30-2.BSpline 1 150,000 30 7 x 7 x 7 1,029 50 Affine 3.BSpline 2 150,000 30 12 x 12 x 12 5,184 50 BSpline 1 4.BSpline 3 250,000 30 17 x 17 x 17 14,739 50 BSpline 2 Table 1: Parameters used for the Affine and BSpline registration steps. samples: the number of sample points randomly selected that actively contribute to the cost function. For the large 3D volumes of this study the sample points correspond to less than 1% of the total image matrix; iterations: max. iteration limit; grid size: number of control grid points, DOF: total degree of freedom; hist.bin: number of histogram bins used in calculating the mutual information cost function; initial pose: starting/input transform for each cycle. BRAINSDemonWarp The BRAINSDemonWarp algorithm [5] is a high-dof nonrigid registration method, based on the Demons concept, which uses an optical flow paradigm to drive and constrain the deformation. Inputs tare the image pair to be registered and control parameters, such as multi-resolution levels, smoothing kernels, global histogram matching etc. Output was analogous to the BSpline modules, consisting of a deformation field and resampled output image. The optimizer uses the MultiResolutionPDEDeformableRegistration filter with nearest-neighbor extra- and intrapolation. Each vector in the deformation field represents the distance between a geometric point in the input space and a point in the output space. The resampled output image is generated by warping the input moving image with the deformation field using the ItkWarpImageFilter [2]. Image values at mapped non-integer positions are interpolated from the grid-positions via the ITK function LinearInterpolateImage for this application. The program uses the Insight Toolkit [2] for all the computations, For a detailed description of the Slicer implementation of BRAINSDemonWarp, see the reference section in [1]. DemonWarp employs a multi-resolution scheme with variable pyramid levels. Used here were three pyramid levels with 1,, scales, respectively, with 10, 15, 20 iterations at each level and a Gaussian smoothing of the deformation field at each iteration with a sigma of 0.3. A symmetrical gradient type was used to compute the demons forces. Deformation Field Resampling/Concatenation Each separate instance of a BSpline or DemonsWarp registration produced a backward mapping deformation field. To produce the final deformation field the

individual fields are first resampled in reverse order and then summed. Resampling occurred with the 3DSlicer resampling tool "ResampleScalar/Vector/DWI" module, which is based on an ITK resampling algorithm [6]. Detailed documentation on the resampling module can be found on the reference pages at www.slicer.org [1]. Hardware / OS All registrations were run on DELL compute servers with quad Xeon 2.0GHz processors, running a UNIX operating system, with 8GB of RAM each and 36GB SCSI hard drives. Results Processing Time / Speed The affine portion completed on average in 1.2±1.1 minutes. The three BSpline iterations all completed in about 11±4 minutes each. The DemonWarp completed in 21±9 minutes on average. Detailed processing times for each of the twenty image pairs are shown in Figure 2 for BSpline and Figure 3 for the DemonWarp. Total processing time for the BSpline method averaged 37±14 minutes (range 7-51 minutes). Total processing time for the DemonWarp method was 58±22 minutes on average (range 10-88 minutes). Distributions for all five elements are compared in Figure 4. 20 15 Affine BSpline 1 Bspline 2 BSpline 3 exam 10 5 0 0 10 20 30 40 50 60 processing time [min] Figure 2: Summary processing times for method 1 (BSpline). Median processing time was less than one minute for the Affine, 13 minutes for each of the BSplines.

20 15 exam 10 5 Affine+BSpline DemonWarp 0 0 10 20 30 40 50 60 70 80 90 processing time [min] Figure 3: Processing time for method 2 (DemonWarp). Average processing time was 23 minutes per case for the DemonWarp. Since the initial pose was the output of method 1, the total processing time on average was 58 minutes. processing time [min] 40 35 30 25 20 15 10 5 0 Affine BSpline 1 Bspline 2 BSpline 3DemonWarp Figure 4: Summary processing times for all 5 elements. Median processing time was less than one minute for the Affine, 13 minutes for each of the BSplines, and 23 minutes for the DemonWarp. Average processing time per case for methods 1 & 2 thus was 37 and 58 minutes, respectively. Accuracy Registration accuracy was evaluated based on alignment of lung boundaries, fissures and expert landmarks and also on the presence of singularities in the final deformation field. Details on the evaluation method can be found in [7]. Summary scores and ranks for the BSpline and DemonWarp methods can be found in Table 2 and Table 3, respectively.

Lung Boundaries Fissures Landmarks Singularities Scan Pair Score Rank Score Rank Score Rank Score Rank 01 0.00 12.00 9.55 25.00 9.13 24.00 0.00 11.50 02 0.00 11.00 0.00 15.00 0.56 20.00 0.00 12.50 03 0.00 17.00 0.00 25.00 0.53 18.00 0.00 12.00 04 0.01 30.00 0.00 16.50 2.40 25.00 0.00 14.00 05 0.00 13.00 0.00 16.00 0.00 5.50 0.00 13.50 06 0.00 16.00 0.00 27.00 0.41 20.00 0.00 14.00 07 0.05 21.00 2.65 23.00 5.87 24.00 0.00 10.00 08 0.03 23.00 5.99 32.00 5.00 30.00 0.00 25.00 09 0.00 25.00 0.04 27.00 0.91 26.00 0.00 13.00 10 0.17 32.00 0.00 15.00 4.47 24.00 0.00 13.00 11 0.24 25.00 4.14 30.00 2.11 24.00 0.00 11.50 12 0.00 25.00 0.00 13.50 0.08 17.00 0.00 14.50 13 0.00 14.00 0.15 28.00 1.20 21.00 0.00 13.00 14 0.06 18.00 7.74 25.00 8.29 24.00 0.00 9.50 15 0.00 26.00 0.00 22.00 0.84 25.00 0.00 12.50 16 0.00 28.00 0.09 19.00 1.07 13.00 0.00 13.50 17 0.00 13.00 0.06 25.50 1.13 20.00 0.00 14.00 18 0.04 17.00 4.05 20.00 5.97 24.00 0.00 10.50 19 0.00 14.00 0.00 12.00 0.63 23.00 0.00 14.50 20 0.05 20.00 6.62 24.00 9.35 24.00 0.00 23.00 Avg 0.03 20.00 2.05 22.02 3.00 21.57 0.00 13.77 Average Ranking Overall 19.34 Final Placement 25 Table 2: Results for the BSpline method for each scan pair, per category and overall. Rankings and final placement are from a total of 34 competing algorithms. Lung Boundaries Fissures Landmarks Singularities Scan Pair Score Rank Score Rank Score Rank Score Rank 01 0.00 11.00 11.78 26.00 10.31 26.00 7.42 33.00 02 0.00 11.00 0.00 30.00 0.42 11.00 0.16 32.00 03 0.00 12.00 0.24 32.00 0.65 22.00 3.68 34.00 04 0.00 25.00 0.11 34.00 5.37 32.00 2.43 33.00 05 0.00 13.00 0.00 16.00 0.00 5.50 0.01 32.00 06 0.01 34.00 0.01 32.00 1.08 33.00 0.23 34.00 07 0.25 23.00 3.30 24.00 6.24 26.00 5.96 34.00 08 0.05 26.00 4.86 29.00 5.34 31.00 3.11 34.00 09 0.00 21.00 0.05 29.00 0.81 25.00 1.41 34.00 10 0.02 25.00 0.18 33.00 7.76 31.00 4.29 33.00 11 0.17 22.00 2.45 28.00 1.92 21.00 2.90 34.00 12 0.00 26.00 0.00 13.50 0.00 5.00 0.00 31.00 13 0.00 18.00 0.73 33.00 1.56 32.00 1.54 34.00 14 0.30 24.00 5.41 19.00 8.08 22.00 8.41 34.00 15 0.00 31.00 0.09 31.00 0.76 20.00 1.01 32.00 16 0.00 29.00 6.05 32.00 4.43 33.00 6.23 33.00 17 0.00 19.00 1.26 33.00 1.46 29.00 0.79 34.00 18 0.12 22.00 5.54 25.00 5.75 23.00 6.33 34.00 19 0.00 14.00 0.01 31.00 0.58 19.00 0.12 32.00 20 0.06 21.00 4.69 21.00 10.17 26.00 8.11 33.00 Avg 0.05 21.35 2.34 27.57 3.64 23.62 3.21 33.20 Average Ranking Overall 26.43 Final Placement 32 Table 3: Results for the DemonWarp method, for each scan pair, per category and overall. Rankings and final placement are from a total of 34 competing algorithms.

4 Discussion Accuracy Registration accuracy was modest compared to other more tailored methods participating in the EMPIRE10 challenge. Judging overall performance however should take the potential preprocessing effort into account: we decided to deliberately not use the masks and restrict total processing time to less than 90 minutes. The results achieved should therefore be interpreted in the context of a fast & fully automated registration without prior editing, i.e. comparability with methods that did use the masks is limited. In a clinical setting the creation of high-precision masks would incur a significant and possibly prohibitive time and resource effort. Hence speed and ease of use were relevant criteria, followed by robustness and accuracy. An additional source of error came from roundoff errors incurred when obtaining the backward mapping deformation fields requested for the competition. The concatenation of nonrigid deformation fields includes a round-off to grid-coordinates at each step, which ultimately limits the final accuracy. Considering that most methods obtained accuracies within a few voxels, we assume that significantly better accuracy can be obtained from the algorithms tested if this resampling step can be avoided. The DemonWarp method is an optical flow algorithm based on the demons principle. Goal was to see if this method as an additional high-dof registration step could further improve the results. While the method did in some cases reduce residual misalignment, it incurred large penalties in others and consistently produced singularities in the final deformation field. In 4 of the 20 cases lung boundary- and landmar-error were significantly increased by the added step, which drastically reduced the overall mean scores. So at least in some cases the DemonWarp appears to have moved away from the correct result. This may also in part also be due to the cumulative roundoff error introduced by the abovementioned concatenation of deformation fields, which for the DemonWarp addition added a fourth deformation to the series and associated increase in total error. In conclusion the combination of BSpline and DemonWarp pipelines in the configuration applied is not the method of choice, particularly in the light of the computational cost (see below). Particularly based on the weighting of singularities the DemonWarp addition to the BSpline pipeline did not improve the results. Further improvement of the alignment beyond that achieved by the finer BSpline grid is likely to require masking to avoid the cost function being dominated by non-lung parenchyma. Processing Time / Speed Premise for the pipeline used was a targeted total processing time of 1 hour. The BSpline registration methods all completed on an average in 37 minutes, and the DemonWarp averaged 58 minutes. However there were several cases in which the DemonWarp exceeded the 1 hour range, with a maximum of 88 minutes for case #2. The images provided in this challenge were very large and deformation fields generated from them could exceed 1GB in size.

The multi-step approach with gradual increases from 12 affine DOF to the BSpline grids of increasing size proved very robust and did not fail in a single case in fully automated processing. However the feasibility of concatenating transforms depends critically on the desired result. Without a specific tool to concatenate the transforms in a forward manner, inevitable interpolation and round-off errors accumulate and can prevent achieving sub-voxel accuracy. If the desired output is the registered image, and some blurring from interpolation is acceptable, then this method is a viable candidate for a robust fast and fully automated approach. If the deformation field is required, then a method for linking the transforms in a manner that minimizes error accumulation is necessary to achieve sub-voxel accuracy. Acknowledgments This work was funded in part by NIH/NCRR P41 RR013218-12S1 and NIH/NCRR P41 RR013218. Special thanks go to Drs. Hans Johnson and Yongqiang Zhao for the implementation of the 3DSlicer BRAINSDemonWarp method, and also to Bill Lorensen and Yinglin Lee for help with the 3DSlicer BSpline module implementation. References 1. 3DSlicer v3.6; (NA-MIC, NAC); 2010; http://www.slicer.org 2. Ibanez, L., Schroeder, W., Ng, L., Cates, J.: The Insight Toolkit (itk) Software Guide. (2005) http://www.itk.org/itksoftwareguide.pdf 3. Mattes, D., Haynor, D.R., Vesselle, H., Lewellen, T.K., Eubank, W.: PET-CT image registration in the chest using free-form deformations. IEEE Trans Med Imaging 22 (2003) 120-128 4. Rueckert, D., Hayes, C., Studholme, C., Summers, P., Leach, M., Hawkes, D.: Non-rigid registration of breast MR images using mutual information. Lecture Notes in Computer Science 1496 (1998) 1144-1152 5. Johnson, H., Zhao, Y.: BRAINSDemonWarp: An Applicaton to Perform Demons Registration. (2009) http://www.insightjournal.org/browse/publication/312 6. Ibanez, L., Schroeder, W., Ng, L., Cates, J.: The Insight Toolkit (itk) Software Guide - sections 6.9, 8.8, 8.9. (2005) http://www.itk.org/itksoftwareguide.pdf 7. Murphy, K., van Ginneken, B., Reinhardt, J., Kabus, S., Ding, K., Deng, X., Pluim, J.: Evaluation of Methods for Pulmonary Image Registration: The EMPIRE10 Study. Grand Challenges in Medical Image Analysis (2010)