Evaluation of atlas selection strategies for atlas-based image segmentation with application to confocal microscopy images of bee brains

This paper evaluates strategies for atlas selection in atlas-based segmentation of three-dimensional biomedical images. Segmentation by intensity-based nonrigid registration to atlas images is applied to confocal microscopy images acquired from the brains of 20 bees. This paper evaluates and compares four different approaches for atlas image selection: registration to an individual atlas image (IND), registration to an average-shape atlas image (AVG), registration to the most similar image from a database of individual atlas images (SIM), and registration to all images from a database of individual atlas images with subsequent multi-classifier decision fusion (MUL). The MUL strategy is a novel application of multi-classifier techniques, which are common in pattern recognition, to atlas-based segmentation. For each atlas selection strategy, the segmentation performance of the algorithm was quantified by the similarity index (SI) between the automatic segmentation result and a manually generated gold standard. The best segmentation accuracy was achieved using the MUL paradigm, which resulted in a mean similarity index value between manual and automatic segmentation of 0.86 (AVG, 0.84; SIM, 0.82; IND, 0.81). The superiority of the MUL strategy over the other three methods is statistically significant (two-sided paired t test, P < 0.001). Both the MUL and AVG strategies performed better than the best possible SIM and IND strategies with optimal a posteriori atlas selection (mean similarity index for optimal SIM, 0.83; for optimal IND, 0.81). Our findings show that atlas selection is an important issue in atlas-based segmentation and that, in particular, multi-classifier techniques can substantially increase the segmentation accuracy.     

Estimation of slipping organ motion by registration with direction-dependent regularization

Accurate estimation of respiratory motion is essential for many applications in medical 4D imaging, for example for radiotherapy of thoracic and abdominal tumors. It is usually done by non-linear registration of image scans at different states of the breathing cycle but without further modeling of specific physiological motion properties. In this context, the accurate computation of respiration-driven lung motion is especially challenging because this organ is sliding along the surrounding tissue during the breathing cycle, leading to discontinuities in the motion field. Without considering this property in the registration model, common intensity-based algorithms cause incorrect estimation along the object boundaries.In this paper, we present a model for incorporating slipping motion in image registration. Extending the common diffusion registration by distinguishing between normal- and tangential-directed motion, we are able to estimate slipping motion at the organ boundaries while preventing gaps and ensuring smooth motion fields inside and outside. We further present an algorithm for a fully automatic detection of discontinuities in the motion field, which does not rely on a prior segmentation of the organ.We evaluate the approach for the estimation of lung motion based on 23 inspiration/expiration pairs of thoracic CT images. The results show a visually more plausible motion estimation. Moreover, the target registration error is quantified using manually defined landmarks and a significant improvement over the standard diffusion regularization is shown.     

Tumour-Targeted Treatment Intensification for Prostate Cancer Using Magnetic Resonance Imaging Guidance

IntroductionLocal control in prostate cancer may be improved with targeted dose escalation to regions with high tumour burden. Multiparametric magnetic resonance imaging (mpMRI) combined with MRI-guided biopsies may aid in defining tumour-dense regions before radiation therapy. Deformable registration techniques may be used to map these tumour regions onto the radiotherapy planning MRI. Radiation therapy delivery techniques such as volumetric modulated arc therapy and high-dose-rate brachytherapy may allow for highly conformal dose escalation, and when coupled with image-guided radiation delivery (ie, cone beam computed technology and fiducial markers), may allow high-precision dose-escalated treatment.MethodsEligible prostate cancer patients were enrolled on a prospective trial of tumour dose escalation. Two strategies were investigated: (1) an integrated boost to external beam radiation for a total tumour dose of 95 Gy in 38 fractions or (2) a focal high-dose-rate boost of 10 Gy before 76 Gy in 38 fractions external beam radiation. Patients underwent MRI-guided biopsy with fiducial marker placement before therapy. mpMRI was acquired and used in conjunction with a non-endorectal coil T2 MRI and computed technology simulation images to define the gross tumour volume via a deformable registration approach for intraprostatic tumour dose escalation.ResultsA case example for each dose escalation strategy illustrates the tumour-targeted approach using MRI guidance.ConclusionsCombining mpMRI sequences with a deformable registration approach may aid in more accurate and reproducible definition of tumour-dense regions. This novel process coupled with daily image guidance may allow high-precision dose-escalated tumour-targeted radiotherapy for prostate cancer.     

Constrained non-rigid registration for use in image-guided adaptive radiotherapy

A constrained non-rigid registration (CNRR) algorithm for use in prostate image-guided adaptive radiotherapy is presented in a coherent mathematical framework. The registration algorithm is based on a global rigid transformation combined with a series of local injective non-rigid multi-resolution cubic B-spline Free Form Deformation (FFD) transformations. The control points of the FFD are used to non-rigidly constrain the transformation to the prostate, rectum, and bladder. As well, the control points are used to rigidly constrain the transformation to the estimated position of the pelvis, left femur, and right femur. The algorithm was tested with both 3D conformal radiotherapy (3DCRT) and intensity-modulated radiotherapy (IMRT) dose plan data sets. The 3DCRT dose plan set consisted of 10 fan-beam CT (FBCT) treatment-day images acquired from four different patients. The IMRT dose plan set consisted of 32 cone-beam CT (CBCT) treatment-day images acquired from 4 different patients. The CNRR was tested with different combinations of anatomical constraints and each test significantly outperformed both rigid and non-rigid registration at aligning constrained bones and critical organs. The CNRR results were used to adapt the dose plans to account for patient positioning errors as well as inter-day bone motion and intrinsic organ deformation. Each adapted dose plan improved performance by lowering radiation distribution to the rectum and bladder while increasing or maintaining radiation distribution to the prostate.     

Combining a deformable model and a probabilistic framework for an automatic 3D segmentation of prostate on MRI

Purpose  Accurate localization and contouring of prostate are crucial issues in prostate cancer diagnosis and/or therapies. Although several semi-automatic and automatic segmentation methods have been proposed, manual expert correction remains necessary. We introduce a new method for automatic 3D segmentation of the prostate gland from magnetic resonance imaging (MRI) scans. Methods  A statistical shape model was used as an a priori knowledge, and gray levels distribution was modeled by fitting histogram modes with a Gaussian mixture. Markov fields were used to introduce contextual information regarding voxels’ neighborhoods. Final labeling optimization is based on Bayesian a posteriori classification, estimated with the iterative conditional mode algorithm. Results  We compared the accuracy of this method, free from any manual correction, with contours outlined by an expert radiologist. In 12 cases, including prostates with cancer and benign prostatic hypertrophy, the mean Hausdorff distance and overlap ratio were 9.94 mm and 0.83, respectively. Conclusion  This new automatic prostate MRI segmentation method produces satisfactory results, even at prostate’s base and apex. The method is computationally feasible and efficient.     

A level set framework with a shape and motion prior for segmentation and region tracking in echocardiography

We describe a level set formulation using both shape and motion prior, for both segmentation and region tracking in high frame rate echocardiographic image sequences. The proposed approach uses the following steps: registration of the prior shape, level set segmentation constrained through the registered shape and region tracking. Registration of the prior shape is expressed as a rigid or an affine transform problem, where the transform minimizing a global region-based criterion is sought. This criterion is based on image statistics and on the available estimated axial motion data. The segmentation step is then formulated through front propagation, constrained with the registered shape prior. The same region-based criterion is used both for the registration and the segmentation step. Region tracking is based on the motion field estimated from the interframe level set evolution. The proposed approach is applied to high frame rate echocardiographic sequences acquired in vivo. In this particular application, the prior shape is provided by a medical expert and the rigid transform is used for registration. It is shown that this approach provides consistent results in terms of segmentation and stability through the cardiac cycle. In particular, a comparison indicates that the results provided by our approach are very close to the results obtained with manual tracking performed by an expert cardiologist on a Doppler Tissue Imaging (DTI) study. These preliminary results show the ability of the method to perform region tracking and its potential for dynamic parametric imaging of the heart.     

Segmentation of brain 3D MR images using level sets and dense registration

This paper presents a strategy for the segmentation of brain from volumetric MR images which integrates 3D segmentation and 3D registration processes. The segmentation process is based on the level set formalism. A closed 3D surface propagates towards the desired boundaries through the iterative evolution of a 4D implicit function. In this work, the propagation relies on a robust evolution model including adaptive parameters. These depend on the input data and on statistical distribution models. The main contribution of this paper is the use of an automatic registration method to initialize the surface, as an alternative solution to manual initialization. The registration is achieved through a robust multiresolution and multigrid minimization scheme. This coupling significantly improves the quality of the method, since the segmentation is faster, more reliable and fully automatic. Quantitative and qualitative results on both synthetic and real volumetric brain MR images are presented and discussed.     

引入靶区外扩预测放射治疗中危及器官自动分割的平均剂量偏差

目的 观察引入靶区外扩预测放射治疗(放疗)中自动分割危及器官(OAR)的平均剂量偏差的价值。方法 将100例接受放疗的直肠癌患者随机分为训练集(n=30)和测试集(n=70)。对训练集手动分割CT图中的靶区,之后分别对膀胱、小肠和双侧股骨头4个OAR进行手动和自动分割。根据自动分割的OAR设计放疗计划,得到对应的剂量分布;利用Python程序统计每个OAR与靶区外扩环重叠区域内的剂量平均值,以之作为代表剂量,用于预测测试集手动与自动分割平均剂量的差异,比较预测平均剂量与实际平均剂量的差异。再次随机将100例分为训练集、测试集各50例,重复上述过程。结果 首次预测显示,测试集70例中,69例膀胱预测与实际剂量差异均<0.5 Gy,69例小肠预测与实际剂量差异均<3 Gy,全部70例双侧股骨头预测与实际剂量差异均<0.5 Gy;对于膀胱、小肠和左、右侧股骨头,预测与实际平均剂量差异的一致性相关系数(CCC)分别为0.96、0.86、0.81和0.69。第2次预测显示,测试集50例中,46例膀胱的预测与实际剂量差异均<0.5 Gy,49例小肠的预测和实际剂量差异均<3 Gy,所有病例双侧股骨头的预测和实际剂量差异均<0.5 Gy;对于膀胱、小肠和左、右侧股骨头,预测与实际平均剂量差异的CCC分别为0.97、0.90、0.82和0.78。结论 引入靶区外扩可有效预测直肠癌放疗中自动分割OAR产生的剂量偏差。     

Automatic Shape-Based Level Set Segmentation for Needle Tracking in 3-D TRUS-Guided Prostate Brachytherapy

Prostate brachytherapy is an effective treatment for early prostate cancer. The success depends critically on the correct needle implant positions. We have devised an automatic shape-based level set segmentation tool for needle tracking in 3-D transrectal ultrasound (TRUS) images, which uses the shape information and level set technique to localize the needle position and estimate the endpoint of needle in real-time. The 3-D TRUS images used in the evaluation of our tools were obtained using a 2-D TRUS transducer from Ultrasonix (Richmond, BC, Canada) and a computer-controlled stepper motor system from Thorlabs (Newton, NJ, USA). The accuracy and feedback mechanism had been validated using prostate phantoms and compared with 3-D positions of these needles derived from experts' readings. The experts' segmentation of needles from 3-D computed tomography images was the ground truth in this study. The difference between automatic and expert segmentations are within 0.1 mm for 17 of 19 implanted needles. The mean errors of automatic segmentations by comparing with the ground truth are within 0.25 mm. Our automated method allows real-time TRUS-based needle placement difference within one pixel compared with manual expert segementation.     

Evaluation of targeting errors in ultrasound-assisted radiotherapy

A method for validating the start-to-end accuracy of a 3-D ultrasound (US)-based patient positioning system for radiotherapy is described. A radiosensitive polymer gel is used to record the actual dose delivered to a rigid phantom after being positioned using 3-D US guidance. Comparison of the delivered dose with the treatment plan allows accuracy of the entire radiotherapy treatment process, from simulation to 3-D US guidance, and finally delivery of radiation, to be evaluated. The 3-D US patient positioning system has a number of features for achieving high accuracy and reducing operator dependence. These include using tracked 3-D US scans of the target anatomy acquired using a dedicated 3-D ultrasound probe during both the simulation and treatment sessions, automatic 3-D US-to-US registration and use of infrared LED (IRED) markers of the optical position-sensing system for registering simulation computed tomography to US data. The mean target localization accuracy of this system was 2.5 mm for four target locations inside the phantom, compared with 1.6 mm obtained using the conventional patient positioning method of laser alignment. Because the phantom is rigid, this represents the best possible set-up accuracy of the system. Thus, these results suggest that 3-D US-based target localization is practically feasible and potentially capable of increasing the accuracy of patient positioning for radiotherapy in sites where day-to-day organ shifts are greater than 1 mm in magnitude.     

A supervised learning framework of statistical shape and probability priors for automatic prostate segmentation in ultrasound images

Prostate segmentation aids in prostate volume estimation, multi-modal image registration, and to create patient specific anatomical models for surgical planning and image guided biopsies. However, manual segmentation is time consuming and suffers from inter-and intra-observer variabilities. Low contrast images of trans rectal ultrasound and presence of imaging artifacts like speckle, micro-calcifications, and shadow regions hinder computer aided automatic or semi-automatic prostate segmentation. In this paper, we propose a prostate segmentation approach based on building multiple mean parametric models derived from principal component analysis of shape and posterior probabilities in a multi-resolution framework. The model parameters are then modified with the prior knowledge of the optimization space to achieve optimal prostate segmentation. In contrast to traditional statistical models of shape and intensity priors, we use posterior probabilities of the prostate region determined from random forest classification to build our appearance model, initialize and propagate our model. Furthermore, multiple mean models derived from spectral clustering of combined shape and appearance parameters are applied in parallel to improve segmentation accuracies. The proposed method achieves mean Dice similarity coefficient value of 0.91 ± 0.09 for 126 images containing 40 images from the apex, 40 images from the base and 46 images from central regions in a leave-one-patient-out validation framework. The mean segmentation time of the procedure is 0.67 ± 0.02 s.     

TRUS–MRI image registration: a paradigm shift in the diagnosis of significant prostate cancer

Accuracy of multiparametric MRI has greatly improved the ability of localizing tumor foci of prostate cancer. This property can be used to perform a TRUS–MR image registration, new technological advance, which allows for an overlay of an MRI onto a TRUS image to target a prostate biopsy toward a suspicious area Three types of registration have been developed: cognitive-based, sensor-based, and organ-based registration. Cognitive registration consists of aiming a suspicious area during biopsy with the knowledge of the lesion location identified on multiparametric MRI. Sensor-based registration consists of tracking in real time the TRUS probe with a magnetic device, achieving a global positioning system which overlays in real-time prostate image on both modalities. Its main limitation is that it does not take into account prostate and patient motion during biopsy. Two systems (Artemis and Uronav) have been developed to partially circumvent this drawback. Organ-based registration (Koelis) does not aim to track the TRUS probe, but the prostate itself to compute in a 3D acquisition the TRUS prostate shape, allowing for a registration with the corresponding 3D MRI shape. This system is not limited by prostate/patient motion and allows for a deformation of the organ during registration. Pros and cons of each technique and the rationale for a targeted biopsy only policy are discussed.     

Cadaver validation of intensity-based ultrasound to CT registration

A method is presented for the rigid registration of tracked B-mode ultrasound images to a CT volume of a femur and pelvis. This registration can allow tracked surgical instruments to be aligned with the CT image or an associated preoperative plan. Our method is fully automatic and requires no manual segmentation of either the ultrasound images or the CT volume. The parameter which is directly related to the speed of sound through tissue has also been included in the registration optimisation process. Experiments have been carried out on six cadaveric femurs and three cadaveric pelves. Registration results were compared with a "gold standard" registration acquired using bone implanted fiducial markers. Results show the registration method to be accurate, on average, to 1.6 mm root-mean-square target registration error.     

Musculoskeletal MRI segmentation using multi-resolution simplex meshes with medial representations

The automatic segmentation of the musculoskeletal system from medical images is a particularly challenging task, due to its morphological complexity, its large variability in the population and its potentially large deformations. In this paper we propose a novel approach for musculoskeletal segmentation and registration based on simplex meshes. Such discrete models have already proven to be efficient and versatile for medical image segmentation. We extend the current framework by introducing a multi-resolution approach and a reversible medial representation, in order to reduce the complexity of geometric and non-penetration constraints computation. Our framework allows both inter and intra-patient registration (involving both rigid and elastic matching). We also show that the introduced representations facilitate morphological analysis. As a case study, we demonstrate that muscles, bones, ligaments and cartilages of the hip and the thigh can be registered at an interactive frame rate, in a time-efficient way (<30 min), with a satisfactory accuracy (～1.5 mm), and with a minimal amount of manual tasks.     

Anatomy-guided multimodal registration by learning segmentation without ground truth: Application to intraprocedural CBCT/MR liver segmentation and registration

Multimodal image registration has many applications in diagnostic medical imaging and image-guided interventions, such as Transcatheter Arterial Chemoembolization (TACE) of liver cancer guided by intraprocedural CBCT and pre-operative MR. The ability to register peri-procedurally acquired diagnostic images into the intraprocedural environment can potentially improve the intra-procedural tumor targeting, which will significantly improve therapeutic outcomes. However, the intra-procedural CBCT often suffers from suboptimal image quality due to lack of signal calibration for Hounsfield unit, limited FOV, and motion/metal artifacts. These non-ideal conditions make standard intensity-based multimodal registration methods infeasible to generate correct transformation across modalities. While registration based on anatomic structures, such as segmentation or landmarks, provides an efficient alternative, such anatomic structure information is not always available. One can train a deep learning-based anatomy extractor, but it requires large-scale manual annotations on specific modalities, which are often extremely time-consuming to obtain and require expert radiological readers. To tackle these issues, we leverage annotated datasets already existing in a source modality and propose an anatomy-preserving domain adaptation to segmentation network (APA2Seg-Net) for learning segmentation without target modality ground truth. The segmenters are then integrated into our anatomy-guided multimodal registration based on the robust point matching machine. Our experimental results on in-house TACE patient data demonstrated that our APA2Seg-Net can generate robust CBCT and MR liver segmentation, and the anatomy-guided registration framework with these segmenters can provide high-quality multimodal registrations.     

Near Real-Time Robust Non-rigid Registration of Volumetric Ultrasound Images for Neurosurgery

Ultrasound images are acquired before and after the resection of brain tumors to help the surgeon to localize the tumor and its extent and to minimize the amount of residual tumor after the resection. Because the brain undergoes large deformation between these two acquisitions, deformable image-based registration of these data sets is of substantial clinical importance. In this work, we present an algorithm for non-rigid registration of ultrasound images (RESOUND) that models the deformation with free-form cubic B-splines. We formulate a regularized cost function that uses normalized cross-correlation as the similarity metric. To optimize the cost function, we calculate its analytic derivative and use the stochastic gradient descent technique to achieve near real-time performance. We further propose a robust technique to minimize the effect of non-corresponding regions such as the resected tumor and possible hemorrhage in the post-resection image. Using manually labeled corresponding landmarks in the pre- and post-resection ultrasound volumes, we illustrate that our registration algorithm reduces the mean target registration error from an initial value of 3.7 to 1.5 mm. We also compare RESOUND with the previous work of Mercier et al. (2013) and illustrate that it has three important advantages: (i) it is fully automatic and does not require a manual segmentation of the tumor, (ii) it produces smaller registration errors and (iii) it is about 30 times faster. The clinical data set is available online on the BITE database website.     

Weakly supervised histopathology cancer image segmentation and classification

Labeling a histopathology image as having cancerous regions or not is a critical task in cancer diagnosis; it is also clinically important to segment the cancer tissues and cluster them into various classes. Existing supervised approaches for image classification and segmentation require detailed manual annotations for the cancer pixels, which are time-consuming to obtain. In this paper, we propose a new learning method, multiple clustered instance learning (MCIL) (along the line of weakly supervised learning) for histopathology image segmentation. The proposed MCIL method simultaneously performs image-level classification (cancer vs. non-cancer image), medical image segmentation (cancer vs. non-cancer tissue), and patch-level clustering (different classes). We embed the clustering concept into the multiple instance learning (MIL) setting and derive a principled solution to performing the above three tasks in an integrated framework. In addition, we introduce contextual constraints as a prior for MCIL, which further reduces the ambiguity in MIL. Experimental results on histopathology colon cancer images and cytology images demonstrate the great advantage of MCIL over the competing methods.     

Multi-spectral brain tissue segmentation using automatically trained k-Nearest-Neighbor classification

Conventional k-Nearest-Neighbor (kNN) classification, which has been successfully applied to classify brain tissue in MR data, requires training on manually labeled subjects. This manual labeling is a laborious and time-consuming procedure. In this work, a new fully automated brain tissue classification procedure is presented, in which kNN training is automated. This is achieved by non-rigidly registering the MR data with a tissue probability atlas to automatically select training samples, followed by a post-processing step to keep the most reliable samples. The accuracy of the new method was compared to rigid registration-based training and to conventional kNN-based segmentation using training on manually labeled subjects for segmenting gray matter (GM), white matter (WM) and cerebrospinal fluid (CSF) in 12 data sets. Furthermore, for all classification methods, the performance was assessed when varying the free parameters. Finally, the robustness of the fully automated procedure was evaluated on 59 subjects. The automated training method using non-rigid registration with a tissue probability atlas was significantly more accurate than rigid registration. For both automated training using non-rigid registration and for the manually trained kNN classifier, the difference with the manual labeling by observers was not significantly larger than inter-observer variability for all tissue types. From the robustness study, it was clear that, given an appropriate brain atlas and optimal parameters, our new fully automated, non-rigid registration-based method gives accurate and robust segmentation results. A similarity index was used for comparison with manually trained kNN. The similarity indices were 0.93, 0.92 and 0.92, for CSF, GM and WM, respectively. It can be concluded that our fully automated method using non-rigid registration may replace manual segmentation, and thus that automated brain tissue segmentation without laborious manual training is feasible.     

Semi-automatic segmentation for prostate interventions

In this paper we report and characterize a semi-automatic prostate segmentation method for prostate brachytherapy. Based on anatomical evidence and requirements of the treatment procedure, a warped and tapered ellipsoid was found suitable as the a-priori 3D shape of the prostate. By transforming the acquired endorectal transverse images of the prostate into ellipses, the shape fitting problem was cast into a convex problem which can be solved efficiently. The average whole gland error between non-overlapping volumes created from manual and semi-automatic contours from 21 patients was 6.63 ± 0.9%. For use in brachytherapy treatment planning, the resulting contours were modified, if deemed necessary, by radiation oncologists prior to treatment. The average whole gland volume error between the volumes computed from semi-automatic contours and those computed from modified contours, from 40 patients, was 5.82 ± 4.15%. The amount of bias in the physicians' delineations when given an initial semi-automatic contour was measured by comparing the volume error between 10 prostate volumes computed from manual contours with those of modified contours. This error was found to be 7.25 ± 0.39% for the whole gland. Automatic contouring reduced subjectivity, as evidenced by a decrease in segmentation inter- and intra-observer variability from 4.65% and 5.95% for manual segmentation to 3.04% and 3.48% for semi-automatic segmentation, respectively. We characterized the performance of the method relative to the reference obtained from manual segmentation by using a novel approach that divides the prostate region into nine sectors. We analyzed each sector independently as the requirements for segmentation accuracy depend on which region of the prostate is considered. The measured segmentation time is 14 ± 1s with an additional 32 ± 14s for initialization. By assuming 1-3 min for modification of the contours, if necessary, a total segmentation time of less than 4 min is required, with no additional time required prior to treatment planning. This compares favorably to the 5-15 min manual segmentation time required for experienced individuals. The method is currently used at the British Columbia Cancer Agency (BCCA) Vancouver Cancer Centre as part of the standard treatment routine in low dose rate prostate brachytherapy and is found to be a fast, consistent and accurate tool for the delineation of the prostate gland in ultrasound images.