PLOSL: Population learning followed by one shot learning pulmonary image registration using tissue volume preserving and vesselness constraints

This paper presents the Population Learning followed by One Shot Learning (PLOSL) pulmonary image registration method. PLOSL is a fast unsupervised learning-based framework for 3D-CT pulmonary image registration algorithm based on combining population learning (PL) and one-shot learning (OSL). The PLOSL image registration has the advantages of the PL and OSL approaches while reducing their respective drawbacks. The advantages of PLOSL include improved performance over PL, substantially reducing OSL training time and reducing the likelihood of OSL getting stuck in local minima. PLOSL pulmonary image registration uses tissue volume preserving and vesselness constraints for registration of inspiration-to-expiration and expiration-to-inspiration pulmonary CT images. A coarse-to-fine convolution encoder-decoder CNN architecture is used to register large and small shape features. During training, the sum of squared tissue volume difference (SSTVD) compensates for intensity differences between inspiration and expiration computed tomography (CT) images and the sum of squared vesselness measure difference (SSVMD) helps match the lung vessel tree. Results show that the PLOSL (SSTVD+SSVMD) algorithm achieved subvoxel landmark error while preserving pulmonary topology on the SPIROMICS data set, the public DIR-LAB COPDGene and 4DCT data sets.     

Selective image similarity measure for bronchoscope tracking based on image registration

We propose a selective method of measurement for computing image similarities based on characteristic structure extraction and demonstrate its application to flexible endoscope navigation, in particular to a bronchoscope navigation system. Camera motion tracking is a fundamental function required for image-guided treatment or therapy systems. In recent years, an ultra-tiny electromagnetic sensor commercially became available, and many image-guided treatment or therapy systems use this sensor for tracking the camera position and orientation. However, due to space limitations, it is difficult to equip the tip of a bronchoscope with such a position sensor, especially in the case of ultra-thin bronchoscopes. Therefore, continuous image registration between real and virtual bronchoscopic images becomes an efficient tool for tracking the bronchoscope. Usually, image registration is done by calculating the image similarity between real and virtual bronchoscopic images. Since global schemes to measure image similarity, such as mutual information, squared gray-level difference, or cross correlation, average differences in intensity values over an entire region, they fail at tracking of scenes where less characteristic structures can be observed. The proposed method divides an entire image into a set of small subblocks and only selects those in which characteristic shapes are observed. Then image similarity is calculated within the selected subblocks. Selection is done by calculating feature values within each subblock. We applied our proposed method to eight pairs of chest X-ray CT images and bronchoscopic video images. The experimental results revealed that bronchoscope tracking using the proposed method could track up to 1600 consecutive bronchoscopic images (about 50 s) without external position sensors. Tracking performance was greatly improved in comparison with a standard method utilizing squared gray-level differences of the entire images.     

Deformable image registration for temporal subtraction of chest radiographs

Purpose

Temporal subtraction images constructed from image registration can facilitate the visualization of pathologic changes. In this study, we propose a deformable image registration (DIR) framework for creating temporal subtraction images of chest radiographs.

Methods

We developed a DIR methodology using two different image similarity metrics, varying flow (VF) and compressible flow (CF). The proposed registration method consists of block matching, filtering, and interpolation. Specifically, corresponding point pairs between reference and target images are initially determined by minimizing a nonlinear least squares formulation using grid-searching optimization. A two-step filtering process, including least median of squares filtering and backward matching filtering, is then applied to the estimated point matches in order to remove erroneous matches. Finally, moving least squares is used to generate a full displacement field from the filtered point pairs.

Results

We applied the proposed DIR method to 10 pairs of clinical chest radiographs and compared it with the demons and B-spline algorithms using the five-point rating score method. The average quality scores were 2.7 and 3 for the demons and B-spline methods, but 3.5 and 4.1 for the VF and CF methods. In addition, subtraction images improved the visual perception of abnormalities in the lungs by using the proposed method.

Conclusion

The VF and CF models achieved a higher accuracy than the demons and the B-spline methods. Furthermore, the proposed methodology demonstrated the ability to create clinically acceptable temporal subtraction chest radiographs that enhance interval changes and can be used to detect abnormalities such as non-small cell lung cancer.     

Tracking of a bronchoscope using epipolar geometry analysis and intensity-based image registration of real and virtual endoscopic images

This paper describes a method for tracking the camera motion of a flexible endoscope, in particular a bronchoscope, using epipolar geometry analysis and intensity-based image registration. The method proposed here does not use a positional sensor attached to the endoscope. Instead, it tracks camera motion using real endoscopic (RE) video images obtained at the time of the procedure and X-ray CT images acquired before the endoscopic examination. A virtual endoscope system (VES) is used for generating virtual endoscopic (VE) images. The basic idea of this tracking method is to find the viewpoint and view direction of the VES that maximizes a similarity measure between the VE and RE images. To assist the parameter search process, camera motion is also computed directly from epipolar geometry analysis of the RE video images. The complete method consists of two steps: (a) rough estimation using epipolar geometry analysis and (b) precise estimation using intensity-based image registration. In the rough registration process, the method computes camera motion from optical flow patterns between two consecutive RE video image frames using epipolar geometry analysis. In the image registration stage, we search for the VES viewing parameters that generate the VE image that is most similar to the current RE image. The correlation coefficient and the mean square intensity difference are used for measuring image similarity. The result obtained in the rough estimation process is used for restricting the parameter search area. We applied the method to bronchoscopic video image data from three patients who had chest CT images. The method successfully tracked camera motion for about 600 consecutive frames in the best case. Visual inspection suggests that the tracking is sufficiently accurate for clinical use. Tracking results obtained by performing the method without the epipolar geometry analysis step were substantially worse. Although the method required about 20 s to process one frame, the results demonstrate the potential of image-based tracking for use in an endoscope navigation system.     

GAN-based disentanglement learning for chest X-ray rib suppression

Clinical evidence has shown that rib-suppressed chest X-rays (CXRs) can improve the reliability of pulmonary disease diagnosis. However, previous approaches on generating rib-suppressed CXR face challenges in preserving details and eliminating rib residues. We hereby propose a GAN-based disentanglement learning framework called Rib Suppression GAN, or RSGAN, to perform rib suppression by utilizing the anatomical knowledge embedded in unpaired computed tomography (CT) images. In this approach, we employ a residual map to characterize the intensity difference between CXR and the corresponding rib-suppressed result. To predict the residual map in CXR domain, we disentangle the image into structure- and contrast-specific features and transfer the rib structural priors from digitally reconstructed radiographs (DRRs) computed by CT. Furthermore, we employ additional adaptive loss to suppress rib residue and preserve more details. We conduct extensive experiments based on 1673 CT volumes, and four benchmarking CXR datasets, totaling over 120K images, to demonstrate that (i) our proposed RSGAN achieves superior image quality compared to the state-of-the-art rib suppression methods; (ii) combining CXR with our rib-suppressed result leads to better performance in lung disease classification and tuberculosis area detection.     

Landmark detection in the chest and registration of lung surfaces with an application to nodule registration

We developed an automated system for registering computed tomography (CT) images of the chest temporally. Our system detects anatomical landmarks, in particular, the trachea, sternum and spine, using an attenuation-based template matching approach. It computes the optimal rigid-body transformation that aligns the corresponding landmarks in two CT scans of the same patient. This transformation then provides an initial registration of the lung surfaces segmented from the two scans. The initial surface alignment is refined step by step in an iterative closest-point (ICP) process. To establish the correspondence of lung surface points, Elias' nearest neighbor algorithm was adopted. Our method improves the processing time of the original ICP algorithm from O(kn log n) to O(kn), where k is the number of iterations and n the number of surface points. The surface transformation is applied to align nodules in the initial CT scan with nodules in the follow-up scan. For 56 out of 58 nodules in the initial CT scans of 10 patients, nodule correspondences in the follow-up scans are established correctly. Our methods can therefore potentially facilitate the radiologist's evaluation of pulmonary nodules on chest CT for interval growth.     

Re-Identification and growth detection of pulmonary nodules without image registration using 3D siamese neural networks

Lung cancer follow-up is a complex, error prone, and time consuming task for clinical radiologists. Several lung CT scan images taken at different time points of a given patient need to be individually inspected, looking for possible cancerogenous nodules. Radiologists mainly focus their attention in nodule size, density, and growth to assess the existence of malignancy. In this study, we present a novel method based on a 3D siamese neural network, for the re-identification of nodules in a pair of CT scans of the same patient without the need for image registration. The network was integrated into a two-stage automatic pipeline to detect, match, and predict nodule growth given pairs of CT scans. Results on an independent test set reported a nodule detection sensitivity of 94.7%, an accuracy for temporal nodule matching of 88.8%, and a sensitivity of 92.0% with a precision of 88.4% for nodule growth detection.     

SLIR: Synthesis,localization, inpainting,and registration for image-guided thermal ablation of liver tumors

Thermal ablation is a minimally invasive procedure for treating small or unresectable tumors. Although CT is widely used for guiding ablation procedures, yet the contrast of tumors against normal soft tissues is often poor in CT scans, aggravating the accurate thermal ablation. In this paper, we propose a fast MR-CT image registration method to overlay pre-procedural MR (pMR) and pre-procedural CT (pCT) images onto an intra-procedural CT (iCT) image to guide the thermal ablation of liver tumors. At the pre-procedural stage, the Cycle-GAN model with mutual information constraint is employed to generate the synthesized CT (sCT) image from the input pMR. Then, pMR-pCT image registration is carried out via traditional mono-modal sCT-pCT image registration. At the intra-procedural stage, the region of the probe and its artifacts are automatically localized and inpainted in the iCT image. Then, an unsupervised registration network (UR-Net) is used to efficiently align the pCT with the inpainted iCT (inpCT) image. The final transform from pMR to iCT is obtained by concatenating the two estimated transforms, i.e., (i) from pMR image space to pCT image space (via sCT) and (ii) from pCT image space to iCT image space (via inpCT). The proposed method has been evaluated over a real clinical dataset and compared with state-of-the-art methods. Experimental results confirm that the proposed method achieves high registration accuracy with fast computation speed.     

Estimation of slipping organ motion by registration with direction-dependent regularization

Accurate estimation of respiratory motion is essential for many applications in medical 4D imaging, for example for radiotherapy of thoracic and abdominal tumors. It is usually done by non-linear registration of image scans at different states of the breathing cycle but without further modeling of specific physiological motion properties. In this context, the accurate computation of respiration-driven lung motion is especially challenging because this organ is sliding along the surrounding tissue during the breathing cycle, leading to discontinuities in the motion field. Without considering this property in the registration model, common intensity-based algorithms cause incorrect estimation along the object boundaries.In this paper, we present a model for incorporating slipping motion in image registration. Extending the common diffusion registration by distinguishing between normal- and tangential-directed motion, we are able to estimate slipping motion at the organ boundaries while preventing gaps and ensuring smooth motion fields inside and outside. We further present an algorithm for a fully automatic detection of discontinuities in the motion field, which does not rely on a prior segmentation of the organ.We evaluate the approach for the estimation of lung motion based on 23 inspiration/expiration pairs of thoracic CT images. The results show a visually more plausible motion estimation. Moreover, the target registration error is quantified using manually defined landmarks and a significant improvement over the standard diffusion regularization is shown.     

Mediastinal atlas creation from 3-D chest computed tomography images: application to automated detection and station mapping of lymph nodes

One important aspect of lung cancer staging is the assessment of mediastinal lymph nodes in 3-D chest computed tomography (CT) images. In the current clinical routine this is done manually by analyzing the 3-D CT image slice by slice to find nodes, evaluate them quantitatively, and assign labels to them for describing the clinical and pathologic extent of metastases. In this paper we present a method to automate the process of lymph node detection and labeling by creation of a mediastinal average image and a novel lymph node atlas containing probability maps for mediastinal, aortic, and N1 nodes. Utilizing a fast deformable registration approach to match the atlas with CT images of new patients, our method can maintain an acceptable runtime. In comparison to previously published methods for mediastinal lymph node detection and labeling it also shows a good sensitivity and positive predictive value.     

Efficient and robust model-to-image alignment using 3D scale-invariant features

This paper presents feature-based alignment (FBA), a general method for efficient and robust model-to-image alignment. Volumetric images, e.g. CT scans of the human body, are modeled probabilistically as a collage of 3D scale-invariant image features within a normalized reference space. Features are incorporated as a latent random variable and marginalized out in computing a maximum a posteriori alignment solution. The model is learned from features extracted in pre-aligned training images, then fit to features extracted from a new image to identify a globally optimal locally linear alignment solution. Novel techniques are presented for determining local feature orientation and efficiently encoding feature intensity in 3D. Experiments involving difficult magnetic resonance (MR) images of the human brain demonstrate FBA achieves alignment accuracy similar to widely-used registration methods, while requiring a fraction of the memory and computation resources and offering a more robust, globally optimal solution. Experiments on CT human body scans demonstrate FBA as an effective system for automatic human body alignment where other alignment methods break down.     

A deep learning framework for unsupervised affine and deformable image registration

Image registration, the process of aligning two or more images, is the core technique of many (semi-)automatic medical image analysis tasks. Recent studies have shown that deep learning methods, notably convolutional neural networks (ConvNets), can be used for image registration. Thus far training of ConvNets for registration was supervised using predefined example registrations. However, obtaining example registrations is not trivial. To circumvent the need for predefined examples, and thereby to increase convenience of training ConvNets for image registration, we propose the Deep Learning Image Registration (DLIR) framework for unsupervised affine and deformable image registration. In the DLIR framework ConvNets are trained for image registration by exploiting image similarity analogous to conventional intensity-based image registration. After a ConvNet has been trained with the DLIR framework, it can be used to register pairs of unseen images in one shot. We propose flexible ConvNets designs for affine image registration and for deformable image registration. By stacking multiple of these ConvNets into a larger architecture, we are able to perform coarse-to-fine image registration. We show for registration of cardiac cine MRI and registration of chest CT that performance of the DLIR framework is comparable to conventional image registration while being several orders of magnitude faster.     

Multiatlas Fusion with a Hybrid CT Number Correction Technique for Subject-Specific Pseudo-CT Estimation in the Context of MRI-Only Radiation Therapy

ObjectiveTo propose a hybrid multiatlas fusion and correction approach to estimate a pseudo–computed tomography (pCT) image from T2-weighted brain magnetic resonance (MR) images in the context of MRI-only radiotherapy.Materials and MethodsA set of eleven pairs of T2-weighted MR and CT brain images was included. Using leave-one-out cross-validation, atlas MR images were registered to the target MRI with multimetric, multiresolution deformable registration. The subsequent deformations were applied to the atlas CT images, producing uncorrected pCT images. Afterward, a three-dimensional hybrid CT number correction technique was used. This technique uses information about MR intensity, spatial location, and tissue label from segmented MR images with the fuzzy c-means algorithm and combines them in a weighted fashion to correct Hounsfield unit values of the uncorrected pCT images. The corrected pCT images were then fused into a final pCT image.ResultsThe proposed hybrid approach proved to be performant in correcting Hounsfield unit values in terms of qualitative and quantitative measures. Average correlation was 0.92 and 0.91 for the proposed approach by taking the mean and the median, respectively, compared with 0.86 for the uncorrected unfused version. Average values of dice similarity coefficient for bone were 0.68 and 0.72 for the fused corrected pCT images by taking the mean and the median, respectively, compared with 0.65 for the uncorrected unfused version indicating a significant bone estimation improvement.ConclusionA hybrid fusion and correction method is presented to estimate a pCT image from T2-weighted brain MR images.     

基于数字模拟技术胸部CT扫描不同组织的噪声

背景：在CT图像中,图像噪声水平与影像质量密切相关,它直接影响和限制着CT对低对比度物质的分辨力。目的：通过图像空间添加噪声软件模拟低剂量CT图像,分析模拟噪声指数与不同组织噪声的相关性及胸部不同组织的噪声变化特点。方法：利用图像空间添加噪声软件对20名自愿者在常规参数下扫描获得的原始图像进行噪声添加,模拟10,30,50,80,100,120,150,180及240mA共9组胸部低剂量图像,记录每幅图像噪声指数,测量主动脉弓层面、心脏层面、肺底3个层面的胸壁软组织、主动脉、脊柱、肺组织及空气中CT值的标准差。结果与结论：图像添加法计算的模拟噪声指数与主动脉弓及其他组织噪声实测噪声差异有显著性意义（P〈0.05）,呈线性相关。胸部低剂量扫描时,当管电流在10-50mA时噪声显著增加,80-240mA时模拟噪声值随着管电流增高降低幅度逐渐减低。胸部各组织120mA组与180mA组图像噪声差异无显著性意义（P〉0.05）。提示图像空间添加噪声软件的噪声指数反映了图像不同组织的整体噪声水平。胸部低剂量CT检查时采用管电流120-180mA时能够保证各组织噪声无明显变化的同时又可以降低辐射剂量。     

Incidental finding of congenital thoracic malformations in adult population

OBJECTIVE: Evaluation of abnormal adult chest computed tomography (CT) scans. STUDY DESIGN: Retrospective series of 3 cases. SETTING: Two University-based hospitals. INTERVENTION: Three adult patients (age range 56 to 61) underwent chest CT scans. Two were trauma patients and one complained of chest pain with a negative cardiac workup. One CT scan revealed a lung mass that was highly suspicious for malignancy and the other two CT scans were suspicious for diaphragmatic hernia. Two patients underwent elective surgery and the third underwent emergent surgery. RESULTS: The CT scan of patient 1 was suspicious for malignancy; however, the pathology of the lung mass revealed only inflammation consistent with an intralobular bronchopulmonary sequestration. Patient 2 had a congenital Morgagni hernia with omentum and colon in the mediastinum, and patient 3 had a diaphragmatic defect combined with a bifid sternum and defective pericardium, fulfilling three criteria of pentalogy of Cantrell. CONCLUSION: Thoracic congenital malformations may go unnoticed and unsuspected until adulthood. They should be kept in mind when reviewing unusual CT scans of the chest in adults. Their surgical management, however, is similar to their counterparts in infancy.     

A protocol for evaluation of similarity measures for non-rigid registration

In this paper, we present a protocol for the evaluation of similarity measures for non-rigid registration. The evaluation is based on five intuitive properties that characterize the behavior of a similarity measure, i.e. the accuracy, capture range, distinctiveness of the optimum, number of local minima, and risk of non-convergence. These five properties are estimated locally from similarity measure values that correspond to a range of systematic local free-form deformations, obtained by displacing control points in random directions from the gold standard position. Global similarity measure properties are obtained by combining the local properties over image regions or over the entire image. The feasibility of the proposed evaluation protocol is demonstrated for three similarity measures: mutual information, normalized mutual information and correlation ratio. The evaluation is carried out on a number of MR and CT images: a pair of simulated MR T1 and MR T2 images of the head, three pairs of real MR T1 and T2 images of the head, six pairs of real MR T1 and CT images of the head, and pairs of MR and CT images of three vertebrae. The protocol may help researchers to select the most appropriate similarity measure for a non-rigid registration task.     

Fully automatic segmentation of the Femur from 3D-CT images using primitive shape recognition and statistical shape models

Purpose

Femur segmentation is well established and widely used in computer-assisted orthopedic surgery. However, most of the robust segmentation methods such as statistical shape models (SSM) require human intervention to provide an initial position for the SSM. In this paper, we propose to overcome this problem and provide a fully automatic femur segmentation method for CT images based on primitive shape recognition and SSM.

Method

Femur segmentation in CT scans was performed using primitive shape recognition based on a robust algorithm such as the Hough transform and RANdom SAmple Consensus. The proposed method is divided into 3 steps: (1) detection of the femoral head as sphere and the femoral shaft as cylinder in the SSM and the CT images, (2) rigid registration between primitives of SSM and CT image to initialize the SSM into the CT image, and (3) fitting of the SSM to the CT image edge using an affine transformation followed by a nonlinear fitting.

Results

The automated method provided good results even with a high number of outliers. The difference of segmentation error between the proposed automatic initialization method and a manual initialization method is less than 1 mm.

Conclusion

The proposed method detects primitive shape position to initialize the SSM into the target image. Based on primitive shapes, this method overcomes the problem of inter-patient variability. Moreover, the results demonstrate that our method of primitive shape recognition can be used for 3D SSM initialization to achieve fully automatic segmentation of the femur.     

Normalized mutual information based registration using k-means clustering and shading correction

In this paper the influence of intensity clustering and shading correction on mutual information based image registration is studied. Instead of the generally used equidistant re-binning, we use k-means clustering in order to achieve a more natural binning of the intensity distribution. Secondly, image inhomogeneities occurring notably in MR images can have adverse effects on the registration. We use a shading correction method in order to reduce these effects. The method is validated on clinical MR, CT and PET images, as well as synthetic MR images. It is shown that by employing clustering with inhomogeneity correction the number of misregistrations is reduced without loss of accuracy thus increasing robustness as compared to the standard non-inhomogeneity corrected and equidistant binning based registration.     

Supervised quality assessment of medical image registration: Application to intra-patient CT lung registration

A novel method for automatic quality assessment of medical image registration is presented. The method is based on supervised learning of local alignment patterns, which are captured by statistical image features at distinctive landmark points. A two-stage classifier cascade, employing an optimal multi-feature model, classifies local alignments into three quality categories: correct, poor or wrong alignment. We establish a reference registration error set as basis for training and testing of the method. It consists of image registrations obtained from different non-rigid registration algorithms and manually established point correspondences of automatically determined landmarks. We employ a set of different classifiers and evaluate the performance of the proposed image features based on the classification performance of corresponding single-feature classifiers. Feature selection is conducted to find an optimal subset of image features and the resulting multi-feature model is validated against the set of single-feature classifiers. We consider the setup generic, however, its application is demonstrated on 51 CT follow-up scan pairs of the lung. On this data, the proposed method performs with an overall classification accuracy of 90%.