Characteristics of general movements in preterm infants assessed by computer-based video analysis

Background: Previous evidence suggests that the variability of the spatial center of infant movements, calculated by computer-based video analysis software, can identify fidgety general movements (GMs) and predict cerebral palsy. Aim: To evaluate whether computer-based video analysis quantifies specific characteristics of normal fidgety movements as opposed to writhing general movements. Methods: A longitudinal study design was applied. Twenty-seven low-to moderate-risk preterm infants (20 boys, 7 girls; mean gestational age 32 [SD 2.7, range 27–36] weeks, mean birth weight 1790 grams [SD 430g, range 1185–2700g]) were videotaped at the ages of 3–5 weeks (period of writhing GMs) and 10–15 weeks (period of fidgety GMs) post term. GMs were classified according to Prechtl’s general movement assessment method (GMA) and by computer-based video analysis. The variability of the centroid of motion (C_SD), derived from differences between subsequent video frames, was calculated by means of computer-based video analysis software; group mean differences between GM periods were reported. Results: The mean variability of the centroid of motion (C_SD) determined by computer-based video analysis was 7.5% lower during the period of fidgety GMs than during the period of writhing GMs (p = 0.004). Conclusion: Our findings support that the variability of the centroid of motion reflects small and variable movements evenly distributed across the body, and hence shows that computer-based video analysis qualifies for assessment of direction and amplitude of FMs in young infants.     

Semi-supervised segmentation of echocardiography videos via noise-resilient spatiotemporal semantic calibration and fusion

We present a novel model for left ventricle endocardium segmentation from echocardiography video, which is of great significance in clinical practice and yet a challenging task due to (1) the severe speckle noise in echocardiography videos, (2) the irregular motion of pathological heart, and (3) the limited training data caused by high annotation cost. The proposed model has three compelling characteristics. First, we propose a novel adaptive spatiotemporal semantic calibration method to align the feature maps of consecutive frames, where the spatiotemporal correspondences are figured out based on feature maps instead of pixels, thereby mitigating the adverse effects of speckle noise in the calibration. Second, we further learn the importance of each feature map of neighbouring frames to the current frame from the temporal perspective so as to distinctively rather than uniformly harness the temporal information to tackle the irregular and anisotropic motions. Third, we integrate these techniques into the mean teacher semi-supervised architecture to leverage a large amount of unlabeled data to improve the segmentation accuracy. We extensively evaluate the proposed method on two public echocardiography video datasets (EchoNet-Dynamic and CAMUS), where the average dice coefficient on the left ventricular endocardium segmentation achieves 92.87% and 93.79%, respectively. Comparisons with state-of-the-art methods also demonstrate the effectiveness of the proposed method by achieving a better segmentation performance with a faster speed.     

Informative frame classification for endoscopy video

Advances in video technology allow inspection, diagnosis and treatment of the inside of the human body without or with very small scars. Flexible endoscopes are used to inspect the esophagus, stomach, small bowel, colon, and airways, whereas rigid endoscopes are used for a variety of minimal invasive surgeries (i.e., laparoscopy, arthroscopy, endoscopic neurosurgery). These endoscopes come in various sizes, but all have a tiny video camera at the tip. During an endoscopic procedure, the tiny video camera generates a video signal of the interior of the human organ, which is displayed on a monitor for real-time analysis by the physician. However, many out-of-focus frames are present in endoscopy videos because current endoscopes are equipped with a single, wide-angle lens that cannot be focused. We need to distinguish the out-of-focus frames from the in-focus frames to utilize the information of the out-of-focus and/or the in-focus frames for further automatic or semi-automatic computer-aided diagnosis (CAD). This classification can reduce the number of images to be viewed by a physician and to be analyzed by a CAD system. We call an out-of-focus frame a non-informative frame and an in-focus frame an informative frame. The out-of-focus frames have characteristics that are different from those of in-focus frames. In this paper, we propose two new techniques (edge-based and clustering-based) to classify video frames into two classes, informative and non-informative frames. However, because intensive specular reflections reduce the accuracy of the classification we also propose a specular reflection detection technique, and use the detected specular reflection information to increase the accuracy of informative frame classification. Our experimental studies indicate that precision, sensitivity, specificity, and accuracy for the specular reflection detection technique and the two informative frame classification techniques are greater than 90% and 95%, respectively.     

Intervention fidelity monitoring of Urban Zen Integrative Therapy (UZIT) for persons with pulmonary hypertension

BackgroundSystematic and consistent dose delivery is critical in intervention research. Few studies testing complementary health approach (CHA) interventions describe intervention fidelity monitoring (IFM) and measurement.ObjectiveTo describe methodological processes in establishing and measuring consistent dose, delivery, and duration of a multi-component CHA intervention.MethodsAdults with pulmonary hypertension received six weekly, 1-hour Urban Zen Integrative Therapy (UZIT) sessions. A total of 78 sessions were delivered and 33% of these sessions were audited. Intervention dose (time allocated to each component), intervention consistency (protocol adherence audits), and intervention delivery (performance and sequence of components) were captured using remote video observation and review of the recorded video. IFM audits were performed at the beginning (n = 16), middle (n = 5), and end (n = 5) of the study.ResultsUZIT interventionists adhered to the intervention protocol (99.3%) throughout the study period. Interventionists delivered UZIT components within the prescribed timeframe: 1) Beginning: gentle body movement (18.9 ± 5.8 min.), restorative pose with guided body awareness meditation (21.3 ± 2.7 min.), and Reiki (22.8 ± 3.1 min.); 2) Middle: gentle body movement (15.9 ± 1.5 min.), pose/body awareness meditation (30.1 ± 6.5 min.), and Reiki (30.1 ± 7.0 min.); 3) End: gentle body movement (18.1 ± 3.6 min.), pose/body awareness meditation (35.3 ± 6.4 min.), and Reiki (34.5 ± 7.0 min.). Essential oil inhalation was delivered during UZIT sessions 100% of the time. Interventionists adhered to treatment delivery behaviors throughout the study period: beginning (98.86%), middle (100%), and end (100%).DiscussionIn this pilot study, we demonstrated that the dose, consistency, and delivery of multi-component CHA therapy can be standardized and monitored to ensure intervention fidelity.     

Analysis of Carotid Artery Transverse Sections in Long Ultrasound Video Sequences

Examination of the common carotid artery (CCA) based on an ultrasound video sequence is an effective method for detecting cardiovascular diseases. Here, we propose a video processing method for the automated geometric analysis of CCA transverse sections. By explicitly compensating the parasitic phenomena of global movement and feature drift, our method enables a reliable and accurate estimation of the movement of the arterial wall based on ultrasound sequences of arbitrary length and in situations where state-of-the-art methods fail or are very inaccurate. The method uses a modified Viola–Jones detector and the Hough transform to localize the artery in the image. Then it identifies dominant scatterers, also known as interest points (IPs), whose positions are tracked by means of the pyramidal Lucas–Kanade method. Robustness to global movement and feature drift is achieved by a detection of global movement and subsequent IP re-initialization, as well as an adaptive removal and addition of IPs. The performance of the proposed method is evaluated using simulated and real ultrasound video sequences. Using the Harris detector for IP detection, we obtained an overall root-mean-square error, averaged over all the simulated sequences, of 2.16?±?1.18 px. The computational complexity of our method is compatible with real-time operation; the runtime is about 30–70?ms/frame for sequences with a spatial resolution of up to 490?×?490 px. We expect that in future clinical practice, our method will be instrumental for non-invasive early-stage diagnosis of atherosclerosis and other cardiovascular diseases.     

Effect on performance of learning a pilates skill with or without a mirror

Mirrors are often used in an instructional environment where precise movements must be learned (e.g., martial arts, Pilates, dance). The potential for mirrors in the learning environment of a Pilates class, to affect the subsequent performance of a Pilates star movement when mirrors are not present, was examined. Twenty subjects learned the Pilates star movement over seven weeks, either with (n=11) or without (n=9), mirrors present in the Pilates studio. Performance of the star without mirrors present was assessed quantitatively before and after the training, by video analysis of the degree of lateral straightness of the subject's body at the start, middle, and end of the star movement. Performance of the star movement without a mirror present improved similarly for both the group that learned with, and the group that learned without, mirrors present (p<0.05). These results indicate that the inclusion of mirrors in a learning environment, to provide immediate visual feedback during learning, does not necessarily enhance the subsequent performance of a skill when mirrors are not present.     

On Evaluating MHC-II Binding Peptide Prediction Methods

Choice of one method over another for MHC-II binding peptide prediction is typically based on published reports of their estimated performance on standard benchmark datasets. We show that several standard benchmark datasets of unique peptides used in such studies contain a substantial number of peptides that share a high degree of sequence identity with one or more other peptide sequences in the same dataset. Thus, in a standard cross-validation setup, the test set and the training set are likely to contain sequences that share a high degree of sequence identity with each other, leading to overly optimistic estimates of performance. Hence, to more rigorously assess the relative performance of different prediction methods, we explore the use of similarity-reduced datasets. We introduce three similarity-reduced MHC-II benchmark datasets derived from MHCPEP, MHCBN, and IEDB databases. The results of our comparison of the performance of three MHC-II binding peptide prediction methods estimated using datasets of unique peptides with that obtained using their similarity-reduced counterparts shows that the former can be rather optimistic relative to the performance of the same methods on similarity-reduced counterparts of the same datasets. Furthermore, our results demonstrate that conclusions regarding the superiority of one method over another drawn on the basis of performance estimates obtained using commonly used datasets of unique peptides are often contradicted by the observed performance of the methods on the similarity-reduced versions of the same datasets. These results underscore the importance of using similarity-reduced datasets in rigorously comparing the performance of alternative MHC-II peptide prediction methods.     

Dual-channel cascade pose estimation network trained on infrared thermal image and groundtruth annotation for real-time gait measurement

Real-time spatiotemporal parameter measurement for gait analysis is challenging. Previous techniques for 3D motion analysis, such as inertial measurement units, marker based motion analysis or the use of depth cameras, require expensive equipment, highly skilled staff and limits feasibility for sustainable applications. In this paper a dual-channel cascaded network to perform contactless real-time 3D human pose estimation using a single infrared thermal video as an input is proposed. An algorithm to calculate gait spatiotemporal parameters is presented by tracking estimated joint locations. Additionally, a training dataset composed of infrared thermal images and groundtruth annotations has been developed. The annotation represents a set of 3D joint locations from infrared optical trackers, which is considered to be the gold standard in clinical applications. On the proposed dataset, our pose estimation framework achieved a 3D human pose mean error of below 21 mm and outperforms state-of-the-art methods. The results reveal that the proposed system achieves competitive skeleton tracking performance on par with the other motion capture devices and exhibited good agreement with a marker-based three-dimensional motion analysis system (3DMA) over a range of spatiotemporal parameters. Moreover, the process is shown to distinguish differences in over-ground gait parameters of older adults with and without Hemiplegia’s disease. We believe that the proposed approaches can measure selected spatiotemporal gait parameters and could be effectively used in clinical or home settings.     

Combined video tracking and image-video registration for continuous bronchoscopic guidance

Objective

Three-dimensional (3D) computed-tomography (CT) images and bronchoscopy are commonly used tools for the assessment of lung cancer. Before bronchoscopy, the physician first examines a 3D CT chest image to select pertinent diagnostic sites and to ascertain possible routes through the airway tree leading to each site. Next, during bronchoscopy, the physician maneuvers the bronchoscope through the airways, basing navigation decisions on the live bronchoscopic video, to reach each diagnostic site. Unfortunately, no direct link exists between the 3D CT image data and bronchoscopic video. This makes bronchoscopy difficult to perform successfully. Existing methods for the image-based guidance of bronchoscopy, either only involve single-frame registration or operate at impractically slow frame rates. We describe a method that combines the 3D CT image data and bronchoscopic video to enable continuous bronchoscopic guidance.

Methods

The method interleaves periodic CT-video registration with bronchoscopic video motion tracking. It begins by using single-frame CT-video registration to register the “Real World” of the bronchoscopic video and the “Virtual World” of the CT-based endoluminal renderings. Next, the method uses an optical-flow-based approach to track bronchoscope movement for a fixed number of video frames and also simultaneously updates the virtual-world position. This process of registration and tracking repeats as the bronchoscope moves.

Results

The method operates robustly over a variety ofphantom and human cases. It typically performs successfully for over 150 frames and through multiple airway generations. In our tests, the method runs at a rate of up to seven frames per second. We have integrated the method into a complete system for the image-based planning and guidance of bronchoscopy.

Conclusion

The method performs over an order of magnitude faster than previously proposed image-based bronchoscopy guidance methods. Software optimization and a modest hardware improvement could enable real-time performance.     

Improved Tracking Performance of Lagrangian Block-Matching Methodologies Using Block Expansion in the Time Domain: In Silico, Phantom and in Vivo Evaluations

The aim of this study was to evaluate tracking performance when an extra reference block is added to a basic block-matching method, where the two reference blocks originate from two consecutive ultrasound frames. The use of an extra reference block was evaluated for two putative benefits: (i) an increase in tracking performance while maintaining the size of the reference blocks, evaluated using in silico and phantom cine loops; (ii) a reduction in the size of the reference blocks while maintaining the tracking performance, evaluated using in vivo cine loops of the common carotid artery where the longitudinal movement of the wall was estimated. The results indicated that tracking accuracy improved (mean = 48%, p < 0.005 [in silico]; mean = 43%, p < 0.01 [phantom]), and there was a reduction in size of the reference blocks while maintaining tracking performance (mean = 19%, p < 0.01 [in vivo]). This novel method will facilitate further exploration of the longitudinal movement of the arterial wall.     

Unsupervised domain adaptation for clinician pose estimation and instance segmentation in the operating room

The fine-grained localization of clinicians in the operating room (OR) is a key component to design the new generation of OR support systems. Computer vision models for person pixel-based segmentation and body-keypoints detection are needed to better understand the clinical activities and the spatial layout of the OR. This is challenging, not only because OR images are very different from traditional vision datasets, but also because data and annotations are hard to collect and generate in the OR due to privacy concerns. To address these concerns, we first study how joint person pose estimation and instance segmentation can be performed on low resolutions images with downsampling factors from 1x to 12x. Second, to address the domain shift and the lack of annotations, we propose a novel unsupervised domain adaptation method, called AdaptOR, to adapt a model from an in-the-wild labeled source domain to a statistically different unlabeled target domain. We propose to exploit explicit geometric constraints on the different augmentations of the unlabeled target domain image to generate accurate pseudo labels and use these pseudo labels to train the model on high- and low-resolution OR images in a self-training framework. Furthermore, we propose disentangled feature normalization to handle the statistically different source and target domain data. Extensive experimental results with detailed ablation studies on the two OR datasets MVOR+ and TUM-OR-test show the effectiveness of our approach against strongly constructed baselines, especially on the low-resolution privacy-preserving OR images. Finally, we show the generality of our method as a semi-supervised learning (SSL) method on the large-scale COCO dataset, where we achieve comparable results with as few as 1% of labeled supervision against a model trained with 100% labeled supervision. Code is available at https://github.com/CAMMA-public/HPE-AdaptOR.     

Automated left ventricular segmentation from cardiac magnetic resonance images via adversarial learning with multi-stage pose estimation network and co-discriminator

Left ventricular (LV) segmentation is essential for the early diagnosis of cardiovascular diseases, which has been reported as the leading cause of death all over the world. However, automated LV segmentation from cardiac magnetic resonance images (CMRI) using the traditional convolutional neural networks (CNNs) is still a challenging task due to the limited labeled CMRI data and low tolerances to irregular scales, shapes and deformations of LV. In this paper, we propose an automated LV segmentation method based on adversarial learning by integrating a multi-stage pose estimation network (MSPN) and a co-discrimination network. Different from existing CNNs, we use a MSPN with multi-scale dilated convolution (MDC) modules to enhance the ranges of receptive field for deep feature extraction. To fully utilize both labeled and unlabeled CMRI data, we propose a novel generative adversarial network (GAN) framework for LV segmentation by combining MSPN with co-discrimination networks. Specifically, the labeled CMRI are first used to initialize our segmentation network (MSPN) and co-discrimination network. Our GAN training includes two different kinds of epochs fed with both labeled and unlabeled CMRI data alternatively, which are different from the traditional CNNs only relied on the limited labeled samples to train the segmentation networks. As both ground truth and unlabeled samples are involved in guiding training, our method not only can converge faster but also obtain a better performance in LV segmentation. Our method is evaluated using MICCAI 2009 and 2017 challenge databases. Experimental results show that our method has obtained promising performance in LV segmentation, which also outperforms the state-of-the-art methods in terms of LV segmentation accuracy from the comparison results.     

A Novel Elastographic Frame Quality Indicator and its use in Automatic Representative-Frame Selection from a Cine Loop

This study was aimed at developing a method for automatically selecting a few representative frames from several hundred axial-shear strain elastogram frames typically obtained during freehand compression elastography of the breast in vivo. This may also alleviate some inter-observer variations that arise at least partly because of differences in selection of representative frames from a cine loop for evaluation and feature extraction. In addition to the correlation coefficient and frame-average axial strain that have been previously used as quality indicators for axial strain elastograms, we incorporated the angle of compression, which has unique effects on axial-shear strain elastogram interpretation. These identified quality factors were computed for every frame in the elastographic cine loop. The algorithm identifies the section having N contiguous frames (N = 10) that possess the highest cumulative quality scores from the cine loop as the one containing representative frames. Data for total of 40 biopsy-proven malignant or benign breast lesions in vivo were part of this study. The performance of the automated algorithm was evaluated by comparing its selection against that by trained radiologists. The observer- identified frame that consisted of a sonogram, axial strain elastogram and axial-shear strain elastogram was compared with the respective images in the frames of the algorithm-identified section using cross-correlation as a similarity measure. It was observed that there was, on average (～standard deviation), 82.2% (～2.2%), 83.4% (～3.8%) and 78.4% (～3.6%) correlation between corresponding images of the observer-selected and algorithm-selected frames, respectively. The results indicate that the automatic frame selection method described here may provide an objective way to select a representative frame while saving time for the radiologist. Furthermore, the frame quality metric described and used here can be displayed in real time as feedback to guide elastographic data acquisition and for training purposes.     

Robust endoscopic image mosaicking via fusion of multimodal estimation

We propose an endoscopic image mosaicking algorithm that is robust to light conditioning changes, specular reflections, and feature-less scenes. These conditions are especially common in minimally invasive surgery where the light source moves with the camera to dynamically illuminate close range scenes. This makes it difficult for a single image registration method to robustly track camera motion and then generate consistent mosaics of the expanded surgical scene across different and heterogeneous environments. Instead of relying on one specialised feature extractor or image registration method, we propose to fuse different image registration algorithms according to their uncertainties, formulating the problem as affine pose graph optimisation. This allows to combine landmarks, dense intensity registration, and learning-based approaches in a single framework. To demonstrate our application we consider deep learning-based optical flow, hand-crafted features, and intensity-based registration, however, the framework is general and could take as input other sources of motion estimation, including other sensor modalities. We validate the performance of our approach on three datasets with very different characteristics to highlighting its generalisability, demonstrating the advantages of our proposed fusion framework. While each individual registration algorithm eventually fails drastically on certain surgical scenes, the fusion approach flexibly determines which algorithms to use and in which proportion to more robustly obtain consistent mosaics.     

EndoSLAM dataset and an unsupervised monocular visual odometry and depth estimation approach for endoscopic videos

Deep learning techniques hold promise to develop dense topography reconstruction and pose estimation methods for endoscopic videos. However, currently available datasets do not support effective quantitative benchmarking. In this paper, we introduce a comprehensive endoscopic SLAM dataset consisting of 3D point cloud data for six porcine organs, capsule and standard endoscopy recordings, synthetically generated data as well as clinically in use conventional endoscope recording of the phantom colon with computed tomography(CT) scan ground truth. A Panda robotic arm, two commercially available capsule endoscopes, three conventional endoscopes with different camera properties, two high precision 3D scanners, and a CT scanner were employed to collect data from eight ex-vivo porcine gastrointestinal (GI)-tract organs and a silicone colon phantom model. In total, 35 sub-datasets are provided with 6D pose ground truth for the ex-vivo part: 18 sub-datasets for colon, 12 sub-datasets for stomach, and 5 sub-datasets for small intestine, while four of these contain polyp-mimicking elevations carried out by an expert gastroenterologist. To verify the applicability of this data for use with real clinical systems, we recorded a video sequence with a state-of-the-art colonoscope from a full representation silicon colon phantom. Synthetic capsule endoscopy frames from stomach, colon, and small intestine with both depth and pose annotations are included to facilitate the study of simulation-to-real transfer learning algorithms. Additionally, we propound Endo-SfMLearner, an unsupervised monocular depth and pose estimation method that combines residual networks with a spatial attention module in order to dictate the network to focus on distinguishable and highly textured tissue regions. The proposed approach makes use of a brightness-aware photometric loss to improve the robustness under fast frame-to-frame illumination changes that are commonly seen in endoscopic videos. To exemplify the use-case of the EndoSLAM dataset, the performance of Endo-SfMLearner is extensively compared with the state-of-the-art: SC-SfMLearner, Monodepth2, and SfMLearner. The codes and the link for the dataset are publicly available at https://github.com/CapsuleEndoscope/EndoSLAM. A video demonstrating the experimental setup and procedure is accessible as Supplementary Video 1.     

Enhancing MR image segmentation with realistic adversarial data augmentation

The success of neural networks on medical image segmentation tasks typically relies on large labeled datasets for model training. However, acquiring and manually labeling a large medical image set is resource-intensive, expensive, and sometimes impractical due to data sharing and privacy issues. To address this challenge, we propose AdvChain, a generic adversarial data augmentation framework, aiming at improving both the diversity and effectiveness of training data for medical image segmentation tasks. AdvChain augments data with dynamic data augmentation, generating randomly chained photo-metric and geometric transformations to resemble realistic yet challenging imaging variations to expand training data. By jointly optimizing the data augmentation model and a segmentation network during training, challenging examples are generated to enhance network generalizability for the downstream task. The proposed adversarial data augmentation does not rely on generative networks and can be used as a plug-in module in general segmentation networks. It is computationally efficient and applicable for both low-shot supervised and semi-supervised learning. We analyze and evaluate the method on two MR image segmentation tasks: cardiac segmentation and prostate segmentation with limited labeled data. Results show that the proposed approach can alleviate the need for labeled data while improving model generalization ability, indicating its practical value in medical imaging applications.     

Real-time localization of articulated surgical instruments in retinal microsurgery

Real-time visual tracking of a surgical instrument holds great potential for improving the outcome of retinal microsurgery by enabling new possibilities for computer-aided techniques such as augmented reality and automatic assessment of instrument manipulation. Due to high magnification and illumination variations, retinal microsurgery images usually entail a high level of noise and appearance changes. As a result, real-time tracking of the surgical instrument remains challenging in in-vivo sequences. To overcome these problems, we present a method that builds on random forests and addresses the task by modelling the instrument as an articulated object. A multi-template tracker reduces the region of interest to a rectangular area around the instrument tip by relating the movement of the instrument to the induced changes on the image intensities. Within this bounding box, a gradient-based pose estimation infers the location of the instrument parts from image features. In this way, the algorithm does not only provide the location of instrument, but also the positions of the tool tips in real-time. Various experiments on a novel dataset comprising 18 in-vivo retinal microsurgery sequences demonstrate the robustness and generalizability of our method. The comparison on two publicly available datasets indicates that the algorithm can outperform current state-of-the art.     

High frame rate video mosaicking microendoscope to image large regions of intact tissue with subcellular resolution

High-resolution microendoscopy (HRME) is a low-cost strategy to acquire images of intact tissue with subcellular resolution at frame rates ranging from 11 to 18 fps. Current HRME imaging strategies are limited by the small microendoscope field of view (∼0.5 mm²); multiple images must be acquired and reliably registered to assess large regions of clinical interest. Image mosaics have been assembled from co-registered frames of video acquired as a microendoscope is slowly moved across the tissue surface, but the slow frame rate of previous HRME systems made this approach impractical for acquiring quality mosaicked images from large regions of interest. Here, we present a novel video mosaicking microendoscope incorporating a high frame rate CMOS sensor and optical probe holder to enable high-speed, high quality interrogation of large tissue regions of interest. Microendoscopy videos acquired at >90 fps are assembled into an image mosaic. We assessed registration accuracy and image sharpness across the mosaic for images acquired with a handheld probe over a range of translational speeds. This high frame rate video mosaicking microendoscope enables in vivo probe translation at >15 millimeters per second while preserving high image quality and accurate mosaicking, increasing the size of the region of interest that can be interrogated at high resolution from 0.5 mm² to >30 mm². Real-time deployment of this high-frame rate system is demonstrated in vivo and source code made publicly available.     

Automatic brain labeling via multi-atlas guided fully convolutional networks

Multi-atlas-based methods are commonly used for MR brain image labeling, which alleviates the burdening and time-consuming task of manual labeling in neuroimaging analysis studies. Traditionally, multi-atlas-based methods first register multiple atlases to the target image, and then propagate the labels from the labeled atlases to the unlabeled target image. However, the registration step involves non-rigid alignment, which is often time-consuming and might lack high accuracy. Alternatively, patch-based methods have shown promise in relaxing the demand for accurate registration, but they often require the use of hand-crafted features. Recently, deep learning techniques have demonstrated their effectiveness in image labeling, by automatically learning comprehensive appearance features from training images. In this paper, we propose a multi-atlas guided fully convolutional network (MA-FCN) for automatic image labeling, which aims at further improving the labeling performance with the aid of prior knowledge from the training atlases. Specifically, we train our MA-FCN model in a patch-based manner, where the input data consists of not only a training image patch but also a set of its neighboring (i.e., most similar) affine-aligned atlas patches. The guidance information from neighboring atlas patches can help boost the discriminative ability of the learned FCN. Experimental results on different datasets demonstrate the effectiveness of our proposed method, by significantly outperforming the conventional FCN and several state-of-the-art MR brain labeling methods.     

Detection of cardiac cycle from intracoronary ultrasound

In this paper, we describe a method automatically to determine the phase of a cardiac cycle for each video frame of an intravascular ultrasound (IVUS) video recorded in vivo. We first review the principle of IVUS video and demonstrate the general applicability of our method. We show that the pulsating heart leads to phasic changes in image content of an IVUS video. With an image processing method, we can reverse this process and reliably extract the heart-beat phase directly from IVUS video. With the phase information, we demonstrate that we can build 3-D (3D) time-variant shapes and measure lumen volume changes within a cardiac cycle. We may also measure the changes of IVUS imaging probe off-center vector within a cardiac cycle, which serves as an indicator of vessel center-line curvature. The cardiac cycle extraction algorithm requires one scan of the IVUS video frames and takes O(n) time to complete, n being the total number of the video frames. The advantage of this method is that it requires no user interaction and no hardware set-up and can be applied to coronary scans of live beating hearts. The extracted heart-beat rate, compared with clinical recordings, has less than 1% error.