A multiresolution diffused expectation-maximization algorithm for medical image segmentation

In this paper a new method for segmenting medical images is presented, the multiresolution diffused expectation-maximization (MDEM) algorithm. The algorithm operates within a multiscale framework, thus taking advantage of the fact that objects/regions to be segmented usually reside at different scales. At each scale segmentation is carried out via the expectation-maximization algorithm, coupled with anisotropic diffusion on classes, in order to account for the spatial dependencies among pixels. This new approach is validated via experiments on a variety of medical images and its performance is compared with more standard methods.     

Authentication and Self-Correction in Sequential MRI Slices

One of the new challenges of Information Technology in the medical world is the protection and authentication of a variety of digital medical files, datasets, and images. In this work, the ability of magnetic resonance imaging (MRI) slice sequences to hide digital data is investigated and more specifically the case that the hidden data are the regions of interest (ROI) of the MRI slices. The regions of non-interest (RONI) are used as cover. The hiding capacity of the whole sequence is taken into account. Any ROI-targeted tampering attempt can be detected, and the original image can be self-restored (under certain conditions) by extracting the ROI from the RONI.     

Wavelet-based resolution recovery using an anatomical prior provides quantitative recovery for human population phantom PET [¹¹C]raclopride data

The objective of this study was to evaluate a resolution recovery (RR) method using a variety of simulated human brain [11C]raclopride positron emission tomography (PET) images. Simulated datasets of 15 numerical human phantoms were processed by a wavelet-based RR method using an anatomical prior. The anatomical prior was in the form of a hybrid segmented atlas, which combined an atlas for anatomical labelling and a PET image for functional labelling of each anatomical structure. We applied RR to both 60 min static and dynamic PET images. Recovery was quantified in 84 regions, comparing the typical 'true' value for the simulation, as obtained in normal subjects, simulated and RR PET images. The radioactivity concentration in the white matter, striatum and other cortical regions was successfully recovered for the 60 min static image of all 15 human phantoms; the dependence of the solution on accurate anatomical information was demonstrated by the difficulty of the technique to retrieve the subthalamic nuclei due to mismatch between the two atlases used for data simulation and recovery. Structural and functional synergy for resolution recovery (SFS-RR) improved quantification in the caudate and putamen, the main regions of interest, from?-30.1% and?-26.2% to?-17.6% and?-15.1%, respectively, for the 60 min static image and from?-51.4% and?-38.3% to?-27.6% and?-20.3% for the binding potential (BP(ND)) image, respectively. The proposed methodology proved effective in the RR of small structures from brain [11C]raclopride PET images. The improvement is consistent across the anatomical variability of a simulated population as long as accurate anatomical segmentations are provided.     

Emotional experience modulates brain activity during fixation periods between tasks

Functional imaging studies have begun to identify a set of brain regions whose brain activity is greater during 'rest' (e.g., fixation) states than during cognitive tasks. It has been posited that these regions constitute a network that supports the brain's default mode, which is temporarily suspended during specific goal-directed behaviors. Exogenous tasks that require cognitive effort are thought to command reallocation of resources away from the brain's default state. However, it remains unknown if brain activity during fixation periods between active task periods is influenced by previous task-related emotional content. We examined brain activity during periods of FIXATION (viewing and rating gray-scale images) interspersed among periods of viewing and rating complex images ('PICTURE') with positive, negative, and neutral affective content. We show that a selected group of brain regions (PCC, precuneus, IPL, vACC) do exhibit activity that is greater during FIXATION (>PICTURE); these regions have previously been implicated in the "default brain network". In addition, we report that activity within precuneus and IPL in the FIXATION period is attenuated by the precedent processing of images with positive and negative emotional content, relative to non-emotional content. These data suggest that the activity within regions implicated in the default network is modulated by the presence of environmental stimuli with motivational salience and, thus, adds to our understanding of the brain function during periods of low cognitive, emotional, or sensory demand.     

Diffraction enhanced imaging of controlled defects within bone, including bone-metal gaps

Gap regions between a bone and an implant, whether existing upon insertion or developing over time, can lead to implant failure. Currently, planar x-ray imaging and CT are the most commonly used methods to evaluate the gap region. An alternative to these available clinical imaging modalities could help to better evaluate bone resorption. Previous experiments with diffraction enhanced imaging (DEI) have shown significant contrast advantages over monochromatic synchrotron radiation (SR) imaging. DEI and planar SR radiography images of bone samples with drill holes and gap regions of known geometry were acquired at the NSLS beamline X15A (Upton, NY, USA). The images acquired with DEI show measurable contrast-to-noise gains when compared to the images acquired using SR radiography.     

基于多模态磁共振图像特征选择的脑胶质瘤分割

脑胶质瘤分割通常需要将肿瘤区域细分为多个不同性质的子区域,往往需要使用多种不同模态的磁共振(MR)图像。近年来,基于深度学习的脑胶质瘤分割研究已成为主流。然而,大多数基于深度学习的方法只是将不同模态MR图像(或底层特征)进行通道维度堆叠后输入到分割网络中,并且在特征提取阶段忽略不同性质子区域分割时所需模态特征的差异性,导致分割性能不够精良。本研究提出一种基于多模态MR图像特征选择的两阶段分割框架进行脑胶质瘤分割。一方面,设计多模态特征选择模块并嵌入到分割网络框架中,对当前分割任务所需多模态MR图像特征进行自动提取和有效选择;另一方面,将多个不同性质的病变组织子区域分为两阶段分割任务,利用第一阶段分割任务结果提供第二阶段分割目标的定位信息。本方法和对比方法分别在BraTS2018(训练集285个患者,验证集66个患者)、BraTS2019(训练集335个患者,验证集125个患者)和BraTS2020(训练集369个患者,验证集125个患者)公开数据集上进行了实验。在BraTS2018数据集上,本方法在完整肿瘤、肿瘤核心和增强肿瘤区域的Dice相似系数分别为0.898、0.854和0.818,Hausdorff距离分别为4.072、6.179和3.763;在BraTS2019数据集上,本方法在上述3个肿瘤区域的Dice相似系数分别为0.892、0.839和0.800,Hausdorff距离分别为6.168、7.077和3.807;在BraTS2020数据集上,本方法在上述3个肿瘤区域的Dice相似系数分别为0.896、0.837和0.803,Hausdorff距离分别为6.223、7.033和4.411。对比实验结果表明,所提方法在增强肿瘤区域和肿瘤核心区域的分割性能具有明显优势,特别是增强肿瘤区域分割性能在BraTS2020数据集上最佳。基于多模态特征选择模块的两阶段分割框架,针对每阶段分割目标实现了不同模态MR图像特征的自动和充分学习,取得了理想的分割结果,为计算机辅助肿瘤诊断提供了可能的解决方案。     

Contorted and ordinary body postures in the human brain

Social interaction and comprehension of non-verbal behaviour requires a representation of people’s bodies. Research into the neural underpinnings of body representation implicates several brain regions including extrastriate and fusiform body areas (EBA and FBA), superior temporal sulcus (STS), inferior frontal gyrus (IFG) and inferior parietal lobule (IPL). The different roles played by these regions in parsing familiar and unfamiliar body postures remain unclear. We examined the responses of this body observation network to static images of ordinary and contorted postures by using a repetition suppression design in functional neuroimaging. Participants were scanned whilst observing static images of a contortionist or a group of objects in either ordinary or unusual configurations, presented from different viewpoints. Greater activity emerged in EBA and FBA when participants viewed contorted compared to ordinary body postures. Repeated presentation of the same posture from different viewpoints lead to suppressed responses in the fusiform gyrus as well as three regions that are characteristically activated by observing moving bodies, namely STS, IFG and IPL. These four regions did not distinguish the image viewpoint or the plausibility of the posture. Together, these data define a broad cortical network for processing static body postures, including regions classically associated with action observation.     

A finite element method to correct deformable image registration errors in low-contrast regions

Image-guided adaptive radiotherapy requires deformable image registration to map radiation dose back and forth between images. The purpose of this study is to develop a novel method to improve the accuracy of an intensity-based image registration algorithm in low-contrast regions. A computational framework has been developed in this study to improve the quality of the 'demons' registration. For each voxel in the registration's target image, the standard deviation of image intensity in a neighborhood of this voxel was calculated. A mask for high-contrast regions was generated based on their standard deviations. In the masked regions, a tetrahedral mesh was refined recursively so that a sufficient number of tetrahedral nodes in these regions can be selected as driving nodes. An elastic system driven by the displacements of the selected nodes was formulated using a finite element method (FEM) and implemented on the refined mesh. The displacements of these driving nodes were generated with the 'demons' algorithm. The solution of the system was derived using a conjugated gradient method, and interpolated to generate a displacement vector field for the registered images. The FEM correction method was compared with the 'demons' algorithm on the computed tomography (CT) images of lung and prostate patients. The performance of the FEM correction relating to the 'demons' registration was analyzed based on the physical property of their deformation maps, and quantitatively evaluated through a benchmark model developed specifically for this study. Compared to the benchmark model, the 'demons' registration has the maximum error of 1.2 cm, which can be corrected by the FEM to 0.4 cm, and the average error of the 'demons' registration is reduced from 0.17 to 0.11 cm. For the CT images of lung and prostate patients, the deformation maps generated by the 'demons' algorithm were found unrealistic at several places. In these places, the displacement differences between the 'demons' registrations and their FEM corrections were found in the range of 0.4 and 1.1 cm. The mesh refinement and FEM simulation were implemented in a single thread application which requires about 45 min of computation time on a 2.6 GHz computer. This study has demonstrated that the FEM can be integrated with intensity-based image registration algorithms to improve their registration accuracy, especially in low-contrast regions.     

Human breast cancer in vitro: matching histo-pathology with small-angle x-ray scattering and diffraction enhanced x-ray imaging

Twenty-eight human breast tumour specimens were studied with small-angle x-ray scattering (SAXS), and 10 of those were imaged by the diffraction enhanced x-ray imaging (DEI) technique. The sample diameter was 20 mm and the thickness 1 mm. Two examples of ductal carcinoma are illustrated by histology images, DEI, and maps of the collagen d-spacing and scattered intensity in the Porod regime, which characterize the SAXS patterns from collagen-rich regions of the samples. Histo-pathology reveals the cancer-invaded regions, and the maps of the SAXS parameters show that in these regions the scattering signal differs significantly from scattering by the surrounding tissue, indicating a degradation of the collagen structure in the invaded regions. The DEI images show the borders between collagen and adipose tissue and provide a co-ordinate system for tissue mapping by SAXS. In addition, degradation of the collagen structure in an invaded region is revealed by fading contrast of the DEI refraction image. The 28 samples include fresh, defrosted tissue and formalin-fixed tissue. The d-values with their standard deviations are given. In the fresh samples there is a systematic 0.76% increase of the d-value in the invaded regions, averaged over 11 samples. Only intra-sample comparisons are made for the formalin-fixed samples, and with a long fixation time, the difference in the d-value stabilizes at about 0.7%. The correspondence between the DEI images, the SAXS maps and the histo-pathology suggests that definitive information on tumour growth and malignancy is obtained by combining these x-ray methods.     

Improved compressed sensing-based cone-beam CT reconstruction using adaptive prior image constraints

Volumetric cone-beam CT (CBCT) images are acquired repeatedly during a course of radiation therapy and a natural question to ask is whether CBCT images obtained earlier in the process can be utilized as prior knowledge to reduce patient imaging dose in subsequent scans. The purpose of this work is to develop an adaptive prior image constrained compressed sensing (APICCS) method to solve this problem. Reconstructed images using full projections are taken on the first day of radiation therapy treatment and are used as prior images. The subsequent scans are acquired using a protocol of sparse projections. In the proposed APICCS algorithm, the prior images are utilized as an initial guess and are incorporated into the objective function in the compressed sensing (CS)-based iterative reconstruction process. Furthermore, the prior information is employed to detect any possible mismatched regions between the prior and current images for improved reconstruction. For this purpose, the prior images and the reconstructed images are classified into three anatomical regions: air, soft tissue and bone. Mismatched regions are identified by local differences of the corresponding groups in the two classified sets of images. A distance transformation is then introduced to convert the information into an adaptive voxel-dependent relaxation map. In constructing the relaxation map, the matched regions (unchanged anatomy) between the prior and current images are assigned with smaller weight values, which are translated into less influence on the CS iterative reconstruction process. On the other hand, the mismatched regions (changed anatomy) are associated with larger values and the regions are updated more by the new projection data, thus avoiding any possible adverse effects of prior images. The APICCS approach was systematically assessed by using patient data acquired under standard and low-dose protocols for qualitative and quantitative comparisons. The APICCS method provides an effective way for us to enhance the image quality at the matched regions between the prior and current images compared to the existing PICCS algorithm. Compared to the current CBCT imaging protocols, the APICCS algorithm allows an imaging dose reduction of 10-40 times due to the greatly reduced number of projections and lower x-ray tube current level coming from the low-dose protocol.     

Computer-Aided Detection and Diagnosis at the Start of the Third Millennium

Computer-aided diagnosis has been under development for more than 3 decades. The rate of progress appears exponential, with either recent approval or pending approval for devices focusing on mammography, chest radiographs, and chest CT. Related technologies improve diagnosis for many other types of medical images including virtual colonography, vascular imaging, as well as automated quantitation of image-derived metrics. A variety of techniques are currently employed with success, likely reflecting the variety of imagery used, as well as the variety of tasks. Most areas of medical imaging have had efforts at computer assistance, and some have even received FDA approval and can be reimbursed. We anticipate that the rapid advance of these technologies will continue, and that application will broaden to cover much of medical imaging. Acceptance of, and integration of computer-aided diagnosis technology with the electronic radiology practice is a current challenge. These challenges will be overcome, and we expect that computer-aided diagnosis will be routinely applied to medical images.     

System for the analysis and visualization of large 3D anatomical trees

Modern micro-CT and multi-detector helical CT scanners can produce high-resolution 3D digital images of various anatomical trees. The large size and complexity of these trees make it essentially impossible to define them interactively. Automatic approaches have been proposed for a few specific problems, but none of these approaches guarantee extracting geometrically accurate multi-generational tree structures. This paper proposes an interactive system for defining and visualizing large anatomical trees and for subsequent quantitative data mining. The system consists of a large number of tools for automatic image analysis, semi-automatic and interactive tree editing, and an assortment of visualization tools. Results are presented for a variety of 3D high-resolution images.     

Registration of magnetic resonance spectroscopic imaging to computed tomography for radiotherapy treatment planning.

The incorporation of multiple imaging modalities into radiotherapy treatment planning offers the potential to improve identification of regions of pathology. This work outlines and evaluates a methodology for registration of magnetic resonance images (MRI) and spectroscopic images (MRSI) to computed tomography (CT) images, and visualization of the multimodality data on the treatment planning workstation. Volumetric magnetic resonance images were acquired during an examination prior to the initiation of radiotherapy. Registration between these images and the treatment planning computed tomography images was performed using an automated alignment routine, and was improved manually using an interactive registration tool. The parameters of the alignment were then used to transform the spectroscopic images into the same reference frame. The spectroscopy data were represented in terms of a statistical measure of abnormality, and embedded within the MRI data as overlaid contours. These images were sent via DICOM transfer to the treatment planning workstation. An analysis of the reproducibility of the     

Detection of Hard Exudates in Retinal Images Using a Radial Basis Function Classifier

Diabetic retinopathy (DR) is one of the most important causes of visual impairment. Automatic recognition of DR lesions, like hard exudates (EXs), in retinal images can contribute to the diagnosis and screening of the disease. The aim of this study was to automatically detect these lesions in fundus images. To achieve this goal, each image was normalized and the candidate EX regions were segmented by a combination of global and adaptive thresholding. Then, a group of features was extracted from image regions and the subset which best discriminated between EXs and retinal background was selected by means of logistic regression (LR). This optimal subset was subsequently used as input to a radial basis function (RBF) neural network. To improve the performance of the proposed algorithm, some noisy regions were eliminated by an innovative postprocessing of the image. The main novelty of the paper is the use of LR in conjunction with RBF and the proposed postprocessing technique. Our database was composed of 117 images with variable color, brightness and quality. The database was divided into a training set of 50 images (from DR patients) and a test set of 67 images (40 from DR patients and 27 from healthy retinas). Using a lesion-based criterion (pixel resolution), a mean sensitivity of 92.1% and a mean positive predictive value of 86.4% were obtained. With an image-based criterion, a mean sensitivity of 100%, mean specificity of 70.4% and mean accuracy of 88.1% were achieved. These results suggest that the proposed method could be a diagnostic aid for ophthalmologists in the screening for DR.     

Visual scene processing in familiar and unfamiliar environments

Humans and animals use information obtained from the local visual scene to orient themselves in the wider world. Although neural systems involved in scene perception have been identified, the extent to which processing in these systems is affected by previous experience is unclear. We addressed this issue by scanning subjects with functional magnetic resonance imaging (fMRI) while they viewed photographs of familiar and unfamiliar locations. Scene-selective regions in parahippocampal cortex (the parahippocampal place area, or PPA), retrosplenial cortex (RSC), and the transverse occipital sulcus (TOS) responded more strongly to images of familiar locations than to images of unfamiliar locations with the strongest effects (>50% increase) in RSC. Examination of fMRI repetition suppression (RS) effects indicated that images of familiar and unfamiliar locations were processed with the same degree of viewpoint specificity; however, increased viewpoint invariance was observed as individual scenes became more familiar over the course of a scan session. Surprisingly, these within-scan-session viewpoint-invariant RS effects were only observed when scenes were repeated across different trials but not when scenes were repeated within a trial, suggesting that within- and between-trial RS effects may index different aspects of visual scene processing. The sensitivity to environmental familiarity observed in the PPA, RSC, and TOS supports earlier claims that these regions mediate the extraction of navigationally relevant spatial information from visual scenes. As locations become familiar, the neural representations of these locations become enriched, but the viewpoint invariance of these representations does not change.     

Comparative evaluation of image segmentation methods for volume quantitation in SPECT.

A wide variety of image segmentation techniques have been proposed for the measurement of organ or lesion volumes in SPECT images. Evaluation of the relative performance of the various methods is difficult due to wide variations in system response characteristics, size, shape, and contrast of the imaged objects, and image acquisition and processing techniques. Selected image segmentation methods for volume quantitation in SPECT were applied to a set of simulated SPECT images containing objects ranging in volume from 1.8 to 113.1 cc. The specific segmentation methods included: (1) operator drawn regions of interest, (2) count-based methods, (3) three levels of fixed thresholds, (4) an adaptive threshold (GLH method), (5) a two-dimensional (2-D) edge detection method, and (6) a three-dimensional (3-D) edge detection method. In general, the 3-D edge detection method required minimal operator intervention while providing the most accurate and consistent estimates of object volume across changes in object contrast and size.     

Medical image registration

Radiological images are increasingly being used in healthcare and medical research. There is, consequently, widespread interest in accurately relating information in the different images for diagnosis, treatment and basic science. This article reviews registration techniques used to solve this problem, and describes the wide variety of applications to which these techniques are applied. Applications of image registration include combining images of the same subject from different modalities, aligning temporal sequences of images to compensate for motion of the subject between scans, image guidance during interventions and aligning images from multiple subjects in cohort studies. Current registration algorithms can, in many cases, automatically register images that are related by a rigid body transformation (i.e. where tissue deformation can be ignored). There has also been substantial progress in non-rigid registration algorithms that can compensate for tissue deformation, or align images from different subjects. Nevertheless many registration problems remain unsolved, and this is likely to continue to be an active field of research in the future.     

Laterality Classification of Fundus Images Using Interpretable Deep Neural Network

In this paper, we aimed to understand and analyze the outputs of a convolutional neural network model that classifies the laterality of fundus images. Our model not only automatizes the classification process, which results in reducing the labors of clinicians, but also highlights the key regions in the image and evaluates the uncertainty for the decision with proper analytic tools. Our model was trained and tested with 25,911 fundus images (43.4% of macula-centered images and 28.3% each of superior and nasal retinal fundus images). Also, activation maps were generated to mark important regions in the image for the classification. Then, uncertainties were quantified to support explanations as to why certain images were incorrectly classified under the proposed model. Our model achieved a mean training accuracy of 99%, which is comparable to the performance of clinicians. Strong activations were detected at the location of optic disc and retinal blood vessels around the disc, which matches to the regions that clinicians attend when deciding the laterality. Uncertainty analysis discovered that misclassified images tend to accompany with high prediction uncertainties and are likely ungradable. We believe that visualization of informative regions and the estimation of uncertainty, along with presentation of the prediction result, would enhance the interpretability of neural network models in a way that clinicians can be benefitted from using the automatic classification system.     

胶囊内窥图像出血识别的BP神经网络算法

在大量的胶囊内窥图像中寻找出血区域或相关病理特征是一件非常费时费力的工作,使用计算机进行胶囊内窥图像出血区域智能检测是必然趋势。本文设计了一种BP人工神经网络应用于内窥图像出血模式的识别,并通过软件编程实现了基于BP神经网络的内窥图像出血区域智能检测的新方法。实验表明该方法能正确检测出内窥图像中的出血区域,从而将内窥图像分类为出血模式与非出血模式,达到了理想的效果。