Experimental determination of subjective similarity for pairs of clustered microcalcifications on mammograms: observer study results

Presentation of images of lesions similar to that of an unknown lesion might be useful to radiologists in distinguishing between benign and malignant clustered microcalcifications on mammograms. Investigators have been developing computerized schemes to select similar images from large databases. However, whether selected images are really similar in appearance is not examined for most of the schemes. In order to retrieve images that are useful to radiologists, the selected images must be similar from radiologists' diagnostic points of view. Therefore, in this study, the data of radiologists' subjective similarity for pairs of clustered microcalcification images were obtained from a number of observers, and the intra- and inter-observer variations and the intergroup correlations were determined to investigate whether reliable similarity ratings by human observers can be determined. Nineteen images of clustered microcalcifications, each of which was paired with six other images, were selected for the observer study. Thus, subjective similarity ratings for 114 pairs of clustered microcalcifications were determined by each observer. Thirteen breast, ten general, and ten nonradiologists participated in the observer study; some of them completed the study multiple times. Although the intraobserver variations for the individual readings and the interobserver variations for pairs of observers were not small, the interobserver agreements were improved by taking the average of readings by the same observers. When the similarity ratings by a number of observers were averaged among the groups of breast, general, and nonradiologists, the mean differences of the ratings between the groups decreased, and good concordance correlations (0.846, 0.817, and 0.785) between the groups were obtained. The result indicates that reliable similarity ratings can be determined by use of this method, and the average similarity ratings by breast radiologists can be considered meaningful and useful for the development and evaluation of a computerized scheme for selection of similar images.     

Computer-aided detection system for clustered microcalcifications: comparison of performance on full-field digital mammograms and digitized screen-film mammograms

We have developed a computer-aided detection (CAD) system to detect clustered microcalcifications automatically on full-field digital mammograms (FFDMs) and a CAD system for screen-film mammograms (SFMs). The two systems used the same computer vision algorithms but their false positive (FP) classifiers were trained separately with sample images of each modality. In this study, we compared the performance of the CAD systems for detection of clustered microcalcifications on pairs of FFDM and SFM obtained from the same patient. For case-based performance evaluation, the FFDM CAD system achieved detection sensitivities of 70%, 80% and 90% at an average FP cluster rate of 0.07, 0.16 and 0.63 per image, compared with an average FP cluster rate of 0.15, 0.38 and 2.02 per image for the SFM CAD system. The difference was statistically significant with the alternative free-response receiver operating characteristic (AFROC) analysis. When evaluated on data sets negative for microcalcification clusters, the average FP cluster rates of the FFDM CAD system were 0.04, 0.11 and 0.33 per image at detection sensitivity level of 70%, 80% and 90% compared with an average FP cluster rate of 0.08, 0.14 and 0.50 per image for the SFM CAD system. When evaluated for malignant cases only, the difference of the performance of the two CAD systems was not statistically significant with AFROC analysis.     

Computer-aided detection of clustered microcalcifications on digital mammograms

A computer-aided diagnosis scheme to assist radiologists in detecting clustered microcalcifications from mammograms is being developed. Starting with a digital mammogram, the scheme consists of three steps. First, the image is filtered so that the signal-to-noise ratio of microcalcifications is increased by suppression of the normal background structure of the breast. Secondly, potential microcalcifications are extracted from the filtered image with a series of three different techniques: a global thresholding based on the grey-level histogram of the full filtered image, an erosion operator for eliminating very small signals, and a local adaptive grey-level thresholding. Thirdly, some false-positive signals are eliminated by means of a texture analysis technique, and a non-linear clustering algorithm is then used for grouping the remaining signals. With this method, the scheme can detect approximately 85% of true clusters, with an average of two false clusters detected per image.     

Computer-aided diagnosis scheme for histological classification of clustered microcalcifications on magnification mammograms

The histological classification of clustered microcalcifications on mammograms can be difficult, and thus often require biopsy or follow-up. Our purpose in this study was to develop a computer-aided diagnosis scheme for identifying the histological classification of clustered microcalcifications on magnification mammograms in order to assist the radiologists' interpretation as a "second opinion." Our database consisted of 58 magnification mammograms, which included 35 malignant clustered microcalcifications (9 invasive carcinomas, 12 noninvasive carcinomas of the comedo type, and 14 noninvasive carcinomas of the noncomedo type) and 23 benign clustered microcalcifications (17 mastopathies and 6 fibroadenomas). The histological classifications of all clustered microcalcifications were proved by pathologic diagnosis. The clustered microcalcifications were first segmented by use of a novel filter bank and a thresholding technique. Five objective features on clustered microcalcifications were determined by taking into account subjective features that experienced the radiologists commonly use to identify possible histological classifications. The Bayes decision rule with five objective features was employed for distinguishing between five histological classifications. The classification accuracies for distinguishing between three malignant histological classifications were 77.8% (7/9) for invasive carcinoma, 75.0% (9/12) for noninvasive carcinoma of the comedo type, and 92.9% (13/14) for noninvasive carcinoma of the noncomedo type. The classification accuracies for distinguishing between two benign histological classifications were 94.1% (16/17) for mastopathy, and 100.0% (6/6) for fibroadenoma. This computerized method would be useful in assisting radiologists in their assessments of clustered microcalcifications.     

An automatic method for the identification and interpretation of clustered microcalcifications in mammograms.

We investigated a method for a fully automatic identification and interpretation process for clustered microcalcifications in mammograms. Mammographic films of 100 patients containing microcalcifications with known histology were digitized and preprocessed using standard techniques. Microcalcifications detected by an artificial neural network (ANN) were clustered and some cluster features served as the input of another ANN trained to differentiate between typical and atypical clusters, while others were fed into an ANN trained on typical clusters to evaluate these lesions. The measured sensitivity for the detection of grouped microcalcifications was 0.98. For the task of differentiation between typical and atypical clusters an Az value of 0.87 was computed, while for the diagnosis an Az value of 0.87 with a sensitivity of 0.97 and a specificity of 0.47 was obtained. The results show that a fully automatic computer system was developed for the identification and interpretation of clustered microcalcitications in mammograms with the ability to differentiate most benign lesions from malignant ones in an automatically selected subset of cases.     

Detection of single and clustered microcalcifications in mammograms using fractals models and neural networks

Microcalcifications (microCas) are often early signs of breast cancer. However, detecting them is a difficult visual task and recognizing malignant lesions is a complex diagnostic problem. In recent years, several research groups have been working to develop computer-aided diagnosis (CAD) systems for X-ray mammography. In this paper, we propose a method to detect and classify microcalcifications. In order to discover the presence of microCas clusters, particular attention is paid to the analysis of the spatial arrangement of detected lesions. A fractal model has been used to describe the mammographic image, thus, allowing the use of a matched filtering stage to enhance microcalcifications against the background. A region growing algorithm, coupled with a neural classifier, detects existing lesions. Subsequently, a second fractal model is used to analyze their spatial arrangement so that the presence of microcalcification clusters can be detected and classified. Reported results indicate that fractal models provide an adequate framework for medical image processing; consequently high correct classification rates are achieved.     

Computerized detection of clustered microcalcifications in digital mammograms: applications of artificial neural networks.

Artificial neural networks have been applied to the differentiation of actual "true" clusters from normal parenchymal patterns and also to the differentiation of actual clusters from false-positive clusters as reported by a computerized scheme for the detection of microcalcifications in digital mammograms. The differentiation was carried out in both the spatial and frequency domains. The performance of the neural networks was evaluated quantitatively by means of receiver operating characteristic (ROC) analysis. It was found that the networks could distinguish clustered microcalcifications from normal nonclustered areas in the frequency domain, and that they could eliminate approximately 50% of false-positive clusters of microcalcifications while preserving 95% of the positive clusters, when applied to the results of the automated detection scheme. A large, comprehensive training database is needed for neural networks to perform reliably in clinical situations.     

Characterization of clustered microcalcifications in digitized mammograms using neural networks and support vector machines

OBJECTIVE: Detection and characterization of microcalcification clusters in mammograms is vital in daily clinical practice. The scope of this work is to present a novel computer-based automated method for the characterization of microcalcification clusters in digitized mammograms. METHODS AND MATERIAL: The proposed method has been implemented in three stages: (a) the cluster detection stage to identify clusters of microcalcifications, (b) the feature extraction stage to compute the important features of each cluster and (c) the classification stage, which provides with the final characterization. In the classification stage, a rule-based system, an artificial neural network (ANN) and a support vector machine (SVM) have been implemented and evaluated using receiver operating characteristic (ROC) analysis. The proposed method was evaluated using the Nijmegen and Mammographic Image Analysis Society (MIAS) mammographic databases. The original feature set was enhanced by the addition of four rule-based features. RESULTS AND CONCLUSIONS: In the case of Nijmegen dataset, the performance of the SVM was Az=0.79 and 0.77 for the original and enhanced feature set, respectively, while for the MIAS dataset the corresponding characterization scores were Az=0.81 and 0.80. Utilizing neural network classification methodology, the corresponding performance for the Nijmegen dataset was Az=0.70 and 0.76 while for the MIAS dataset it was Az=0.73 and 0.78. Although the obtained high classification performance can be successfully applied to microcalcification clusters characterization, further studies must be carried out for the clinical evaluation of the system using larger datasets. The use of additional features originating either from the image itself (such as cluster location and orientation) or from the patient data may further improve the diagnostic value of the system.     

Evaluation of information-theoretic similarity measures for content-based retrieval and detection of masses in mammograms

The purpose of this study was to evaluate image similarity measures employed in an information-theoretic computer-assisted detection (IT-CAD) scheme. The scheme was developed for content-based retrieval and detection of masses in screening mammograms. The study is aimed toward an interactive clinical paradigm where physicians query the proposed IT-CAD scheme on mammographic locations that are either visually suspicious or indicated as suspicious by other cuing CAD systems. The IT-CAD scheme provides an evidence-based, second opinion for query mammographic locations using a knowledge database of mass and normal cases. In this study, eight entropy-based similarity measures were compared with respect to retrieval precision and detection accuracy using a database of 1820 mammographic regions of interest. The IT-CAD scheme was then validated on a separate database for false positive reduction of progressively more challenging visual cues generated by an existing, in-house mass detection system. The study showed that the image similarity measures fall into one of two categories; one category is better suited to the retrieval of semantically similar cases while the second is more effective with knowledge-based decisions regarding the presence of a true mass in the query location. In addition, the IT-CAD scheme yielded a substantial reduction in false-positive detections while maintaining high detection rate for malignant masses.     

Comparison of similarity measures for the task of template matching of masses on serial mammograms

We conducted a study to evaluate the effectiveness of twelve different similarity measures in matching the corresponding masses on temporal pairs of current and prior mammograms. To perform this comparison we implemented each of the twelve similarity measures in the final stage of our multistage registration technique for automated registration of breast lesions in serial mammograms. The multistage technique consists of three stages. In the first stage an initial fan-shape search region was estimated on the prior mammogram based on the geometrical position of the mass on the current mammogram. In the second stage, the location of the fan-shape region was refined by warping, based on an affine transformation and simplex optimization. A new refined search region was defined on the prior mammogram. In the third stage, a search for the best match between the lesion template from the current mammogram and a structure on the prior mammogram was carried out within the search region. Our data set consisted of 318 temporal pairs. We performed three experiments, using a different subset of the 318 temporal pairs for each experiment. In each experiment we further tested how the performance of the similarity measures varied as the size of the search region increased or decreased. We evaluated the twelve similarity measures based on four criteria. The first criterion was the mean Euclidean distance, which was the average distance of the true location of the mass to the location detected by the similarity measure. The second criterion was the percentage of temporal pairs that were aligned so that 50% or more of the lesion area overlapped. The third criterion was the percentage of pairs that were aligned so that 75% or more of the lesion area overlapped. The fourth and final criterion was the robustness of the similarity measure. Our results showed that three of the similarity measures, Pearson's correlation, the cosine coefficient, and Goodman and Kruskal's Gamma coefficient, provide significantly higher accuracy (p < 0.05) in the task of matching the corresponding masses on serial mammograms than the other nine similarity measures.     

Joint two-view information for computerized detection of microcalcifications on mammograms

We are developing new techniques to improve the accuracy of computerized microcalcification detection by using the joint two-view information on craniocaudal (CC) and mediolateral-oblique (MLO) views. After cluster candidates were detected using a single-view detection technique, candidates on CC and MLO views were paired using their radial distances from the nipple. Candidate pairs were classified with a similarity classifier that used the joint information from both views. Each cluster candidate was also characterized by its single-view features. The outputs of the similarity classifier and the single-view classifier were fused and the cluster candidate was classified as a true microcalcification cluster or a false-positive (FP) using the fused two-view information. A data set of 116 pairs of mammograms containing microcalcification clusters and 203 pairs of normal images from the University of South Florida (USF) public database was used for training the two-view detection algorithm. The trained method was tested on an independent test set of 167 pairs of mammograms, which contained 71 normal pairs and 96 pairs with microcalcification clusters collected at the University of Michigan (UM). The similarity classifier had a very low FP rate for the test set at low and medium levels of sensitivity. However, the highest mammogram-based sensitivity that could be reached by the similarity classifier was 69%. The single-view classifier had a higher FP rate compared to the similarity classifier, but it could reach a maximum mammogram-based sensitivity of 93%. The fusion method combined the scores of these two classifiers so that the number of FPs was substantially reduced at relatively low and medium sensitivities, and a relatively high maximum sensitivity was maintained. For the malignant microcalcification clusters, at a mammogram-based sensitivity of 80%, the FP rates were 0.18 and 0.35 with the two-view fusion and single-view detection methods, respectively. When the training and test sets were switched, a similar improvement was obtained, except that both the fusion and single-view detection methods had superior test performances on the USF data set than those on the UM data set. Our results indicate that correspondence of cluster candidates on two different views provides valuable additional information for distinguishing FPs from true microcalcification clusters.     

Receiver operating characteristic analysis for the detection of simulated microcalcifications on mammograms using hardcopy images

The aim of this study was to compare mammography systems based on three different detectors--a conventional screen-film (SF) combination, an a-Si/CsI flat-panel (FP)-based detector, and a charge-coupled device (CCD)-based x-ray phosphor-based detector--for their performance in detecting simulated microcalcifications (MCs). 112-150 microm calcium carbonate grains were used to simulate MCs and were overlapped with a slab phantom of simulated 50% adipose/50% glandular breast tissue-equivalent material referred to as the uniform background. For the tissue structure background, 200-250 microm calcium carbonate grains were used and overlapped with an anthropomorphic breast phantom. All MC phantom images were acquired with and without magnification (1.8 X). The hardcopy images were reviewed by five mammographers. A five-point confidence level rating was used to score each detection task. Receiver operating characteristic (ROC) analysis was performed, and the areas under the ROC curves (A(z)s) were used to compare the performances of the three mammography systems under various conditions. The results showed that, with a uniform background and contact images, the FP-based system performed significantly better than the SF and the CCD-based systems. For magnified images with a uniform background, the SF and the FP-based systems performed equally well and significantly better than the CCD-based system. With tissue structure background and contact images, the SF system performed significantly better than the FP and the CCD-based systems. With magnified images and a tissue structure background, the SF and the CCD-based systems performed equally well and significantly better than the FP-based system. In the detection of MCs in the fibroglandular and the heterogeneously dense regions, no significant differences were found except that the SF system performed significantly better than the CCD-based system in the fibroglandular regions for the contact images.     

Knowledge-based computer-aided detection of masses on digitized mammograms: a preliminary assessment

The purpose of this work was to develop and evaluate a computer-aided detection (CAD) scheme for the improvement of mass identification on digitized mammograms using a knowledge-based approach. Three hundred pathologically verified masses and 300 negative, but suspicious, regions, as initially identified by a rule-based CAD scheme, were randomly selected from a large clinical database for development purposes. In addition, 500 different positive and 500 negative regions were used to test the scheme. This suspicious region pruning scheme includes a learning process to establish a knowledge base that is then used to determine whether a previously identified suspicious region is likely to depict a true mass. This is accomplished by quantitatively characterizing the set of known masses, measuring "similarity" between a suspicious region and a "known" mass, then deriving a composite "likelihood" measure based on all "known" masses to determine the state of the suspicious region. To assess the performance of this method, receiver-operating characteristic (ROC) analyses were employed. Using a leave-one-out validation method with the development set of 600 regions, the knowledge-based CAD scheme achieved an area under the ROC curve of 0.83. Fifty-one percent of the previously identified false-positive regions were eliminated, while maintaining 90% sensitivity. During testing of the 1,000 independent regions, an area under the ROC curve as high as 0.80 was achieved. Knowledge-based approaches can yield a significant reduction in false-positive detections while maintaining reasonable sensitivity. This approach has the potential of improving the performance of other rule-based CAD schemes.     

Combined adaptive enhancement and region-growing segmentation of breast masses on digitized mammograms.

As an ongoing effort to develop a computer aid for detection of masses on mammograms, we recently designed an object-based region-growing technique to improve mass segmentation. This segmentation method utilizes the density-weighted contrast enhancement (DWCE) filter as a pre-processing step. The DWCE filter adaptively enhances the contrast between the breast structures and the background. Object-based region growing was then applied to each of the identified structures. The region-growing technique uses gray-scale and gradient information to adjust the initial object borders and to reduce merging between adjacent or overlapping structures. Each object is then classified as a breast mass or normal tissue based on extracted morphological and texture features. In this study we evaluated the sensitivity of this combined segmentation scheme and its ability to reduce false positive (FP) detections on a data set of 253 digitized mammograms, each of which contained a biopsy-proven breast mass. It was found that the segmentation scheme detected 98% of the 253 biopsy-proven breast masses in our data set. After final FP reduction, the detection resulted in 4.2 FP per image at a 90% true positive (TP) fraction and 2.0 FPs per image at an 80% TP fraction. The combined DWCE and object-based region growing technique increased the initial detection sensitivity, reduced merging between neighboring structures, and reduced the number of FP detections in our automated breast mass detection scheme.     

A comparison of global ratings and checklist scores from an undergraduate assessment using an anesthesia simulator.

PURPOSE: To determine the correlation between global ratings and criterion-based checklist scores, and inter-rater reliability of global ratings and criterion-based checklist scores, in a performance assessment using an anesthesia simulator. METHOD: All final-year medical students at the University of Toronto were invited to work through a 15-minute faculty-facilitated scenario using an anesthesia simulator. Students' performances were videotaped and analyzed by two faculty using a 25-point criterion-based checklist and a five-point global rating of competency (1 = clear failure, 5 = superior performance). Correlations between global ratings and checklist scores, as well as specific performance competencies (knowledge, technical skills, and judgment), were determined. Checklist and global scores were converted to percentages; means of the two marks were compared. Mean reliability of a single rater for both checklist and global ratings was determined. RESULTS: The correlation between checklist and global ratings was.74. Mean ratings of both checklist and global scores were low (58.67, SD = 14.96, and 57.08, SD = 24.27, respectively); these differences were not statistically significant. For a single rater, the mean reliability score across rater pairs for checklist scores was.77 (range.58-.93). Mean reliability score across rater pairs for global ratings was.62 (.40-.77). Global ratings correlated more highly with technical skills and judgment (r =.51 and r =.53, respectively) than with knowledge. (r =.24) CONCLUSION: Inter-rater reliability was higher for checklist scores than for global ratings; however, global ratings demonstrated acceptable inter-rater reliability and may be useful for competency assessment in performance assessments using simulators.     

Support vector methods for survival analysis: a comparison between ranking and regression approaches

Objective

To compare and evaluate ranking, regression and combined machine learning approaches for the analysis of survival data.

Methods

The literature describes two approaches based on support vector machines to deal with censored observations. In the first approach the key idea is to rephrase the task as a ranking problem via the concordance index, a problem which can be solved efficiently in a context of structural risk minimization and convex optimization techniques. In a second approach, one uses a regression approach, dealing with censoring by means of inequality constraints. The goal of this paper is then twofold: (i) introducing a new model combining the ranking and regression strategy, which retains the link with existing survival models such as the proportional hazards model via transformation models; and (ii) comparison of the three techniques on 6 clinical and 3 high-dimensional datasets and discussing the relevance of these techniques over classical approaches fur survival data.

Results

We compare svm-based survival models based on ranking constraints, based on regression constraints and models based on both ranking and regression constraints. The performance of the models is compared by means of three different measures: (i) the concordance index, measuring the model’s discriminating ability; (ii) the logrank test statistic, indicating whether patients with a prognostic index lower than the median prognostic index have a significant different survival than patients with a prognostic index higher than the median; and (iii) the hazard ratio after normalization to restrict the prognostic index between 0 and 1. Our results indicate a significantly better performance for models including regression constraints above models only based on ranking constraints.

Conclusions

This work gives empirical evidence that svm-based models using regression constraints perform significantly better than svm-based models based on ranking constraints. Our experiments show a comparable performance for methods including only regression or both regression and ranking constraints on clinical data. On high dimensional data, the former model performs better. However, this approach does not have a theoretical link with standard statistical models for survival data. This link can be made by means of transformation models when ranking constraints are included.     

Confidence intervals for test scores and significance tests for test score differences: a comparison of methods

Two methods of setting confidence intervals for test scores and testing the significance of test score differences are compared with respect to their simplicity and the similarity of their results. The conventional method, which is based on obtained scores, is unquestionably simpler than the technically correct method, which is based on estimated true scores, but the two methods may give rather different results, depending on the reliability coefficient(s) of the test(s), the distance of the score(s) from the mean, and the confidence or significance level. The standardization data for the WISC-R are used to illustrate the effects of those factors.     

Determination for different severity of anemia in thalassemia: concordance and discordance among sib pairs

The degree of anemia in beta(0)-thalassemia/hemoglobin E disease is highly variable. As part of an attempt to identify determinants of this variability of severity we studied concordance and discordance of hemoglobin levels among sib pairs. The distribution of differences of hemoglobin levels in 216 sib pairs from 98 families showed a remarkable skewness toward the lower values with a mode at 0-0.5 gm/dl. The prevailing concordance of hemoglobin levels in patients from the same families and the persistence of the patterns indicate that polygenic factors are mainly responsible for the variability of anemia in this disease.