Added value of deep learning-based computer-aided diagnosis and shear wave elastography to b-mode ultrasound for evaluation of breast masses detected by screening ultrasound

Low specificity and operator dependency are the main problems of breast ultrasound (US) screening. We investigated the added value of deep learning-based computer-aided diagnosis (S-Detect) and shear wave elastography (SWE) to B-mode US for evaluation of breast masses detected by screening US.Between February 2018 and June 2019, B-mode US, S-Detect, and SWE were prospectively obtained for 156 screening US-detected breast masses in 146 women before undergoing US-guided biopsy. S-Detect was applied for the representative B-mode US image, and quantitative elasticity was measured for SWE. Breast Imaging Reporting and Data System final assessment category was assigned for the datasets of B-mode US alone, B-mode US plus S-Detect, and B-mode US plus SWE by 3 radiologists with varied experience in breast imaging. Area under the receiver operator characteristics curve (AUC), sensitivity, and specificity for the 3 datasets were compared using Delong''s method and McNemar test.Of 156 masses, 10 (6%) were malignant and 146 (94%) were benign. Compared to B-mode US alone, the addition of S-Detect increased the specificity from 8%–9% to 31%–71% and the AUC from 0.541–0.545 to 0.658–0.803 in all radiologists (All P < .001). The addition of SWE to B-mode US also increased the specificity from 8%–9% to 41%–75% and the AUC from 0.541–0.545 to 0.709–0.823 in all radiologists (All P < .001). There was no significant loss in sensitivity when either S-Detect or SWE were added to B-mode US.Adding S-Detect or SWE to B-mode US improved the specificity and AUC without loss of sensitivity.     

Ultrasonographic morphological characteristics determined using a deep learning-based computer-aided diagnostic system of breast cancer

To investigate the correlations between ultrasonographic morphological characteristics quantitatively assessed using a deep learning-based computer-aided diagnostic system (DL-CAD) and histopathologic features of breast cancer.This retrospective study included 282 women with invasive breast cancer (<5 cm; mean age, 54.4 [range, 29–85] years) who underwent surgery between February 2016 and April 2017. The morphological characteristics of breast cancer on B-mode ultrasonography were analyzed using DL-CAD, and quantitative scores (0–1) were obtained. Associations between quantitative scores and tumor histologic type, grade, size, subtype, and lymph node status were compared.Two-hundred and thirty-six (83.7%) tumors were invasive ductal carcinoma, 18 (6.4%) invasive lobular carcinoma, and 28 (9.9%) micropapillary, apocrine, and mucinous. The mean size was 1.8 ± 1.0 (standard deviation) cm, and 108 (38.3%) cases were node positive. Irregular shape score was associated with tumor size (P < .001), lymph nodes status (P = .001), and estrogen receptor status (P = .016). Not-circumscribed margin (P < .001) and hypoechogenicity (P = .003) scores correlated with tumor size, and non-parallel orientation score correlated with histologic grade (P = .024). Luminal A tumors exhibited more irregular features (P = .048) with no parallel orientation (P = .002), whereas triple-negative breast cancer showed a rounder/more oval and parallel orientation.Quantitative morphological characteristics of breast cancers determined using DL-CAD correlated with histopathologic features and could provide useful information about breast cancer phenotypes.     

Multi-population generalizability of a deep learning-based chest radiograph severity score for COVID-19

To tune and test the generalizability of a deep learning-based model for assessment of COVID-19 lung disease severity on chest radiographs (CXRs) from different patient populations.A published convolutional Siamese neural network-based model previously trained on hospitalized patients with COVID-19 was tuned using 250 outpatient CXRs. This model produces a quantitative measure of COVID-19 lung disease severity (pulmonary x-ray severity (PXS) score). The model was evaluated on CXRs from 4 test sets, including 3 from the United States (patients hospitalized at an academic medical center (N = 154), patients hospitalized at a community hospital (N = 113), and outpatients (N = 108)) and 1 from Brazil (patients at an academic medical center emergency department (N = 303)). Radiologists from both countries independently assigned reference standard CXR severity scores, which were correlated with the PXS scores as a measure of model performance (Pearson R). The Uniform Manifold Approximation and Projection (UMAP) technique was used to visualize the neural network results.Tuning the deep learning model with outpatient data showed high model performance in 2 United States hospitalized patient datasets (R = 0.88 and R = 0.90, compared to baseline R = 0.86). Model performance was similar, though slightly lower, when tested on the United States outpatient and Brazil emergency department datasets (R = 0.86 and R = 0.85, respectively). UMAP showed that the model learned disease severity information that generalized across test sets.A deep learning model that extracts a COVID-19 severity score on CXRs showed generalizable performance across multiple populations from 2 continents, including outpatients and hospitalized patients.     

Inverting the structure–property map of truss metamaterials by deep learning

Inspired by crystallography, the periodic assembly of trusses into architected materials has enjoyed popularity for more than a decade and produced countless cellular structures with beneficial mechanical properties. Despite the successful and steady enrichment of the truss design space, the inverse design has remained a challenge: While predicting effective truss properties is now commonplace, efficiently identifying architectures that have homogeneous or spatially varying target properties has remained a roadblock to applications from lightweight structures to biomimetic implants. To overcome this gap, we propose a deep-learning framework, which combines neural networks with enforced physical constraints, to predict truss architectures with fully tailored anisotropic stiffness. Trained on millions of unit cells, it covers an enormous design space of topologically distinct truss lattices and accurately identifies architectures matching previously unseen stiffness responses. We demonstrate the application to patient-specific bone implants matching clinical stiffness data, and we discuss the extension to spatially graded cellular structures with locally optimal properties.

Opportunities for selecting materials in the engineering design process have fundamentally changed over the past decade due to innovation in materials systems with tailored properties. Specifically with maturing additive manufacturing techniques across length scales, metamaterials or architected materials have gained momentum, whose periodic or nonperiodic structural architecture on smaller scales has enabled us to control the characteristic material behavior on larger scales. Popular examples are periodic assemblies of truss, plate, or shell networks, whose underlying unit cell (UC) architecture can be leveraged to explore a tremendous design space, including previously unattainable effective material property and functionality combinations. This includes lightweight materials with high specific stiffness and strength (1–3), materials for acoustic wave guiding (4–6) and impact energy absorption (7) and further for heat transfer (8–10) and vibration control (11, 12), to mention but a few properties of interest. Beneficial structure–property relations have been identified by user intuition as well as by computational optimization (13) and by taking inspiration from natural cellular architectures (14). While the result is an ever-growing space of candidate UC architectures, the selection process for a specific application too often relies on lookup tables (15) rather than inverse design. The latter is the challenge of identifying architectures that possess specific effective properties on demand (as opposed to the well-understood forward problem of extracting effective properties from a given UC).To illustrate this challenge and, moreover, to offer a solution, we focus on one of the most fundamental effective properties, the elastic stiffness of a material, which in general is direction dependent or anisotropic. The elastic stiffness not only governs the linear stress–strain response, but also is essential for, e.g., wave motion, buckling, and limit loads. Anisotropic behavior is frequently encountered in nature, e.g., in bone (16). Consequently, material solutions for bone implants should ideally reproduce the mechanical and physiological properties of their natural analogs. This, in turn, requires finding cellular architectures whose effective anisotropic stiffness matches that of bone (which commonly varies not only from patient to patient but even from location to location within a specific bone). If unsuccessful, mismatches in stiffness can lead to stress shielding with detrimental consequences for bone atrophy (17, 18). Prior research has identified a myriad of architectures with tailored anisotropy—behaving stiff in some directions and compliant in others. Most popular for its simple fabrication and property extraction has been the class of truss lattices (13, 18–28). In principle, arbitrary thermodynamically admissible stiffness combinations can be achieved by the complex arrangement of trusses (20), which may be guided by topology optimization schemes (also referred to as inverse homogenization) (13). Most prior work, however, has relied on simple parametric studies (21, 29) (with a limited range of achievable [an]isotropy), structural optimization techniques (13, 27) (which are expensive in three dimensions [3D] and may not guarantee manufacturability), or the selection from a precomputed UC database (14, 15) (which becomes prohibitively large for high-dimensional parameter spaces). Moreover, identified topologies for different elastic stiffnesses are often incompatible and hence not continuously convertible into each other, which prevents their use in structures with spatially varying, locally optimized stiffness, such as in bone.In recent years, machine-learning (ML) algorithms such as deep neural networks (NNs) have gained attention to meet the inverse design challenge. Existing approaches of single- and multiscale topology optimization commonly leverage data-driven surrogate models for the structure-to-property map, which bypass expensive computational homogenization of the microscale UC and thereby accelerate structural optimization (30–34). Generative models based on variational autoencoders and generative adversarial networks typically search for optimal designs with target properties from within a continuous latent (design) space of reduced dimensionality (23, 28, 35–37). The biggest challenge in inverse-designing truss architectures is the lack of a unifying design parameterization describing the enormous set of truss lattices identified over the years. While a pixelated/voxelated microstructure representation has been successful for, e.g., composite materials (37–40), it is highly inefficient for sparse 3D truss UCs. Other approaches, like the library of tens of thousands of unique truss lattices introduced recently (14) with inspiration from molecular structures, do not admit a consistent design parameterization (unlike their molecular analogs). Consequently, existing works (23, 25, 30, 41) have typically considered only a small number of fixed lattice topologies, whose superposition with different strut thicknesses and/or base materials results in a limited design space for the effective properties—but with the added benefit of enabling spatial gradients. In addition, the common focus on cubic and hence orthotropic UCs (26, 35) ignores shear–normal and shear–shear coupling components in the effective stiffness tensor—although it has been recognized that those may be beneficial for, among others, compliance minimization and wave guidance (22, 42). By contrast, a completely random topology (based on a random placement and connection of struts in a truss) results in an overwhelmingly high-dimensional and nonlinear parameterization with low symmetry and a prohibitively small fraction of mechanically useful UCs (not even to think of smooth spatial transitions between different random topologies). More recently, graph neural networks (leveraging the analogy between trusses and graphs) have shown promising results in predicting the response of truss metamaterials (albeit in a semisupervised setting of the forward problem only), and their use for design optimization is the subject of current research (43).We here propose an ML-driven inverse design framework for the instant prediction of diverse truss lattices with fully tailorable 3D anisotropic stiffness. Our framework admits an enormous, unified design space of topologically distinct lattices with an efficient design parameterization. The property space spans several orders of magnitude in stiffness, including previously unexploited shear–shear and shear–normal couplings. The inverse design admits variational sampling to propose multiple distinct architectures that exhibit a given target stiffness response—all while maintaining the ability to smoothly transition between different UCs, which is an ongoing challenge for periodic structures (44).     

A stratified analysis of a deep learning algorithm in the diagnosis of diabetic retinopathy in a real‐world study

BackgroundThe aim of our research was to prospectively explore the clinical value of a deep learning algorithm (DLA) to detect referable diabetic retinopathy (DR) in different subgroups stratified by types of diabetes, blood pressure, sex, BMI, age, glycosylated hemoglobin (HbA1c), diabetes duration, urine albumin‐to‐creatinine ratio (UACR), and estimated glomerular filtration rate (eGFR) at a real‐world diabetes center in China.MethodsA total of 1147 diabetic patients from Shanghai General Hospital were recruited from October 2018 to August 2019. Retinal fundus images were graded by the DLA, and the detection of referable DR (moderate nonproliferative DR or worse) was compared with a reference standard generated by one certified retinal specialist with more than 12 years of experience. The performance of DLA across different subgroups stratified by types of diabetes, blood pressure, sex, BMI, age, HbA1c, diabetes duration, UACR, and eGFR was evaluated.ResultsFor all 1674 gradable images, the area under the receiver operating curve, sensitivity, and specificity of the DLA for referable DR were 0.942 (95% CI, 0.920‐0.964), 85.1% (95% CI, 83.4%‐86.8%), and 95.6% (95% CI, 94.6%‐96.6%), respectively. The DLA showed consistent performance across most subgroups, while it showed superior performance in the subgroups of patients with type 1 diabetes, UACR ≥ 30 mg/g, and eGFR < 90 mL/min/1.73m².ConclusionsThis study showed that the DLA was a reliable alternative method for the detection of referable DR and performed superior in patients with type 1 diabetes and diabetic nephropathy who were prone to DR.     

Adversarial attack on deep learning-based dermatoscopic image recognition systems: Risk of misdiagnosis due to undetectable image perturbations

Deep learning algorithms have shown excellent performances in the field of medical image recognition, and practical applications have been made in several medical domains. Little is known about the feasibility and impact of an undetectable adversarial attacks, which can disrupt an algorithm by modifying a single pixel of the image to be interpreted. The aim of the study was to test the feasibility and impact of an adversarial attack on the accuracy of a deep learning-based dermatoscopic image recognition system.First, the pre-trained convolutional neural network DenseNet-201 was trained to classify images from the training set into 7 categories. Second, an adversarial neural network was trained to generate undetectable perturbations on images from the test set, to classifying all perturbed images as melanocytic nevi. The perturbed images were classified using the model generated in the first step. This study used the HAM-10000 dataset, an open source image database containing 10,015 dermatoscopic images, which was split into a training set and a test set. The accuracy of the generated classification model was evaluated using images from the test set. The accuracy of the model with and without perturbed images was compared. The ability of 2 observers to detect image perturbations was evaluated, and the inter observer agreement was calculated.The overall accuracy of the classification model dropped from 84% (confidence interval (CI) 95%: 82–86) for unperturbed images to 67% (CI 95%: 65–69) for perturbed images (Mc Nemar test, P < .0001). The fooling ratio reached 100% for all categories of skin lesions. Sensitivity and specificity of the combined observers calculated on a random sample of 50 images were 58.3% (CI 95%: 45.9–70.8) and 42.5% (CI 95%: 27.2–57.8), respectively. The kappa agreement coefficient between the 2 observers was negative at -0.22 (CI 95%: −0.49–−0.04).Adversarial attacks on medical image databases can distort interpretation by image recognition algorithms, are easy to make and undetectable by humans. It seems essential to improve our understanding of deep learning-based image recognition systems and to upgrade their security before putting them to practical and daily use.     

A web-based Delphi study on the indications of chest radiographs for patients in ICUs

BACKGROUND: Strategies for ordering bedside chest radiographs (CXRs) have substantial logistic and financial consequences in the ICU. Many of the indications for CXRs in the ICU are controversial, such as the ordering of daily routine CXRs for intubated patients. The opinions of intensivists about ordering CXRs have not been reported. Comparing these opinions to established guidelines and identifying situations where opinions diverge in the absence of guidelines are of considerable interest. METHODS: We asked 190 intensivists from 34 ICUs in the area of Paris, France, to anonymously complete a 29-item questionnaire about their opinions regarding the ordering of CXRs; each item described a clinical scenario. Of the 29 scenarios, 10 dealt with the placement of medical devices, 8 with the presence of medical devices, and 11 with other clinical situations. The study was based on a Delphi process deployed over the Internet through an original software application. Three Delphi rounds were run between January and March 2006, using the same questionnaire. Detailed feedback for the answers given during the previous round was supplied to each intensivist solicited for updating his answers. RESULTS: Eighty-two intensivists from 32 ICUs completed the study. A consensus emerged that routine CXRs were necessary for eight scenarios and unnecessary for two scenarios. The study also shed light on items without a consensus. In particular, 75% of intensivists (58% on the first round) did not support obtaining daily routine CXRs in intubated patients. CONCLUSION: The study underlines situations in which intensivists do not support the guidelines and outlines recommendations likely to be followed in clinical practice.     

Performance of an AI based CAD system in solid lung nodule detection on chest phantom radiographs compared to radiology residents and fellow radiologists

BackgroundDespite the decreasing relevance of chest radiography in lung cancer screening, chest radiography is still frequently applied to assess for lung nodules. The aim of the current study was to determine the accuracy of a commercial AI based CAD system for the detection of artificial lung nodules on chest radiograph phantoms and compare the performance to radiologists in training.MethodsSixty-one anthropomorphic lung phantoms were equipped with 140 randomly deployed artificial lung nodules (5, 8, 10, 12 mm). A random generator chose nodule size and distribution before a two-plane chest X-ray (CXR) of each phantom was performed. Seven blinded radiologists in training (2 fellows, 5 residents) with 2 to 5 years of experience in chest imaging read the CXRs on a PACS-workstation independently. Results of the software were recorded separately. McNemar test was used to compare each radiologist’s results to the AI-computer-aided-diagnostic (CAD) software in a per-nodule and a per-phantom approach and Fleiss-Kappa was applied for inter-rater and intra-observer agreements.ResultsFive out of seven readers showed a significantly higher accuracy than the AI algorithm. The pooled accuracies of the radiologists in a nodule-based and a phantom-based approach were 0.59 and 0.82 respectively, whereas the AI-CAD showed accuracies of 0.47 and 0.67, respectively. Radiologists’ average sensitivity for 10 and 12 mm nodules was 0.80 and dropped to 0.66 for 8 mm (P=0.04) and 0.14 for 5 mm nodules (P<0.001). The radiologists and the algorithm both demonstrated a significant higher sensitivity for peripheral compared to central nodules (0.66 vs. 0.48; P=0.004 and 0.64 vs. 0.094; P=0.025, respectively). Inter-rater agreements were moderate among the radiologists and between radiologists and AI-CAD software (K’=0.58±0.13 and 0.51±0.1). Intra-observer agreement was calculated for two readers and was almost perfect for the phantom-based (K’=0.85±0.05; K’=0.80±0.02); and substantial to almost perfect for the nodule-based approach (K’=0.83±0.02; K’=0.78±0.02).ConclusionsThe AI based CAD system as a primary reader acts inferior to radiologists regarding lung nodule detection in chest phantoms. Chest radiography has reasonable accuracy in lung nodule detection if read by a radiologist alone and may be further optimized by an AI based CAD system as a second reader.     

Clinical assessment of suspected deep vein thrombosis: comparison between a score and empirical assessment

OBJECTIVES: To assess the accuracy and agreement of two methods of clinical evaluation: a formal score based on a number of items of fixed value (the so-called Wells' score), and an empirical assessment based on a predefined list of items that can be weighted individually. Clinical probability is essential to manage suspected deep vein thrombosis (DVT) and should be assessed before any diagnostic test. DESIGN: An open, nonrandomised, one-centre study. SETTING: One centre in Switzerland (a university hospital delivering primary-tertiary care). SUBJECTS: Two hundred and seventy outpatients with a prevalence of DVT of 21.1% (final diagnosis), out of an initial population of 328 patients, of which 52 had to be excluded because of a history of DVT (score not applicable) or because of insufficient clinical information (n = 6). RESULTS: Agreement between the two assessment tools was poor (kappa value of 0.32), but accuracy was excellent, with a prevalence of DVT of 1.3%, 18.1%, and 100% (empirical assessment), and 3.2%, 19.4%, and 73.9% (Wells' score), for a low, intermediate or high clinical probability estimate, respectively. The main differences between the two methods were that (i) the empirical method performed slightly better in categorizing patients in the high probability class, and (ii) Wells' score categorized more patients in the low probability class. When applied to two validated diagnostic strategies, the empirical assessment required slightly fewer phlebograms in both strategies, and Wells' score required fewer repeat ultrasonograms (in the strategy that requires this procedure). CONCLUSIONS: Clinical probability assessment can be done with a similar accuracy either empirically or using a score. Institutions should incorporate clinical probability assessment with either method depending upon their diagnostic strategy for suspected DVT.