Quantifying how tests reduce diagnostic uncertainty

Background

Diagnostic tests are commonly evaluated from sensitivity and specificity, which are robust and independent of prevalence, but clinically not intuitive. Many clinicians prefer to use positive and negative predictive values (PPV and NPV), but they are frequently applied as if they are independent of prevalence, whereas this may make an important difference.

Methods and results

We present graphs that allow easy reference to appropriate values and demonstrate that PPV and NPV are not independent of prevalence. PPV and NPV figures reflect the prior probability of the case having a positive diagnosis, estimated clinically from the history, examination and other results, as well as the impact of the test result. To avoid the common error of allowing for prior probability twice, and to interpret the impact of the test result alone, we present graphs of the proportionate reduction in uncertainty score (PRU), calculated from sensitivity, specificity and prevalence. These plots show the extent to which either a positive or negative test result affects the remaining degree of uncertainty about a diagnosis in either direction, according to likely clinical prevalence.

Conclusions

PRU plots demonstrate the discriminatory value of tests more clearly than sensitivity and specificity from which they are derived, and should be published alongside them.Clinicians usually judge the likelihood of a patient having a particular diagnosis by considering information from several sources and integrating these separate assessments. Much of this evaluation is reached apparently unconsciously and instantaneously from a knowledge of populations and diseases, such as the immediately obvious widely different diagnostic implications of jaundice when it occurs in a well 2 day old baby girl compared to an elderly alcoholic man. However, the diagnostic implications of laboratory and imaging test results are generally considered more consciously and formally based on their published performance data.The power of diagnostic tests is usually reported either as sensitivity and specificity, or their derived likelihood or diagnostic odds ratios, or as their positive and negative predictive values (PPV and NPV). These allow clinicians to evaluate how well a test can distinguish between patients from different diagnostic groups. However, there are problems with all these ways of summarising data. PPV and NPV best address the questions that clinicians intuitively want to ask but are of limited usefulness because they can only be applied to groups of patients with the same disease prevalence (or individuals with a similar disease probability) as the population from which the data were derived.I therefore propose a method of plotting the diagnostic implications of positive or negative test results which may make their interpretation more intuitive and convenient to clinicians, allowing them to appreciate their implications easily and quickly from a graph, for any level of estimated disease probability.     

What the clinician really needs to know: questioning the clinical usefulness of sensitivity and specificity in studies of screening tests

This article challenges the American Academy of Pediatrics recommendations for use of sensitivity and specificity in evaluating the validity of developmental screening tests. It emphasizes the clinician's need to know the prevalence of delay in the population to be screened and to estimate predictive values for that population. If one knows the prevalence, sensitivity and specificity can be useful in deriving predictive values but must meet the assumption of stability. Four common, often unrecognized, sources of instability in estimates of sensitivity and specificity are examined, namely, verification bias, small sample sizes, construct-irrelevant variance, and errors in choice and use of diagnostic procedures.     

Quantifying how tests reduce diagnostic uncertainty.

BACKGROUND: Diagnostic tests are commonly evaluated from sensitivity and specificity, which are robust and independent of prevalence, but clinically not intuitive. Many clinicians prefer to use positive and negative predictive values (PPV and NPV), but they are frequently applied as if they are independent of prevalence, whereas this may make an important difference. METHODS AND RESULTS: We present graphs that allow easy reference to appropriate values and demonstrate that PPV and NPV are not independent of prevalence. PPV and NPV figures reflect the prior probability of the case having a positive diagnosis, estimated clinically from the history, examination and other results, as well as the impact of the test result. To avoid the common error of allowing for prior probability twice, and to interpret the impact of the test result alone, we present graphs of the proportionate reduction in uncertainty score (PRU), calculated from sensitivity, specificity and prevalence. These plots show the extent to which either a positive or negative test result affects the remaining degree of uncertainty about a diagnosis in either direction, according to likely clinical prevalence. CONCLUSIONS: PRU plots demonstrate the discriminatory value of tests more clearly than sensitivity and specificity from which they are derived, and should be published alongside them.     

超声剪切波弹性成像联合血液生化指标对胆道闭锁的诊断价值

目的 探讨超声剪切波弹性成像联合血液生化指标对胆道闭锁(biliary atresia,BA)的诊断价值.方法 对湖南省儿童医院临床拟诊为胆汁淤积性肝病的患者行常规超声检查及肝脏剪切波弹性成像(shear wave elastography,SWE)检查,同时收集距SWE检查近3天内肝功能血生化指标.经术中胆管造影及肝...     

Understanding diagnostic tests 2: likelihood ratios, pre- and post-test probabilities and their use in clinical practice

The sensitivity and specificity of a test cannot be used to estimate probability of disease in individual patients. They can, however, be combined into a single measure called the likelihood ratio which is, clinically, more useful than sensitivity or specificity. Likelihood ratios provide a summary of how many times more (or less) likely patients with a disease are to have a particular result than patients without the disease. Using the principles of the Bayes theorem, likelihood ratios can be used in conjunction with pre-test probability of disease to estimate an individual's post-test probability of disease, that is his or her chance of having disease once the result of a test is known. The Fagan's nomogram is a graphical tool which, in routine clinical practice, allows one to combine the likelihood ratio of a test with a patient's pre-test probability of disease to estimate post-test probability. CONCLUSION: Likelihood ratios summarize information about a diagnostic test by combining sensitivity and specificity. The Fagan's nomogram is a useful and convenient graphical tool that allows likelihood ratios to be used in conjunction with a patient's pre-test probability of disease to estimate the post-test probability of disease.     

What does neutropenia tell about the existence of sepsis in the newborn? Predictive value of clinical laboratory tests

A laboratory test can have one of three different functions. As a confirmatory test it can prove a diagnosis, as a screening test it can indicate the probability, that a disease in question is actually present, and as a monitoring test it can be used to guide a medical therapy. To assess the clinical value of a screening test the criteria of sensitivity, specificity and predictive value are commonly used. The meaning of these terms is explained using the example of neutropenia as a screening test for sepsis on the first day of life. For the neonates admitted during their first day of life to the neonatology units of the obstetric and paediatric departments of Zurich University a neutrophil count below 3 X 10(9)/1 as a test for sepsis has a sensitivity of 67%, a specificity of 96.7%, a predictive value of the positive test of 20%, and a predictive value of the negative test of 99.6%. The predictive value of a test is strongly dependent on the prevalence of the disease looked for in the population under study. Variation of the cut off point of a screening test results in a change of its sensitivity and predictive value of the positive test in the opposite direction.     

The predictive value of specific immunoglobulin E levels for the first diagnosis of cow''s milk allergy. A critical analysis of pediatric literature

Investigators have tried to identify a level of seric specific immunoglobulin E (IgE) that had a sufficient predictive value to diagnose a food allergy without having will resort to the food challenge. To search in literature, all the studies that have estimated the possibility to identify a level of seric specific cow milk IgE with a positive predictive value (PPV) of 95% for the first diagnosis of cow's milk allergy (CMA) in pediatric age. We have identified six studies, nearly all studies suffer from relevant methodological bias. Proposed cut-off are all different. The studied pediatric populations were highly selected. Also neglecting the methodological bias of the studies and the great difference of value between the proposed cut-off, it always remains to consider that the pre-test probability of having a CMA between the children enrolled in the six studies included in this review is particularly high. The likelihood ratio helps to transfer the results of a study on a diagnostic test just to our population, and it is more realistic rather than to entrust itself to the PPV or the negative predictive value, that are much influenced from the prevalence of the disease in the studied population.     

Interpretation of Diagnostic Tests

Clinicians can now base clinical decisions on the results of rigorous studies of the performance of diagnostic tests. In selecting the tests, clinicians should take into account their sensitivity (the proportion of patients with the disease who have a positive test result) and specificity (the proportion of patients without disease who have a negative test result). Sensitivity and specificity are affected by the “spectrum” of patients studied i.e. the severity of disease in those with disease and the clinical characteristics of those without disease. Test results are interpreted by predictive values—the proportion of patients with a positive (negative) test result who have (do not have) the disease. Predictive values depend on both test sensitivity and specificity and clisease prevalence. The information content of a test is further increased by taking into account where the cut-off point between normal and abnormal is placed, the degree of abnormality of the result, and the results of the preceding tests (done either in parallel or in series). Describing test performance in terms of likelihood ratios facilitates this process. Readers should be able to crlffque published studies of diagnostic tests which are still far from perfect.     

超声心动图诊断法洛四联症的临床价值评估

目的评价超声心动图诊断法洛四联症（TOF）的准确性、可靠性和效益,探讨法洛四联症无创伤性术前诊断方法及手术指征。方法以手术后诊断为金标准,评价234例TOF患儿的二维超声心动图（2-DE）诊断及心导管诊断结果。结果对于TOF的诊断,2-DE诊断灵敏度97.9%,特异度99.8%,符合率99.6%;心导管诊断灵敏度99.5%,特异度100%,符合率99.9%;两者的差异无统计学意义。在TOF合并心血管畸形的诊断中,2-DE诊断CA横跨ROVT、侧支血管、PDA的灵敏度、特异度、符合率与手术后诊断存在明显差异。结论2-DE可替代心导管造影检查用于TOF的诊断。如果术前2-DE检查发现冠状动脉显示不清或怀疑走行异常、存在侧支血管、PDA与侧支血管难以鉴别及肺动脉分支发育不良时,需进一步行心导管造影检查。     

The predictive value of skin prick testing for challenge-proven food allergy: a systematic review

Immunoglobulin E-mediated (IgE) food allergy affects 6-8% of children, and the prevalence is believed to be increasing. The gold standard of food allergy diagnosis is oral food challenges (OFCs); however, they are resource-consuming and potentially dangerous. Skin prick tests (SPTs) are able to detect the presence of allergen-specific IgE antibodies (sensitization), but they have low specificity for clinically significant food allergy. To reduce the need for OFCs, it has been suggested that children forgo an OFC if their SPT wheal size exceeds a cutoff that has a high predictability for food allergy. Although data for these studies are almost always gathered from high-risk populations, the 95% positive predictive values (PPVs) vary substantially between studies. SPT thresholds with a high probability of food allergy generated from these studies may not be generalizable to other populations, because of highly selective samples and variability in participant's age, test allergens, and food challenge protocol. Standardization of SPT devices and allergens, OFC protocols including standardized cessation criteria, and population-based samples would all help to improve generalizability of PPVs of SPTs.     

Accuracy of neck circumference for diagnosing overweight in six- and seven-year-old children

ObjectiveTo estimate the accuracy of neck circumference measurement as a method of diagnosing excess weight of six and seven-year-old children.Methods1026 six and seven-year-old children were included and anthropometric data were collected using cut-off points for the Body Mass Index (BMI) Z-score, in addition to the measurement of their neck circumference in centimeters. Pearson's correlation coefficient was used to assess the correlation between neck circumference and BMI. Sensitivity, specificity, positive and negative predictive values were calculated. The Receiver Operating Characteristic curve was used to measure the accuracy of neck circumference as a diagnostic method for excess weight.ResultsA positive linear correlation value was observed between neck circumference and BMI 0.572 (p < 0.001). The accuracy value of the global ROC curve was 0.772 (p < 0.001). Sensitivity and specificity showed low values, but high positive predictive values were observed, especially between measures of 30 and 31 cm.ConclusionNeck circumference showed accuracy of 77.2% as a diagnostic method for overweightness in six and seven-year-old children.     

Low antithrombin III: accurate predictor of idiopathic respiratory distress syndrome in premature neonates

A prospective study was performed in premature neonates to determine the predictive values of antithrombin III (AT III) deficiency immediately after birth, for the subsequent development of idiopathic respiratory distress syndrome (IRDS), intraventricular haemorrhage (IVH) and death. Of the 81 premature infants studied, 24 developed IRDS (30%). Of these 24 premature infants, 8 also developed IVH and 9 infants died within the follow-up period of 7 days. The mean plasma AT III level was significantly lower in the infants developing IRDS (0.23 U/ml vs 0.35 U/ml,P<0.0005). Within this study group 33 neonates of less than 30 weeks' gestation showed a prevalence for IRDS of 48%. In this group, AT III activity levels below 0.30 U/ml were 8.5 times as likely to result from infants with IRDS than from infants without IRDS. The diagnostic accuracy indices of criteria for the development of IRDS were: a sensitivity of 100%, a specificity of 88%, a positive predictive value of 89% and a negative predictive value of 100%. The predictive values for the development of IVH and occurrence of death were insignificant. Therefore, in premature neonates the combination of less than 30 weeks' gestation and an AT III below 0.30 U/ml is highly suggestive of IRDS and may facilitate the evaluation of early treatment.Abbreviations AT III antithrombin III - DIC disseminated intravascular coagulation - IRDS idiopathic respiratory distress syndrome - IVH intraventricular haemorrhage - NPV negative predictive value - PPV positive predictive value     

Heart Size on Chest X-Ray as a Predictor of Cardiac Enlargement by Echocardiography in Children

To determine the usefulness of heart size on chest radiograph (CXR) in predicting cardiac enlargement (CE) in children, we prospectively evaluated 95 consecutive outpatients, who had both a CXR and echocardiography performed. Their median age was 5.0 years (2 days to 19.9 years). All patients underwent CXR assessment by a pediatric radiologist, with classification of cardiac silhouette as normal, borderline, or enlarged. Echocardiographic assessment of CE was performed by a pediatric echocardiographer. Sensitivity, specificity, and predictive values of the pediatric radiologist's interpretation of heart size on CXR were estimated. The presence of CE by echocardiography was used as the gold standard. Seventy-nine patients (83.2%) had no CE on CXR, and 16 patients (16.8%) had CE. Sensitivity of the CXR to identify CE was 58.8%, 95% confidence interval (CI) [32.9, 81.6], with a positive predictive value of 62.5% [35.4, 84.8]. Specificity was 92.3% [84.0, 97.1], with a negative predictive value of 91.1% [82.6, 96.4]. These data suggest that the assessment of CE on CXR to predict CE by echocardiography has a relatively high specificity and negative predictive value, but a low sensitivity and positive predictive value. The limitations of CXR as a diagnostic test should be understood by clinicians using the test when screening children for cardiac disease.     

Application of the fetal alcohol syndrome facial photographic screening tool in a foster care population

We determined the prevalence of fetal alcohol syndrome (FAS) in a foster care population and evaluated the performance of the FAS Facial Photographic Screening Tool. All children enrolled in a Washington State Foster Care Passport Program were screened for three conditions: (1) the FAS facial phenotype from a photograph, (2) evidence of brain damage with prenatal alcohol exposure from their Health and Education passport, and/or (3) other syndromes identifiable from a facial photograph. Screen-positives received diagnostic evaluations at a FAS Diagnostic and Prevention Network clinic. The prevalence of FAS in this foster care population was 10 to 15/1000, or 10 to 15 times greater than in the general population. The screening tool performed with 100% sensitivity, 99.8% specificity, 85.7% predictive value positive, and 100% predictive value negative. We conclude that the foster care population is a high-risk population for FAS. The screening tool performed with very high accuracy and could be used to track FAS prevalence over time in foster care to accurately assess the effectiveness of primary prevention efforts.     

Proposal for a standardized interpretation of the atopy patch test in children with atopic dermatitis and suspected food allergy

The interpretation of the atopy patch test (APT) to foods is not standardized. This study aimed to validate the reading of the APT in terms of the diagnostic accuracy of individual skin signs. Eighty-seven children (mean age 2.4 +/- 2.5 yr, range 0.5-13.5; 57 male) with atopic dermatitis (AD) and suspected food allergies underwent APT to cow's milk, hen's egg, wheat and soy. Twelve-millimetre Finn chambers were applied for 48 h, and results were read after 48 and 72 h. Skin changes were graded for erythema, induration, papule formation and 'crescendo' phenomenon (increase of skin sign severity from 48 to 72 h). Food allergy was assessed by double blind, placebo-controlled food challenges (DBPCFC). Sensitivity, specificity and predictive values were calculated for each skin signs in relation to challenge outcome. Of 165 DBPCFC children, 75 (45%) were positive. The combination of any skin induration plus papules (seven or more), or of moderate erythema plus any induration plus seven or more papules had a positive predictive value (PPV) and specificity for the challenge outcome of 100%; however, the sensitivity was low (8% and 15%). The best diagnostic accuracy for single signs was found for induration beyond the Finn chamber margin (PPV 88%, specificity 99%, sensitivity 9%) and presence of at least seven papules (PPV 80%, specificity 96% sensitivity 21%). Presence of both induration and of at least seven papules at 72 h were the APT skin signs with the greatest diagnostic accuracy for food allergy in children with AD.     

A comparison of chromogen test strip (Chemstrip bG) and serum glucose values in newborns

Although glucose oxidase-peroxidase chromogen test strips are frequently used to estimate serum glucose values in newborns, previous studies have not evaluated multiobserver variability of test strip readings and have included few infants with hypoglycemia. We compared values of 272 samples of serum glucose with values simultaneously obtained by chromogen test strips (Chemstrip bG) in newborns. The diagnostic sensitivity of a chromogen test strip less than 2.2 mmol/L for predicting a serum glucose level less than 1.9 mmol/L was 86% (95% confidence interval [CI], 75% to 94%), with 78% specificity (95% CI, 73% to 84%). The positive predictive value in our specimens, with a 21% prevalence of serum glucose levels less than 1.9 mmol/L, was 52% (95% CI, 41% to 62%), with a negative predictive value of 95% (95% CI, 91% to 100%). Fifty-eight of our serum glucose values were less than 1.9 mmol/L and the levels obtained by chromogen test strip were greater than or equal to 2.2 mmol/L in 8 of these cases. Review of these 8 cases showed that a delay in performing the laboratory glucose oxidase serum glucose could account for the discrepancy in 2 cases. Chromogen test strips are readily available and easy to use, but more sensitive, specific, accurate, and precise methods of serum glucose screening in newborns are needed.     

Cerebrospinal fluid soluble CD27 levels in children with non-Hodgkin lymphomas

The authors investigated the diagnostic value of cerebrospinal fluid (CSF) soluble CD27 (sCD27) for leptomeningeal involvement of non-Hodgkin lymphomas (NHL). Cytospin slides were prepared from CSF samples of 64 children treated for NHL. sCD27 levels were determined by sandwich ELISA method using two CD27 monoclonal antibodies. 8/194 (4.1%) samples were considered tumor-positive by cytology. Mean sCD27 values were 5.8 and 13.8 U/mL in tumor-negative and tumor-positive samples, respectively (p =.18). 26/194 samples were false positive and 2/194 false negative (cutoff: 7 U/mL) (sensitivity, 75 %; specificity, 86%; positive predictive value, 18.8%; negative predictive value, 98.8%; accuracy, 85.6%). With these results, the value of adding sCD27 determination to the cytological CSF examination remains questionable.     

Validity of the CRAFFT substance abuse screening test among adolescent clinic patients

OBJECTIVE: To determine the accuracy of the CRAFFT substance abuse screening test. DESIGN: Criterion standard validation study comparing the score on the 6-item CRAFFT test with screening categories determined by a concurrently administered substance-use problem scale and a structured psychiatric diagnostic interview. Screening categories were "any problem" (ie, problem use, abuse, or dependence), "any disorder" (ie, abuse or dependence), and "dependence." SETTING: A large, hospital-based adolescent clinic. PARTICIPANTS: Patients aged 14 to 18 years arriving for routine health care. MAIN OUTCOME MEASURES: The CRAFFT receiver operating characteristic curve, sensitivity, specificity, positive predictive value, and negative predictive value. RESULTS: Of the 538 participants, 68.4% were female, and 75.8% were from racial and ethnic minority groups. Diagnostic classifications for substance use during the past 12 months were no use (49.6%), occasional use (23.6%), problem use (10.6%), abuse (9.5%), and dependence (6.7%). Classifications were strongly correlated with the CRAFFT score (Spearman rho, 0.72; P<.001). A CRAFFT score of 2 or higher was optimal for identifying any problem (sensitivity, 0.76; specificity, 0.94; positive predictive value, 0.83; and negative predictive value, 0.91), any disorder (sensitivity, 0.80; specificity, 0.86; positive predictive value, 0.53; and negative predictive value, 0.96) and dependence (sensitivity, 0.92; specificity, 0.80; positive predictive value, 0.25; and negative predictive value 0.99). Approximately one fourth of participants had a CRAFFT score of 2 or higher. Validity was not significantly affected by age, sex, or race. CONCLUSION: The CRAFFT test is a valid means of screening adolescents for substance-related problems and disorders, which may be common in some general clinic populations.     

组织因子途径抑制物及前降钙素在新生儿败血症诊断中的价值

目的 了解组织因子途径抑制物(TFPI)、前降钙素(PCT)在新生儿败血症诊断中的价值.方法 通过检测48例败血症新生儿及30例健康新生儿血TFPI、PCT、C-反应蛋白(CRP)浓度,比较各炎症指标对诊断败血症的灵敏度、特异度、阳性预测值、阴性预测值和约登指数,评价它们对该病的早期诊断价值.结果 (1)以TFPI≥30μg/L、CRP≥8 mg/L、PCT≥2 ng/ml为阳性标准,三指标对诊断败血症的灵敏度分别为86.92%、89.83%、87.50%,差异无显著性(P＞0.05),其中PCT的特异度96.67%、阳性预测值97.50%、阴性预测值83.32%、约登指数0.84;(2)在败血症组中,20例血培养阳性患儿的TFPI值为(35.5±4.5)μg/L,28例血培养阴性患儿的TFPI值为(34.3±3.2)μg/L,差异无显著性(P＞0.05);但是相对于正常对照组(26.9±5.24)μg/L,败血症组的TFPI值明显升高(P＜0.05).结论 TFP≥30μg/L对诊断新生儿败血症是一个具有较高灵敏度(86.92%)、中度特异度(59.3%)的指标;TFPI对新生儿败血症早期诊断有一定的价值,但均不及PCT及CRP,所有检测指标中PCT特异度、阳性预测值、阴性预测值、约登指数均最高.     

Dermatoglyphic (fingerprint) patterns in celiac disease

Fingerprints were obtained from 46 patients with celiac disease and compared with those of 46 control subjects matched for sex and ethnic origin. Whorls were more frequent and ulnar loops were less frequent, significantly, in celiac patients than in controls. A digital pattern of four or more whorls was present in 69% of celiac patients, but in only 28% of controls (p less than 0.001). As a diagnostic test, the fingerprint's sensitivity, specificity, and positive and negative predictive values were 66, 73, 67, and 71%, respectively. Similarly, a pattern of four or less ulnar loops was evident in 44% of celiac patients as opposed to only 19% of controls (p less than 0.005). Sensitivity, specificity, and positive and negative predictive values were 46%, 81%, 62%, and 63%, respectively. We conclude that particular dermatoglyphic patterns are significantly more common in patients with celiac disease than in controls. We therefore suggest that this marker be used as a diagnostic clue, indicating the need for further investigation.