Subjective assessment of adnexal masses with the use of ultrasonography: an analysis of interobserver variability and experience.

OBJECTIVE: The aim of the study was to evaluate the subjective assessment of ultrasonographic images for discriminating between malignant and benign adnexal masses. STUDY DESIGN: The study was prospective. Initially, one ultrasonographer preoperatively assessed 300 consecutive patients with adnexal masses. Subsequently, the recorded transparent photographic prints were independently assessed by five investigators, with different qualifications and level of experience, who were also given a brief clinical history of the patients (i.e. the age, menstrual status, family history of ovarian cancer, previous pelvic surgery and the presenting symptoms). The diagnostic performance of the observers was compared with the histopathology classification of malignant or benign tumors. The end-points were accuracy, interobserver agreement and the possible effect of experience. RESULTS: The first ultrasonographer and the most experienced investigator both obtained an accuracy of 92%. There was very good agreement between these two investigators in the classification of the adnexal masses (Cohen's kappa 0.85). The less experienced observers obtained a significantly lower accuracy, which varied between 82% and 87%. Their interobserver agreement was moderate to good (Cohen's kappa 0.52 to 0.76). CONCLUSION: Experienced ultrasonographers using some clinical information and their subjective assessment of ultrasonographic images can differentiate malignant from benign masses in most cases. The accuracy and the level of interobserver agreement are both correlated with experience. About 10% of masses were extremely difficult to classify (only < 50% of assessors were correct).     

Accuracy and reproducibility of two scales in causality assessment of unexpected hepatotoxicity

What is known and Objective: There is almost no published information about reliability of scales for causality assessment in hepatotoxicity at pharmacovigilance centres. The aim of this study was to compare two commonly used scales in cases of unexpected hepatotoxicity, in evaluating their accuracy and reproducibility at pharmacovigilance centres (in signal detection). Methods: Two scales [Council for International Organizations of Medical Sciences or Rousel Uclaf Causality Assessment Method (CIOMS/RUCAM) and NARANJO] were compared in 19 cases of unexpected hepatotoxicity reported during 2004–2009. Data of the cases (with initial clinical assessments) were collected by a network of medical specialists using a structured reporting form. Later, two independent observers assessed each case using both scales. The accuracy and reproducibility of the scales were analysed by Kappa weighted (Kw) test. Results: Both scales (CIOMS/RUCAM vs. NARANJO) showed moderate agreement with the initial clinical assessments (accuracy) for observer A (Kw: 0·56 vs. 0·60) and substantial agreement for observer B (Kw: 0·72 vs. 0·70), with high agreement between observers (Kw: 0·84 vs. 0·67). Both observers (A vs. B) found low agreement between scales (Kw: 0·21 vs. 0·50), with lower scores for the CIOMS/RUCAM scale in 11 and nine cases, respectively. For an early perception of unexpected serious reactions, the scale is more useful if it is not asked for ‘previous knowledge’ and if it gives higher causality score. What is new and conclusion: The CIOMS/RUCAM scale showed similar accuracy, but better reproducibility (agreement between observers) than the NARANJO scale, and therefore is recommended for use at pharmacovigilance centres. Fine‐tuning of the CIOMS/RUCAM method could contribute to better detection of unexpected hepatotoxicity.     

Incidence of upper limb oedema in patients with acute hemiparetic stroke

Purpose.?Assessment of the incidence of upper limb oedema in an acute care setting by means of clinical and volumetric evaluation.

Methods.?Patients with acute hemiparetic stroke were recruited from 2006 until 2009 (n == 125). Baseline measurements consisted of the National Institute of Health Stroke Scale, Fugl-Meyer Assessment Arm Section and demographic characteristics. Oedema assessment was performed at 7 days after inclusion and at 1 month and 3 months follow-up. A standardised water displacement method (objective measurement) was used to define oedema and was compared to data from visual inspection and palpation (subjective measurement).

Results.?In literature, the incidence of upper limb oedema ranges from 16–83%%, defined by a variety of definitions. Oedema incidence in this study was defined by strict criteria using water displacement and ranged from 9–13.9%%, while the incidence of oedema defined by visual inspection and palpation ranged from 6–18.5%% during the different stages of follow-up. The agreement (Kappa) between both measurements ranged from 0.23–0.38, which is not more than ‘moderate’ but comparable to the agreement of 0.34 found in literature.

Conclusion.?An objective and subjective assessment of oedema was used; the agreement between both methods was only moderate. The incidence of oedema found in this study is lower than the incidences found in literature.     

运用主观评估法评价α酮酸对老年透析患者营养的改善作用

目的本研究采用主观评估（微型营养评估法与主观全面评估法）与客观指标结合的综合评估方法,初步探讨复方α酮酸制剂对于透析患者蛋白营养不良的治疗作用。方法选择上海交通大学附属第一人民医院透析中心血清白蛋白低于35g/L老年血液透析及腹膜透析患者各24例,透析时间均大于一年,其中,血液透析组及腹膜透析组各设对照组12例。除对照组外,所有患者均服用开同6个月。营养评估采用综合评估法,即客观营养指标结合主观营养评估。客观指标包括：体重指数（BMI）、血白蛋白（Alb）、血前白蛋白和平均每日每公斤体重能量与蛋白质摄入（DEI、DPI）。主观评估采用了两种评分方法：即通用的主观全面评估法（Subject Global Assessment,SGA）与专门用于老年患者的微型营养评估法（Mini NutritionalAssessment,MNA）,同时测定了患者饮食中的蛋白质与能量摄入情况。结果①伴营养不良的患者普遍存在SGA与MNA评分偏低,提示主客观的评估一致性较好;②与对照组相比,应用酮酸制剂治疗的透析患者3个月后,蛋白质与能量的摄入明显上升,虽体重指数改变不明显,但血清白蛋白水平明显升高,;治疗6个月以后,SGA评分、MNA评分明显上升;低血清白蛋白血症进一步改善,而蛋白质与能量的摄入未进一步增加,患者的体重指数BMI、透析充分性、C反应蛋白（CRP）、血脂改变仍不明显。结论复方酮酸制剂能够提高血清白蛋白水平,改善营养状况,微型营养评估法更适合老年透析患者的营养评估。     

Kappa统计量评价半月板MRI诊断

目的　应用Kappa统计量评价不同观察者对膝关节半月板撕裂MRI诊断的一致性。 方法　由一位有经验的骨关节放射科医生回顾分析 13 9例半月板MRI图像 ,所得结果与原始MRI报告结果进行Kappa统计分析。 结果　对内侧半月板 ,MRI初诊和MRI再诊具有极好的诊断一致性 ,Kappa值为 0 .79;对外侧半月板 ,两者具有好的诊断一致性 ,Kappa值为 0 .70。结论　MRI初诊和MRI再诊对膝关节内外侧半月板的诊断一致性良好     

Variability in Doppler ultrasound influences referral of patients for carotid surgery.

OBJECTIVE: Colour Doppler ultrasound is operator dependent, but it is unclear how much clinical impact this might have on patient referral for carotid endarterectomy. Our aim was to quantify the interobserver variability of Doppler ultrasound. METHODS: Consecutive patients attending for carotid Doppler ultrasound underwent two examinations on the same day, in random order, by two of three observers blind to each other's results. Severity of stenosis was assessed using standard velocity criteria and lesion appearance. RESULTS: A total of 189 patients were scanned (378 ICAs). Of the 134 ICAs scanned by observers 1 and 2, observer 1 classified 11 as 80-99% stenosis (operable), compared with nine by observer 2. Of the 206 ICAs scanned by observers 1 and 3, observer 1 classified 11 as 80-99% stenosis, compared with only five by observer 3. Of the 38 ICAs scanned by observers 2 and 3, observer 2 classified 2 as 80-99% stenosis compared with none by observer 3. Overall, clinical management would differ in 10/378 (3%) of ICAs, but in 10/22 (45%) of those considered operable by one of the three observers. CONCLUSION: There was clinically important interobserver variability in the assessment of ICA disease, which could result in serious errors if endarterectomy were performed on the basis of a single Doppler ultrasound.     

Subjective assessment of right ventricle enlargement from computed tomography pulmonary angiography images

To retrospectively evaluate prognostic accuracy of subjective assessment of right ventricle (RV) enlargement on CT pulmonary angiography (CTPA) images in comparison with objective measures of RV enlargement in patients with acute pulmonary embolism (PE). For 200 consecutive patients with acute PE, two readers blinded to patient outcomes subjectively determined whether the maximum RV diameter was greater than that of the left ventricle (LV) using axial CTPA images. For the objective measurements, RV/LV diameter ratios were calculated using axial images and 4-chamber reformatted images. For all assessments, sensitivities and specificities for predicting PE-related death within 30-days and a composite outcome including PE-related death or the need for intensive therapies were compared. The agreement between two readers was 91.5% (kappa = 0.83) and all other assessments had pair-wise agreement over 75% (kappa = 0.53–0.72). There was no significant difference in sensitivity between the subjective and objective methods for predicting both outcomes. The specificity for subjective RV enlargement (55.4–67.7%) was significantly higher than objective measures (45.8–53.1%), except for the 4-chamber views where, for one reader, the specificity of the subjective evaluation was higher but did not reach statistical significance. Complex measurements of RV/LV diameter ratios may not be needed to maximize the prognostic value from CTPA. The radiologist who interprets the CTPA images should report RV enlargement when the RV diameter subjectively appears larger than the LV.     

Observer agreement in the assessment of minor and major cerebral aneurysm recurrence

We studied the use of minor and major recurrence classifications with regard to observer variability, which directly affects validity of these terms as outcome measures. Recurrences at the time of a three to six month follow-up angiogram were assessed independently by two experienced observers in 83 cases. Intra- and interobserver concordance is approximately 90% for assessment of minor and major recurrence.     

Sonographic estimation of amniotic fluid volume. Subjective assessment versus pocket measurements

Amniotic fluid volume (AFV) estimation is an important part of routine obstetric sonography. Despite the clinical importance placed upon excessive or diminished AFV in pregnancy, there is little uniformity in the way it is estimated sonographically. We compared AFV estimations obtained using two commonly employed sonographic methods "subjective" visual assessment and amniotic fluid pocket measurements. Estimates obtained using both methods correlated closely. In addition, there was excellent intraobserver and interobserver agreement among estimates obtained using subjective criteria. This supports the belief that experienced observers tend to agree on the sonographic appearance of normal, excessive or decreased AFV. Methods for AFV estimation and potential pitfalls are discussed.     

Interobserver Agreement in Assessment of Clinical Variables in Children with Blunt Head Trauma

Objectives: To be useful in development of clinical decision rules, clinical variables must demonstrate acceptable agreement when assessed by different observers. The objective was to determine the interobserver agreement in the assessment of historical and physical examination findings of children undergoing emergency department (ED) evaluation for blunt head trauma. Methods: This was a prospective cohort study of children younger than 18 years evaluated for blunt head trauma at one of 25 EDs in the Pediatric Emergency Care Applied Research Network (PECARN). Patients were excluded if injury occurred more than 24 hours prior to evaluation, if neuroimaging was obtained at another hospital prior to evaluation, or if the patient had a clinically trivial mechanism of injury. Two clinicians independently completed a standardized clinical assessment on a templated data form. Assessments were performed within 60 minutes of each other and prior to clinician review of any neuroimaging (if obtained). Agreement between the two observers beyond that expected by chance was calculated for each clinical variable, using the kappa (κ) statistic for categorical variables and weighted kappa for ordinal variables. Variables with a lower 95% confidence limit (LCL) of κ > 0.4 were considered to have acceptable agreement. Results: Fifteen‐hundred pairs of observations were obtained. Acceptable agreement was achieved in 27 of the 32 variables studied (84%). Mechanism of injury (low, medium, or high risk) had κ = 0.83. For subjective symptoms, kappa ranged from 0.47 (dizziness) to 0.93 (frequency of vomiting); all had 95% LCL > 0.4. Of the physical examination findings, kappa ranged from 0.22 (agitated) to 0.89 (Glasgow Coma Scale [GCS] score). The 95% LCL for kappa was <0.4 for four individual signs of altered mental status and for quality (i.e., boggy or firm) of scalp hematoma if present. Conclusions: Both subjective and objective clinical variables in children with blunt head trauma can be assessed by different observers with acceptable agreement, making these variables suitable candidates for clinical decision rules.     

迭代重建结合自动管电流调节低剂量扫描技术在肺结节检查中的应用

目的探讨胸部宝石CT自动管电流调节(ATCM)结合自适应迭代重建(ASIR)低剂量扫描技术在肺结节检查中的应用。方法对80例常规胸部CT检查发现肺结节的患者采用宝石能谱CT行两次扫描。第1次扫描采用单纯ATCM技术,第2次扫描采用ATCM技术结合50%ASIR重建。噪声指数根据体重指数(BMI)确定,第1次扫描(纯ATCM组):BMI≤22 kg/m2、噪声指数为14,BMI>22kg/m2、噪声指数为20;第2次扫描(ATCM结合ASIR组):BMI≤22?kg/m2、噪声指数为20,BMI>22?kg/m2、噪声指数为26。由2名放射科医师分别对图像质量和图像噪声进行主观评分(5分制),并采用Kappa检验观察不同医师间评定结果的一致性。记录两组的CT剂量容积指数(CTDIvol),并采用配对样本t检验比较。比较两次扫描肺结节的检出率,分析两次扫描影像显示结节的数目和大小。结果 ATCM结合ASIR组CTDIvol为(1.60±0.45)mGy,单纯ATCM组CTDIvol为(4.26±0.41)mGy,结合ASIR技术可以降低约62%的剂量,两组差异有统计学意义(t=26.81,P<0.05)。单纯ATCM组,2名医师的图像质量评分分别为(4.50±0.48)分和(4.35±0.27)分,Kappa=0.862,P<0.01;ATCM结合ASIR组,2名医师的图像质量评分分别为(4.96±0.36)分和(4.82±0.45)分,Kappa=0.795,P<0.01。单纯ATCM组,2名医师的图像噪声评分分别为(4.18±0.52)分和(3.90±0.42)分,Kappa=0.648,P<0.01;ATCM结合ASIR组,2名医师的图像噪声评分分别为(4.42±0.43)分和(4.41±0.26)分,Kappa=0.788,P<0.01。所有图像质量评分均达到满足影像诊断的要求。两种扫描方案扫描检出肺结节的数目相等,且肺结节的直径间差异无统计学意义(P>0.05)。结论胸部CT扫描可提升噪声指数,并且应用ASIR技术获得的图像能够保证图像质量符合临床诊断要求,并进一步减少辐射剂量,更适合对肺结节进行筛查、初步诊断和随访。     

Pretest probability assessment for selective rest sestamibi scans in stable chest pain patients

The objective of this study was to determine whether pretest probability assessments permit more selective testing of chest pain patients with technetium-99m sestamibi scanning. Pretest probabilities of cardiac ischemia were measured both objectively (Acute Cardiac Ischemia Time-Insensitive Predictive Instrument [ACI-TIPI]) and subjectively (physician's estimate of the probability of unstable angina). Two groups were defined: patients whose postsestamibi scan led to a "downgrade" of the intensity of monitoring and those that resulted in no change in monitoring intensity. Sixty-five patients met study criteria; 25 had a disposition downgrade and 40 had no change. Pretest ACI-TIPI scores were similar in the two groups (29% +/- 18% versus 27% +/- 11%, mean +/- standard deviation; P = .95) as were the physician's assessment of unstable angina (39% +/- 22% versus 40% +/- 24%; P = .75). Objective or subjective pretest probabilities are not significantly different in patients who are likely to have their disposition altered by sestamibi scanning.     

Automated edge-detection technique for measurement of brachial artery reactivity: a comparison of concordance with manual measurements

Concerns have been raised about the reproducibility of brachial artery reactivity (BAR), because subjective decisions regarding the location of interfaces may influence the measurement of very small changes in lumen diameter. We studied 120 consecutive patients with BAR to address if an automated technique could be applied, and if experience influenced reproducibility between two observers, one experienced and one inexperienced. Digital cineloops were measured automatically, using software that measures the leading edge of the endothelium and tracks this in sequential frames and also manually, where a set of three point-to-point measurements were averaged. There was a high correlation between automated and manual techniques for both observers, although less variability was present with expert readers. The limits of agreement overall for interobserver concordance were 0.13 +/- 0.65 mm for the manual and 0.03 +/- 0.74 mm for the automated measurement. For intraobserver concordance, the limits of agreement were - 0.07 +/- 0.38 mm for observer 1 and - 0.16 +/- 0.55 mm for observer 2. We concluded that BAR measurements were highly concordant between observers, although more concordant using the automated method, and that experience does affect concordance. Care must be taken to ensure that the same segments are measured between observers and serially.     

National Emergency Department Overcrowding Study tool is not useful in an Australian emergency department

Objective: To determine the accuracy and usefulness of the National Emergency Department Overcrowding Study (NEDOCS) tool in an urban hospital ED in Australia by direct comparison with subjective assessment by senior ED staff. Method: A sample of simultaneous subjective and objective data pairs were collected six times a day for a period of 3 weeks. All senior medical staff in the ED answered a brief questionnaire along with the senior charge nurse for the ED. Simultaneously, the senior charge nurse also documented the total number of patients in the ED, the number of patients awaiting admission, the number of patients on ventilators, the longest time waited by an ED patient for ward bed, and the waiting time for the last patient from the Waiting Room placed on a trolley. The objective indicators were entered into a Web‐based NEDOCS tool and transformed scores were compared with the averaged and transformed subjective scores for each sample time. Bland–Altmann and Kappa statistics were used to test the agreement between the objective and subjective measuring methods. Results: The mean difference between the subjective and objective methods was small (3.5 [95% confidence interval ?0.875–7.878] ); however, the 95% limits of agreement was wide (?46.52–53.43). The Kappa statistic used to assess the extent of reproducibility between categorical variables was 0.31 (95% confidence interval 0.17–0.45). Conclusion: The present study suggests that NEDOCS method of processing the objective overcrowding data does not accurately reflect the subjective assessment of the senior staff working at that time in the ED. This might be because the assumptions of the original NEDOCS study are flawed.     

Reproducibility of contrast-enhanced transrectal ultrasound of the prostate

Transrectal three-dimensional (3-D) contrast-enhanced power Doppler ultrasound (US) is a novel technique for studying possible prostate malignancy. Before studies can be performed to investigate the clinical validity of the technique, reproducibility of the contrast US studies must be proven. Reproducibility of contrast US was studied in 10 patients with biopsy-proven prostate cancer. The studies performed included static investigations and dynamic investigations of the prostate vasculature. All studies were double performed. The assessment of reproducibility was done objectively using a computer program and, subjectively, by visual assessment. The results indicate high reproducibility of static contrast investigations, for both the objective and subjective assessment. The subjective assessment of the dynamic studies was also highly reproducible. The objective assessment of the dynamic contrast studies, however, was less reproducible, mainly due to motion artefact. We concluded that, especially static 3-D contrast-enhanced, power Doppler investigations of the prostate are highly reproducible.     

子宫动脉频谱舒张期切迹在妊娠期高血压疾病诊断中的可靠性检验

目的 探讨子宫动脉频谱舒张期切迹在妊娠期高血压疾病的可靠性.方法 两名医师主观判断66例妊娠期高血压疾病中出现子宫动脉舒张期切迹,比较两者的一致性.结果 两名医师评判出现子宫动脉舒张期切迹的一致性检验Kappa值为0.922.结论 主观判断子宫动脉舒张期切迹在妊娠期高血压疾病中是可行的.     

International Headache Society classification: interobserver reliability in the diagnosis of primary headaches

We assessed interobserver reliability of the International Headache Society (IBIS) classification for diagnosis of primary headaches. The study was performed on 103 patients consecutively seen at two Headache Centres. Each patient was given a structured interview recorded on videotape. Four experienced clinicians then reviewed the interviews separately and made a diagnosis of headache according to IHS criteria at the one- and two-digit levels. At both the one- and the two-digit level the agreement was substantial (Kappa = 0.74 and 0.65, respectively). The analysis of reliability for each of nine items necessary for diagnosis showed an agreement ranging from substantial (Kappa = 0.69) to almost perfect (Kappa = 0.89). Our results indicate that the IHS classification has a good reliability for the diagnosis of primary headaches at the one- and two-digit levels.     

Is single observer identification of wound infection a reliable endpoint?

The majority of wound studies over the past 20 years have relied on single observer determination of infection presence or absence as a study endpoint. Nevertheless, there are few data on the reliability of those determinations. If single observer determinations are not reliable, the validity of studies using this endpoint should be questioned. Two physicians independently evaluated wounds for presence or absence of the primary endpoint of infection and the secondary endpoints of purulence, tenderness, warmth, and erythema. Kappa values were calculated as measures of inter-observer agreement, which was used as a measure of reliability. Of 125 patients enrolled, 115 wounds were available for analysis. Kappa =.57 [95%CI.39 to .75] for the primary endpoint of infection, .84 for purulence, .72 for tenderness, .48 for warmth, and .48 for erythema. Agreement on two features of infection and overall agreement on the presence or absence of wound infection was only ‘moderate.’ Single observer determination of wound infection by inexperienced observers using imprecise definitions, though commonly used, is an unreliable measure for scientific study.     

Nutritional status of patients with advanced cancer: the value of using the subjective global assessment of nutritional status as a screening tool

In patients suffering from advanced neoplastic disease, malnutrition is a common complication affecting both the survival and quality of life. In order to monitor early dietary interventions, an assessment of patients' nutritional status is essential. We assessed the nutritional status of 46 patients using two different methods: 1) an objective method of nutritional assessment and 2) the subjective global assessment (SGA) technique. It was found that 28 patients were characterized as malnourished by means of the objective method and 30 patients according to the SGA. The correlation of the results of the assessments between the two methods was high and a validation test of the SGA gave a sensitivity of 96% and specificity of 83%. The most frequent symptoms affecting food intake were anorexia, early satiety, dry mouth, pain and nausea. The results show that the SGA represents an easy method for assessment of the nutritional status in such cancer patients and that it can therefore be used as a screening tool. The high incidence of malnutrition in this group of patients, and their rare use of nutrient supplements, both indicate the importance of early nutritional assessment, and nutritional intervention when appropriate.