Radiation exposure from diagnostic imaging: agreement between self-report and medical records

Data on diagnostic imaging procedures from a highly structured interview were compared to medical records in a case-control study of radiography and acute myelogenous leukemia. Three hundred and twenty-eight cases and 315 controls (78% of participants) had medical records available from an average of 71% of providers. Proxies were used for 49% of cases because of rapid fatality. Mean agreement (number of procedures in medical records subtracted from number in interview) showed similar levels of overreporting in cases [0.6; 95% confidence interval (CI): 0.0, 1.1] and controls (0.7; CI: 0.2, 1.3) with few procedures (<= 10). Most participants with more procedures underreported exposure, and underreporting increased with increasing exposure, especially among cases [mean (CI) agreement = -2.1 (-4.3, 0.0) for 11-20 procedures, -6.4 (-13.6, 0.7) for >20 procedures] and case proxies. High-dose, fluoroscopic, and non-routine procedures were self-reported more accurately than low-dose, non-fluoroscopic, and routine procedures, respectively (p < 0.01 for each comparison), and tended to be underreported. Case-control differences in agreement were non-significant for these categories of procedures. We conclude that diagnostic imaging procedures of most interest to the AML-radiography hypothesis are self-reported accurately but that underreporting does occur and might lead to attenuated risk estimates.     

Comparison of risk adjustment measures based on self-report, administrative data, and pharmacy records to predict clinical outcomes

Comparing clinical outcomes in observational studies often requires adjustment for comorbid disease. The objective of this study was to compare the performance of risk adjustment measures derived from different data sources to predict the clinical outcomes of mortality and hospitalization. We compared the predictive ability of self-reported comorbidity measures to those derived from administrative diagnosis codes and pharmacy data to predict all-cause mortality and hospitalizations in a large sample of veterans receiving care in the Veterans Affairs outpatient clinic setting. In logistic regression models to predict mortality adjusting for age and gender, the Seattle Index of Comorbidity, SF-36, Charlson Index, Diagnosis Cost Groups, and RxRisk had similar discriminatory power ranging between 0.73 and 0.74. The Adjusted Clinical Groups and Chronic Illness and Disability Payment System were less accurate in prediction mortality. Although all measures performed less well in predicting hospitalizations, administrative measures performed better than self-reported measures. We conclude that self-reported morbidity measures had similar performance to administrative and pharmacy measures to predict mortality in a larger outpatient sample, but under-performed these measures in predicting hospitalization. While models using self-report measures can typically only be run on subsamples of patients for which models using administrative and pharmacy measures can be run, models combining self-reported morbidity and other measures performed better than models with a single measure. The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.     

Individuals with type 2 diabetes and depressive symptoms exhibited lower adherence with self-care

OBJECTIVE: Our study aimed to determine whether depressive symptoms are associated with poor self-care behaviors among patients with type 2 diabetes. METHODS: Study subjects were 168 patients with diabetes, aged >30 years, who had a diabetes history of 1-15 years. Using a self-reported questionnaire, we evaluated diabetes self-care behaviors and depressive symptoms. Self-care behaviors were evaluated in five categories: medication taking, self-monitoring of blood glucose (SMBG), diet, exercise, and participation in patient education programs. Depressive symptoms were evaluated using the Centers for Epidemiologic Studies-Depression (CES-D) scales. Multiple logistic regression analyses were used to determine the association between self-care behaviors and depressive symptoms. RESULTS: Higher depressive-symptom scores were associated with poor self-care behaviors, significantly with poor participation in education programs (odds ratio OR=1.21, 95% confidence interval CI=1.06-1.38) and poor diet (OR=1.11, 95% CI=1.01-1.22), and marginally with poor medication taking (OR=1.14, 95% CI=1.00-1.31). Depressive symptoms were not significantly associated with either SMBG or exercise. CONCLUSIONS: These data suggest that the evaluation and control of depressive symptoms among diabetic patients would improve their adherence to self-care behaviors.     

贫困农村居民自报患病的影响因素分析

本文利用10个贫困县3103户农民的入户调查资料，采用计量经济学模型，分析影响农村居民自报患病情况的主要影响因素，得出健康意识、经济状况与心理压力是影响农民自报患病与否的主要因素。     

Minimal error in self-report of having had DXA, but self-report of its results was poor

OBJECTIVE: To examine the validity of both self-report of having had dual-energy x-ray absorptiometry (DXA) and self-reported DXA test results. METHODS: Participants were recruited in 2003 and 2004 from a population-based study of women aged 65-90 years examining osteoporosis management in Ontario, Canada (N=871). Women reporting having had a DXA were eligible and asked to report the results of their most recent test. Participant responses were compared against DXA reports obtained from physicians. RESULTS: We obtained 413 (81%) physician records among the 510 women who reported having had a DXA test. Of these, the positive predictive value for self-report of having had a DXA was 93%. The weighted kappa statistic for the agreement between self-report and actual DXA results was 0.42 (95% confidence interval=0.34-0.49). Although 84% of those with normal bone reported this, only 29% of those with osteopenia and 62% with osteoporosis reported their results correctly. Self-report of a clinical diagnosis of osteoporosis was better among those with a low trauma fracture, yet underestimated osteoporosis prevalence by 24%. CONCLUSION: There is minimal measurement error in self-report of having had a DXA test. Self-report of DXA results will underestimate osteoporosis prevalence and is not a good proxy for clinical diagnosis.     

Comorbidity information in older patients at an emergency visit: self-report vs. administrative data had poor agreement but similar predictive validity

OBJECTIVES: The objectives of this study were (1) to ascertain the level of agreement between the Charlson Comorbidity Index (CCI) based on self-report vs. administrative records, and factors affecting agreement and (2) to compare the predictive validity of the two indices in a sample of older emergency department (ED) patients. STUDY DESIGN AND SETTING: The study was a secondary analysis of data from a randomized trial of an ED-based intervention. The self-report and administrative CCI were compared using the intraclass correlation coefficient (ICC). Factors examined for effect on agreement included health service utilization, age, and sex. The predictive validity of the indices was compared using subsequent health services utilization and functional decline as outcomes. Participants (n=520) were recruited at four university-affiliated Montreal hospitals. Eligibility criteria included 65 years of age or older, able to speak English or French, and discharged to the community. RESULTS: Agreement between the two sources was poor to fair (overall weighted ICC 0.43 [95% confidence interval [CI]: 0.40, 0.47]). The predictive validity was similar for the two indices (area under the receiver-operating characteristic curve 0.51-0.66, depending on the outcomes). CONCLUSION: Agreement between self-report and administrative comorbidity data is only poor to fair but both have comparable predictive validity.     

Evidence of self-report bias in assessing adherence to guidelines.

OBJECTIVE: To assess trends in the use of self-report measures in research on adherence to practice guidelines since 1980, and to determine the impact of response bias on the validity of self-reports as measures of quality of care. METHODS: We conducted a MEDLINE search using defined search terms for the period 1980 to 1996. Included studies evaluated the adherence of clinicians to practice guidelines, official policies, or other evidence-based recommendations. Among studies containing both self-report (e.g. interviews) and objective measures of adherence (e.g. medical records), we compared self-reported and objective adherence rates (measured as per cent adherence). Evidence of response bias was defined as self-reported adherence significantly exceeding the objective measure at the 5% level. RESULTS: We identified 326 studies of guideline adherence. The use of self-report measures of adherence increased from 18% of studies in 1980 to 41% of studies in 1985. Of the 10 studies that used both self-report and objective measures, eight supported the existence of response bias in all self-reported measures. In 87% of 37 comparisons, self-reported adherence rates exceeded the objective rates, resulting in a median over-estimation of adherence of 27% (absolute difference). CONCLUSIONS: Although self-reports may provide information regarding clinicians' knowledge of guideline recommendations, they are subject to bias and should not be used as the sole measure of guideline adherence.     

Poor agreement found between self-report and a public registry on duration of sickness absence

ObjectiveTo investigate the agreement between self-report and a public registry regarding the occurrence and duration of sickness absence (SA) in patients with sciatica.Study Design and SettingObservational 2-year longitudinal study including 227 patients. Self-report SA data were obtained by postal questionnaires covering recall periods of 3, 6, and 12 months and compared with data from the Norwegian National Sickness Benefit Register.ResultsThe percent agreement on the occurrence of SA between self-report and registry was above 85% for all three recall periods. The patients overestimated the duration of their SA by 2.4 (95% confidence interval, 1.1–3.7) weeks for the 3-month recall period. The 95% limits of agreement were generally wide, varying from ?12.5 to 17.3 weeks for the 3-month recall period to ?38.8 to 37.2 weeks for the 12-month period. For the 3-, 6-, and 12-month recall periods, 48.1%, 29.8%, and 27.3% of the patients reported an SA duration that differed by ≤1 week from that recorded in the registry.ConclusionMainly because of low precision, the agreement on the duration of SA was poor for all three recall periods. The agreement between self-report and registry regarding the occurrence of SA was good.     

Measures of agreement for vectorcardiography data

In this paper, nonparametric methods are proposed for quantifying agreement and disagreement between different measurement methods when the results of the measurements are rotation matrices. First, the expected squared distance between two matrices is used to quantify the measurement agreement. Two choices of such distance are considered—the Frobenius distance and geodesic distance. Second, the notion of ‘concordance correlation coefficient’, a commonly used measure of agreement, is extended to the space of rotation matrices. Such generalized concordance coefficient can be treated as a normalized expected squared distance. Since no two measurement systems can be expected to be in perfect agreement, it becomes necessary to define a notion of practical agreement. We define such a notion. Moreover, for both proposed methods, the percentile bootstrap procedure is implemented to provide a confidence interval to help make a decision concerning practical agreement/disagreement in real‐life applications. The methodology is illustrated using two data sets, one based on an application involving vectorcardiography data (Biometrika 1972; 59 :665–676) and the other based on a synthetic data set. Copyright © 2009 John Wiley & Sons, Ltd.     

Derivation of adherence metrics from electronic dosing records

Numerous adherence variables have been created from electronic dosing records hindering synthesis of the vast body of adherence research. To elucidate the mathematical foundation for electronic adherence monitoring and to understand how diverse electronic adherence metrics are related to each other and the underlying construct of adherence behavior. Several representative adherence metrics are derived mathematically and their relationship to the underlying consumption (or dosing event) rate analyzed. Data from a 3-month study of 286 individuals on single-drug antihypertensive therapy are then used to empirically study the statistical properties of several of these electronic adherence metrics. As suggested by their common link to the consumption (or dosing event) rate, the analyzed electronic adherence metrics were generally strongly correlated (r <- .6 and > .4). The lowest correlation (r = .15) involved the ratio of the observed number of doses to the recommended number (called average adherence), which tended to emphasize quantity consumed, and the ratio of the observed to maximum mean squared rate deviation (MSRD ratio), which focused more on dose timing. Despite their different formulations, electronic adherence variables are generally closely correlated. Adherence metrics that average the consumption rate over multiple doses (by summing up the number of doses and dividing by the monitored time) may be less sensitive to short-term fluctuations in medication intake. Metrics that are more sensitive to timing variability may thus be preferable when timing as well as quantity of dosing are of interest.     

Does the concordance between medical records and patient self-report vary with patient characteristics?

Few studies of the concordance between patient self-report and medical record data have examined how concordance varies with patient characteristics, and results of such studies have been mixed. Given discrepancies in the quality of care received across patient cohorts, it is important to understand the degree to which concordance metrics are robust across patient characteristics.We hypothesized that concordance between ambulatory medical record and patient survey data varies by patient demographic characteristics, especially education, income, and race/ethnicity. We present the results of bivariate and multivariate analyses including data from 1,270 patients with at least one of: diabetes, ischemic heart disease, asthma or COPD, or low back pain sampled from 39 West Coast medical organizations.We present total agreement, kappa, and survey sensitivity and specificity, stratified by patient demographic and health status characteristics. We conducted logistic regressions to test the impact of patient demographic characteristics, domain of medical care, and health status on these three measures of concordance. Survey sensitivity varied significantly by race/ethnicity in bivariate analyses, but this effect was erased in multivariate analyses. Our findings do not support the hypothesis that patient education, income, or race/ethnicity have an independent effect on concordance when controlling for other factors. However, concordance varied significantly by patient health status. The medical record and patient self-report do not measure quality comparably across patient cohorts. We recommend continued efforts to improve survey data collection across different patient populations and to improve the quality of clinical data.     

Older Australians' medication use: self-report by phone showed good agreement and accuracy compared with home visit

ObjectiveTo ascertain the accuracy of telephone-interview method for measuring older people's medication use (“self-report by phone”) by determining agreement between results from this method and from a home visit (“home inventory”).Study Design and SettingAn agreement study involving community-dwelling patients aged more than 65 years, selected from four general practices in the Hunter Region of Australia. Commonly used classes of drugs were selected for comparison.ResultsOf 154 patients, 14 participants were ineligible, because they had hearing problems (9) or did not use any medicines (5). The response rate was 70% (98 of 140). The observed overall agreement and prevalence-adjusted and bias-adjusted kappa coefficients were very high for all prescribed drug categories, but lower for over-the-counter (OTC) and complementary medicines. Specificity of the self-report by phone compared with home inventory was consistently high across all drug classes. Sensitivity values were more than 89% for all drug classes but were lower for OTC and complementary medicines (74%) and paracetamol (78%). Similar patterns were found for negative predictive values. Positive predictive values were lower for drugs used on an as-needed basis.ConclusionMeasuring patient's medication use by telephone is an accurate and relatively inexpensive alternative to home-inventory methods, and has merit for use in future studies of older patients' drug use.     

Using automated pharmacy records to assess the management of tuberculosis

We used automated pharmacy dispensing data to characterize tuberculosis (TB) management for 45 health maintenance organization (HMO) members. Pharmacy records distinguished patients treated in HMOs from those treated elsewhere. For cases treated in HMOs, they provided useful information about appropriateness of prescribed regimens and adherence to therapy.     

Using pharmacy data to identify those with chronic conditions in Emilia Romagna, Italy

BACKGROUND AND OBJECTIVES: Automated pharmacy data have been used to develop a measure of chronic disease status in the general population. The objectives of this project were to refine and apply a model of chronic disease identification using Italian automated pharmacy data; to describe how this model may identify patterns of morbidity in Emilia Romagna, a large Italian region; and to compare estimated prevalence rates using pharmacy data with those available from a 2000 Emilia Romagna disease surveillance study. METHODS: Using the Chronic Disease Score, a list of chronic conditions related to the consumption of drugs under the Italian pharmaceutical dispensing system was created. Clinical review identified medication classes within the Italian National Therapeutic Formulary that were linked to the management of each chronic condition. Algorithms were then tested on pharmaceutical claims data from Emilia Romagna for 2001 to verify the applicability of the classification scheme. RESULTS: Thirty-one chronic condition drug groups (CCDGs) were identified. Applying the model to the pharmacy data, approximately 1.5 million individuals (37.1%) of the population were identified as having one or more of the 31 CCDGs. The 31 CCDGs accounted for 77% (E556 million) of 2001 pharmaceutical expenditures. Cardiovascular diseases, rheumatological conditions, chronic respiratory illness, gastrointestinal diseases and psychiatric diseases were the most frequent chronic conditions. External validation comparing rates of the diseases found through using pharmacy data with those of a 2000 Emilia Romagna disease surveillance study showed similar prevalence of illness. CONCLUSIONS: Using Italian automated pharmacy data, a measure of population-based chronic disease status was developed. Applying the model to pharmaceutical claims from Emilia Romagna 2001, a large proportion of the population was identified as having chronic conditions. Pharmacy data may be a valuable alternative to survey data to assess the extent to which large populations are affected by chronic conditions.     

Comparison of general practitioner records and patient self-report questionnaires for estimation of costs

This study compared resource use data and estimated costs over a 12-month period, based on patient self-report questionnaires and general practitioner records. The level of agreement was measured by weighted kappa. Differences in total costs were plotted against mean total costs. Of 324 patients with complete GP records for the period only 85 (26.2%) had complete self-report data. The recorded number of contacts per patient was higher for patient questionnaires than GP records (17.20 vs. 12.64), and the level of agreement between the two sources was moderate (kappa(w)=0.465). The plots of differences in total costs showed a slight downward bias, suggesting that estimation of total cost from GP records is lower than patient questionnaires. The incompleteness of patient questionnaires forces reliance on GP records as the main source of information for economic evaluations.     

Clustering based on adherence data

Adherence to a medical treatment means the extent to which a patient follows the instructions or recommendations by health professionals. There are direct and indirect ways to measure adherence which have been used for clinical management and research. Typically adherence measures are monitored over a long follow-up or treatment period, and some measurements may be missing due to death or other reasons. A natural question then is how to describe adherence behavior over the whole period in a simple way. In the literature, measurements over a period are usually combined just by using averages like percentages of compliant days or percentages of doses taken. In the paper we adapt an approach where patient adherence measures are seen as a stochastic process. Repeated measures are then analyzed as a Markov chain with finite number of states rather than as independent and identically distributed observations, and the transition probabilities between the states are assumed to fully describe the behavior of a patient. The patients can then be clustered or classified using their estimated transition probabilities. These natural clusters can be used to describe the adherence of the patients, to find predictors for adherence, and to predict the future events. The new approach is illustrated and shown to be useful with a simple analysis of a data set from the DART (Development of AntiRetroviral Therapy in Africa) trial in Uganda and Zimbabwe.     

Measures of agreement between many raters for ordinal classifications

Screening and diagnostic procedures often require a physician's subjective interpretation of a patient's test result using an ordered categorical scale to define the patient's disease severity. Because of wide variability observed between physicians' ratings, many large‐scale studies have been conducted to quantify agreement between multiple experts' ordinal classifications in common diagnostic procedures such as mammography. However, very few statistical approaches are available to assess agreement in these large‐scale settings. Many existing summary measures of agreement rely on extensions of Cohen's kappa. These are prone to prevalence and marginal distribution issues, become increasingly complex for more than three experts, or are not easily implemented. Here we propose a model‐based approach to assess agreement in large‐scale studies based upon a framework of ordinal generalized linear mixed models. A summary measure of agreement is proposed for multiple experts assessing the same sample of patients' test results according to an ordered categorical scale. This measure avoids some of the key flaws associated with Cohen's kappa and its extensions. Simulation studies are conducted to demonstrate the validity of the approach with comparison with commonly used agreement measures. The proposed methods are easily implemented using the software package R and are applied to two large‐scale cancer agreement studies. Copyright © 2015 John Wiley & Sons, Ltd.