Methods for assessing reliability and validity for a measurement tool: a case study and critique using the WHO haemoglobin colour scale

Before introducing a new measurement tool it is necessary to evaluate its performance. Several statistical methods have been developed, or used, to evaluate the reliability and validity of a new assessment method in such circumstances. In this paper we review some commonly used methods. Data from a study that was conducted to evaluate the usefulness of a specific measurement tool (the WHO Colour Scale) is then used to illustrate the application of these methods. The WHO Colour Scale was developed under the auspices of the WHO to provide a simple portable and reliable method of detecting anaemia. This Colour Scale is a discrete interval scale, whereas the actual haemoglobin values it is used to estimate are on a continuous interval scale and can be measured accurately using electrical laboratory equipment. The methods we consider are: linear regression, correlation coefficients, paired t-tests plotting differences against mean values and deriving limits of agreement; kappa and weighted kappa statistics, sensitivity and specificity, an intraclass correlation coefficient and the repeatability coefficient. We note that although the definition and properties of each of these methods is well established inappropriate methods continue to be used in medical literature for assessing reliability and validity, as evidenced in the context of the evaluation of the WHO Colour Scale.     

Cost effectiveness of interventions for lateral epicondylitis: results from a randomised controlled trial in primary care

OBJECTIVE: Lateral epicondylitis is a common complaint, with an annual incidence between 1% and 3% in the general population. The Dutch College of General Practitioners in The Netherlands has issued guidelines that recommend a wait-and-see policy. However, these guidelines are not evidence based. DESIGN AND SETTING: This paper presents the results of an economic evaluation in conjunction with a randomised controlled trial to evaluate the effects of three interventions in primary care for patients with lateral epicondylitis. PATIENTS AND INTERVENTIONS: Patients with pain at the lateral side of the elbow were randomised to one of three interventions: a wait-and-see policy, corticosteroid injections or physiotherapy. MAIN OUTCOME MEASURES AND RESULTS: Clinical outcomes included general improvement, pain during the day, elbow disability and QOL. The economic evaluation was conducted from a societal perspective. Direct and indirect costs (in 1999 values) were measured by means of cost diaries over a period of 12 months. Differences in mean costs between groups were evaluated by applying non-parametric bootstrap techniques. The mean total costs per patient for corticosteroid injections were euro430, compared with euro631 for the wait-and-see policy and euro921 for physiotherapy. After 12 months, the success rate in the physiotherapy group (91%) was significantly higher than in the injection group (69%), but only slightly higher than in the wait-and-see group (83%). The differences in costs and effects showed no dominance for any of the three groups. The incremental cost-utility ratios were (approximately): euro7000 per utility gain for the wait-and-see policy versus corticosteroid injections; euro12000 per utility gain for physiotherapy versus corticosteroid injections, and euro34500 for physiotherapy versus the wait-and-see policy. CONCLUSIONS: The results of this economic evaluation provided no reason to update or amend the Dutch guidelines for GPs, which recommend a wait-and-see policy for patients with lateral epicondylitis.     

The role of cyclooxygenase in n-6 and n-3 polyunsaturated fatty acid mediated effects on cell proliferation,PGE(2) synthesis and cytotoxicity in human colorectal carcinoma cell lines

This study was conducted to investigate the role of the enzyme cyclooxygenase (COX) and its prostaglandin product PGE(2) in n-6 and n-3 polyunsaturated fatty acid (PUFA)-mediated effects on cellular proliferation of two human colorectal carcinoma cell lines. The long chain PUFAs eicosapentaenoic acid (EPA; 20:5n-3) and arachidonic acid (AA; 20:4n-6) both inhibited cell proliferation of Caco-2 cells compared with the long chain fatty acids alpha-linolenic acid (ALA; 18:3n-3) and linoleic acid (LA; 18:2n-6). Neither incubation with PGE(2) nor reduction in PGE(2) synthesis by EPA compared with AA led to differential effects on cell proliferation in Caco-2 cells. This suggests that n-6 and n-3 PUFA-mediated cell proliferation in Caco-2 cells is not regulated via PGE(2) levels. AA and EPA had no effect on growth of HT-29 colon cancer cells with a low COX activity. However, stimulation of COX-2 activity by IL-1 beta resulted in a decrease in cell proliferation and an induction of cytotoxicity by AA as well as by EPA. Both inhibition of the COX pathway by indomethacin as well as inhibition of direct lipid peroxidation by antioxidants such as vitamin E and C diminished the anti-proliferative effects of AA as well as EPA. Also, malondialdehyde, a product of lipid peroxidation and COX-activity was decreased by addition of vitamin E and partially decreased by indomethacin. These data support the hypothesis that growth inhibitory and cytotoxic effects of PUFAs with methylene-interrupted double bonds such as AA and EPA are due to peroxidation products that are generated during lipid peroxidation and COX activity.     

Treatment of vasovagal syncope: pacemaker or crossing legs?

A 50-year-old male patient continued to experience syncope after implantation of a pacemaker. During cardiovascular examination, the patient showed a typical vasovagal response, with normal pacemaker function. Leg crossing, which prohibits the pooling of blood in the legs and abdomen, at the onset of symptoms helped to prevent this response. The authors recommend a course of leg crossing as a measure to treat vasovagal syncope.     

Combination of calcium channel blockers and beta blockers for patients with exercise-induced angina pectoris: a double-blind parallel-group comparison of different classes of calcium channel blockers. The Netherlands Working Group on Cardiovascular Research (WCN).

The combination of calcium channel blockers and beta blockers is more effective for the treatment of exercise-induced angina pectoris than beta blocker monotherapy. Since ischemia in exercise-induced angina is essentially preceded by an increase in heart rate, calcium channel blockers with negative chronotropic property may perform better for this purpose than nonchronotropic compounds. A 335-patient, 10-week, double-blind, parallel-group comparison of amlodipine 5 and 10 mg, diltiazem XR 200 and 300 mg, and mibefradil 50 and 100 mg treatment added to baseline beta blocker treatment was performed. Exercise testing (ETT) was performed by bicycle ergometry. Although none of the calcium channel blockers improved duration of exercise or amount of workload, all of them significantly delayed onset of 1 mm ST segment depression on ETT (p<0.001 for any treatment versus baseline). In addition, mibefradil, both low- and high-dose treatment, produced the largest delays (low dose: different from diltiazem and amlodipine by 24.1 and 29.8 s, p<0.003 and <0.001, respectively; high dose: different from diltiazem and amlodipine by 33.7 and 37.0 s, p<0.001 and <0.001, respectively). These effects were linearly correlated to the amount of rate pressure product (RPP) reduction. Serious symptoms of dizziness likewise occurred significantly more frequently with mibefradil (p<0.05) and led 19 patients taking mibefradil to withdraw from the trial. The authors conclude that calcium channel blockers with negative chronotropic property provide better delay of ischemia in patients with exercise-induced angina but that the concomitant risk of intolerable dizziness largely reduces this benefit.     

Criteria-based audit to improve a district referral system in Malawi: A pilot study

Background  

To study the feasibility of using criteria-based audit to improve a district referral system.     

Is fatigue in Marfan syndrome related to orthostatic intolerance?

Patients with Marfan syndrome have a tall stature, which could be associated with low orthostatic tolerance. Fatigue, a common complaint of these patients, is also related to orthostatic intolerance. Treatment with beta-blockers, to prevent aortic complications, could be a reinforcing factor of both. This study aimed to investigate (1) the relationship between symptoms of orthostatic tolerance and in patients with Marfan syndrome, and (2) whether termination of beta-blocker therapy improves orthostatic tolerance. Symptoms of fatigue and orthostatic complaints were assessed in 49 subjects using questionnaires (MFI-20 and Autonomic Symptoms Profile). Marfan patients have a high level of fatigue and orthostatic complaints when compared to the general population. Fatigue and orthostatic tolerance are significantly correlated. Orthostatic tolerance was assessed in 9 additional subjects by an active-standing test and head-up tilt for 5 minutes, and 24 hours blood pressure monitoring, once during beta-blocker therapy and once after ceasing beta-blockers for 2 weeks. During hemodynamic testing Marfan patients frequently showed Initial Orthostatic Hypotension and an abnormally high initial heart rate response. Ceasing beta-blockers did not affect the initial blood pressure response. Patients with Marfan syndrome are fatigued and have low orthostatic tolerance, which are significantly correlated. Patients could be educated in physical counterpressure maneuvers to increase orthostatic tolerance.     

Leg crossing with muscle tensing, a physical counter-manoeuvre to prevent syncope, enhances leg blood flow

In patients with orthostatic intolerance, the mechanisms to maintain BP (blood pressure) fail. A physical counter-manoeuvre to postpone or even prevent orthostatic intolerance in these patients is leg crossing combined with muscle tensing. Although the central haemodynamic effects of physical counter-manoeuvres are well documented, not much is known about the peripheral haemodynamic events. Therefore the purpose of the present study was to examine the peripheral haemodynamic effects of leg crossing combined with muscle tensing during 70 degrees head-up tilt. Healthy subjects (n=13) were monitored for 10 min in the supine position followed by 10 min in 70 degrees head-up tilt and, finally, for 2 min of leg crossing with muscle tensing in 70 degrees head-up tilt. MAP (mean arterial BP), heart rate, stroke volume, cardiac output and total peripheral resistance were measured continuously by Portapres. Leg blood flow was measured using Doppler ultrasound. Leg vascular conductance was calculated as leg blood flow/MAP. A significant increase in MAP (13 mmHg), stroke volume (27%) and cardiac output (18%), a significant decrease in heart rate (-5 beats/min) and no change in total peripheral resistance during the physical counter-manoeuvre were observed when compared with baseline 70 degrees head-up tilt. A significant increase in leg blood flow (325 ml/min) and leg vascular conductance (2.9 arbitrary units) were seen during the physical counter-manoeuvre when compared with baseline 70 degrees head-up tilt. In conclusion, the present study indicates that the physical counter-manoeuvre of leg crossing combined with muscle tensing clearly enhances leg blood flow and, at the same time, elevates MAP.