Concordance of respiratory care plans generated by protocols from different hospitals: a comparative study

OBJECTIVE: To assess whether respiratory care protocols from different hospitals result in similar care plans for identical patients, we asked: 1. Does applying respiratory care protocols from different hospitals to standardized patient vignettes produce identical care plans? 2. If there are differences in the care plans produced, what is the extent of the difference, and for which modalities are the differences greatest? 3. Does installing the protocol in a computerized information management system to generate the respiratory care plan improve the level of agreement? 4. Do protocols from different hospitals agree with regard to indications for respiratory care treatments and use of the Clinical Practice Guidelines from the American Association for Respiratory Care? METHODS: Protocols were compared by applying each of 4 hospitals' protocols to 15 patient vignettes that we developed, with various respiratory problems. With each vignette, 3 experienced respiratory therapist evaluators developed respiratory care plans, using both a manual (paper-based) and a computer-aided approach. RESULTS: The overall degree of agreement among the 4 protocols was moderate (kappa 0.60, 95% confidence interval 0.46-0.71). The degree of concordance differed for the individual respiratory care modalities; concordance was generally highest for oxygen, aerosol delivery, and pulse oximetry, and was lower for bronchopulmonary hygiene and hyperinflation. Concordance regarding indications for therapy also differed among the modalities; concordance was greatest for the indications for incentive spirometry, bronchodilator use, and pulse oximetry. The concordance of care plans developed with the computer-aided approach resembled that of the manual approach (kappa 0.62, 95% confidence interval 0.45-0.77). CONCLUSIONS: Our results suggest moderate agreement between care plans generated with respiratory care protocols from different hospitals. The sources of differences included differences in the indications for therapy, different degrees of protocol compliance with the American Association for Respiratory Care Clinical Practice Guidelines, and subjectivity in the indications for therapy. This study identifies opportunities to lessen regional variation in respiratory care, by encouraging uniform application of protocols and evidence-based guidelines.