A trial of structured implicit review of randomly selected peer review organization cases

OBJECTIVE: In order to learn more about peer review's acceptability, efficiency, and reliability, we performed structured implicit review with and without use of a structured case review form on a random selection of peer review organization cases. METHODS: We compared the results between methods and with previously obtained review results. Twenty-five charts with physician review completed during the Health Care Financing Administration's Third Scope of Work were randomly selected for rereview. Eight physician advisors, none of whom had seen any of these charts previously, were divided into two groups. Both groups received identical formal instruction in the structured implicit review method. Half of the physicians used a structured review form when performing chart review. The other half did not use this form but completed their reviews using the previously used reporting form. Participating physicians were instructed in the structured review method as described by Rand Corporation. The review process was examined regarding acceptability and efficiency. Review results were analyzed for reliability regarding identification of adverse and potential adverse effects and identification of the source of quality concerns. RESULTS: Instructions regarding structured implicit review methods were understood easily and accepted by physician advisors. Use of the structured review form was less efficient, averaging 50% longer per review. There was no difference in the rate at which adverse events were detected. Potentially serious adverse events were found less often using structured review than in the original review. There was greater agreement among reviewers using the structured form than among those using the historic worksheet, but structured review using the Rand form identified fewer potentially significant adverse events than did the reviewers using the historic worksheets. CONCLUSIONS: Application of structured implicit review methods is clearly feasible for peer review organization case review. Use of a simple worksheet was more time efficient than use of a highly structured form. There was not only less variation in review results but also identification of fewer potentially significant adverse events when the highly structured form was used. Teaching the structured approach to chart review may be more important to obtaining good results than using a structured review form.