Manual semantic tagging to improve access to information in narrative electronic medical records

OBJECTIVE: To assess effectiveness of information retrieval from narrative patient records utilizing manually supplied semantic tags. DESIGN: Retrospective evaluation of narrative electronic record notes with respect to consistency of manually supplied semantic tags. Assessment of retrieval effectiveness using string matching methods and methods based on structural characteristics of data. SETTING: Department of Neurology in a Norwegian university hospital using a document based electronic patient record system offering template assisted manual semantic indexing of textual record notes. MEASUREMENTS: Proportion of consistently tagged information. Retrieval effectiveness expressed as recall (sensitivity) and precision (positive predictive value). RESULTS: Seventy-five percent of the content was consistently tagged. Retrieval models based on simple string matching performed better than models based on semantic tags alone with respect to best mean recall and mean precision at high recall-levels. Models combining both simple string matching and semantic tagging performed better than separate models with respect to best mean recall (0.95) and mean precision at high levels of recall. CONCLUSION: Although template assisted semantic indexing of narrative electronic patient record notes showed suboptimal consistency, it significantly improved retrieval effectiveness with respect to best mean recall obtained by individual strategies and with respect to precision at any recall level.