A framework and model for evaluating clinical decision support architectures

In this paper, we develop a four-phase model for evaluating architectures for clinical decision support that focuses on: defining a set of desirable features for a decision support architecture; building a proof-of-concept prototype; demonstrating that the architecture is useful by showing that it can be integrated with existing decision support systems and comparing its coverage to that of other architectures. We apply this framework to several well-known decision support architectures, including Arden Syntax, GLIF, SEBASTIAN, and SAGE.     

The EHR-ARCHE project: Satisfying clinical information needs in a Shared Electronic Health Record System based on IHE XDS and Archetypes

PurposeWhile contributing to an improved continuity of care, Shared Electronic Health Record (EHR) systems may also lead to information overload of healthcare providers. Document-oriented architectures, such as the commonly employed IHE XDS profile, which only support information retrieval at the level of documents, are particularly susceptible for this problem. The objective of the EHR-ARCHE project was to develop a methodology and a prototype to efficiently satisfy healthcare providers’ information needs when accessing a patient's Shared EHR during a treatment situation. We especially aimed to investigate whether this objective can be reached by integrating EHR Archetypes into an IHE XDS environment.MethodsUsing methodical triangulation, we first analysed the information needs of healthcare providers, focusing on the treatment of diabetes patients as an exemplary application domain. We then designed ISO/EN 13606 Archetypes covering the identified information needs. To support a content-based search for fine-grained information items within EHR documents, we extended the IHE XDS environment with two additional actors. Finally, we conducted a formative and summative evaluation of our approach within a controlled study.ResultsWe identified 446 frequently needed diabetes-specific information items, representing typical information needs of healthcare providers. We then created 128 Archetypes and 120 EHR documents for two fictive patients. All seven diabetes experts, who evaluated our approach, preferred the content-based search to a conventional XDS search. Success rates of finding relevant information was higher for the content-based search (100% versus 80%) and the latter was also more time-efficient (8–14 min versus 20 min or more).ConclusionsOur results show that for an efficient satisfaction of health care providers’ information needs, a content-based search that rests upon the integration of Archetypes into an IHE XDS-based Shared EHR system is superior to a conventional metadata-based XDS search.     

Factors associated with improved completion of computerized clinical reminders across a large healthcare system

OBJECTIVE: To analyze the relationship of completion rates for a standardized set of computerized clinical reminders across a large healthcare system to practice and provider characteristics. METHODS: The relationship between completion rate for 13 standardized reminders at 49 primary care practices in the VA New England Healthcare System for a 30-day period and practice characteristics, provider demographics and, via survey, provider attitudes was analyzed. RESULTS: There was no difference in clinical reminder completion rate between staff physicians versus nurse practitioners/physician assistants (87.6% versus 88.1%) but both were better than residents (76.6%, p<0.0001). With residents excluded, there were no differences between hospital and community-based clinics or between teaching and non-teaching sites. Clinical reminder completion rate was lower for sites that did not fully utilize support staff in completion process versus sites that did (82.4% versus 88.1%, p<0.0001). Analysis of survey results showed no correlation of completion rate with provider demographics or attitudes towards reminders. However there was significant correlation with frequency of receiving individual feedback on reminder completion (r=0.288, p=0.004). CONCLUSION: Completion of computerized clinical reminders was not affected by a variety of provider characteristics, including professional training, demographics and provider attitude, although was lower among residents than staff providers. However incorporation of support staff into clinic processes and individualized feedback to providers were strongly associated with improved completion. These findings demonstrate the importance of considering practice and provider factors and not just technical elements when implementing informatics tools.     

SANDS: a service-oriented architecture for clinical decision support in a National Health Information Network

In this paper, we describe and evaluate a new distributed architecture for clinical decision support called SANDS (Service-oriented Architecture for NHIN Decision Support), which leverages current health information exchange efforts and is based on the principles of a service-oriented architecture. The architecture allows disparate clinical information systems and clinical decision support systems to be seamlessly integrated over a network according to a set of interfaces and protocols described in this paper. The architecture described is fully defined and developed, and six use cases have been developed and tested using a prototype electronic health record which links to one of the existing prototype National Health Information Networks (NHIN): drug interaction checking, syndromic surveillance, diagnostic decision support, inappropriate prescribing in older adults, information at the point of care and a simple personal health record. Some of these use cases utilize existing decision support systems, which are either commercially or freely available at present, and developed outside of the SANDS project, while other use cases are based on decision support systems developed specifically for the project. Open source code for many of these components is available, and an open source reference parser is also available for comparison and testing of other clinical information systems and clinical decision support systems that wish to implement the SANDS architecture.The SANDS architecture for decision support has several significant advantages over other architectures for clinical decision support. The most salient of these are:

1. Greater modularity than other architectures, allowing for work to be distributed.

2. The potential for creating and sustaining a commercial market for clinical decision support.

3. Reduced cost and risk of trying new decision support systems because of its ability to easily integrate a variety of decision support services, and to easily remove them, if desired, as well.

4. Significant freedom for developers of clinical decision support systems to choose the way they represent knowledge and internally implement their system, in comparison to other approaches which constrain such developers to a particular knowledge representation formalism.

5. Unification of the direction and agenda of decision support research and development with promising near-term efforts to improve interoperability of clinical systems.

Identifying asthma exacerbations in a pediatric emergency department: a feasibility study

BACKGROUND: Asthma is a common pediatric chronic disease and is estimated to account for more than 2million emergency department visits per year. Asthma guidelines have demonstrated improved outcomes, but remain underutilized due to several barriers. Computerized methods to automatically identify asthma exacerbations may be beneficial to initiate guideline recommended treatment, but have not been described. The goal of the study was to examine the accuracy of an algorithm to identify asthma patients at triage in real-time using only electronically available data. METHODS: During a 9-month period, the five most frequent presenting chief complaints for Emergency Department asthma patients aged 2-18 years were identified and accounted for >95% of asthma visits: wheezing, shortness of breath, fever, cough, and dyspnea. During a following 1-month period (November 2004), medical records of all patients with one of the five chief complaints were reviewed to establish a reference standard diagnosis. An asthma identification algorithm was developed that considered only data available in electronic format at the time of triage and included the presenting chief complaint, information from the computerized problem list (past medical history; current medications, such as beta-agonists, steroids, and other asthma medications), and ICD-9 billing codes from previous encounters. RESULTS: From 1835 Emergency Department visits, 368 visits (154 with asthma) had one of the five chief complaints and were included. A problem list was available in 203 (55.2%) and an ICD-9 code in 68 (18.5%) patients. Wheezing accounted for 56.5% of asthma visits, while fever was the most frequent chief complaint among all patients (43.8%). The asthma identification algorithm had a sensitivity of 44.8% (95% CI: 36.8-53.0%), a specificity of 91.6% (CI: 87.0-94.9%), a positive predictive value of 79.3% (CI: 69.3-87.3%) and a negative predictive value of 69.8% (CI: 64.0-75.1%). The positive and negative likelihood ratios were 5.3 (CI: 3.3-8.6) and 0.6 (CI: 0.5-0.7), respectively. CONCLUSION: The simple identification algorithm demonstrated good accuracy for identifying asthma episodes. The algorithm may represent a promising and feasible approach to create computerized reminders or automatic triggers that can facilitate the initiation of guideline-based asthma treatment in the Emergency Department.     

Integrating evidence into clinical information systems for nursing decision support

PURPOSE: To illustrate approaches for providing decision support for evidence-based nursing practice through integration of evidence into clinical information systems (CISs) with examples from our experience at Columbia University Medical Center. ORGANIZING CONSTRUCT: Examples are organized according to three types of decision support functions: information management, focusing attention, and patient-specific consultation. METHODS: Three decision support tools that are integrated into three types of CISs are discussed: (1) infobuttons that provide context-specific access to digital sources of evidence; (2) automated Fall-Injury Risk Assessment; and (3) personal digital assistant-based screening reminders, screening assessments, and tailored documentation templates for the identification and management of obesity, depression, and tobacco cessation. The informatics infrastructure for implementing these decision support tools is described from the perspective of components identified in the published literature. CONCLUSIONS: Efforts to facilitate application of evidence into nursing practice are unlikely to be successful unless the approaches used are integrated into the clinical workflow. Our approaches use a variety of informatics methods to integrate evidence into CISs as a mechanism for providing decision support for evidence-based practice in a manner consistent with nursing workflow.     

Utilization of evidence-based computerized order sets in pediatrics

Little is known about utilization of different evidence-based order sets within computerized physician order entry (CPOE) systems. We designed a retrospective study of resident and attending physician order set utilization to evaluate the use of three evidence-based computerized order sets (asthma, post-appendectomy care, and community-acquired pneumonia (CAP)), and examine patient and admission characteristics associated with order set utilization in pediatrics. We studied all 529 asthma patients, 277 appendectomy patients, and 210 CAP patients admitted between 1 November 2001 and 30 November 2003 during implementation of standardized order sets at a large, independent, not-for-profit pediatric institution. We analyzed order set utilization for the three order sets and tested the relationship between order set use and potential factors associated with utilization. Order set utilization varied by condition (X(2)=339.2, p<0.001), with the asthma order set use rate highest (88.1%), followed by appendectomy order set utilization (79.4%), and substantially lower CAP order set use (21.1%). We found that trends in order set utilization also varied by condition. Only the asthma order set showed a trend of increasing use after implementation (z= -3.02, p=0.002). In addition, factors associated with order set utilization varied. Uses of the asthma and post-appendectomy order sets were associated with factors such as admission unit and case complexity. CAP order set utilization was associated with case complexity but not admission source. We conclude that health services organizations looking to implement computerized order sets to reduce unnecessary practice variation while promoting best practices must consider the different factors that may influence the use of each order set rather than relying on a one-size-fits-all implementation strategy. Further, issues such as the level of physician involvement in order set development and consensus around order set content may be particularly important factors influencing order set utilization.     

A Bayesian model for triage decision support

OBJECTIVE: To compare triage decisions of an automated emergency department triage system with decisions made by an emergency specialist. METHODS: In a retrospective setting, data extracted from charts of 90 patients with chief complaint of non-traumatic abdominal pain were used as input for triage system and emergency medicine specialist. The final disposition and diagnoses of the physicians who visited the patient in Emergency Department (ED) as reflected in the medical records were considered as control. Results were compared by chi(2)-test and a binary logistic regression model. RESULTS: Compared to emergency medicine specialist, triage system had higher sensitivity (90% versus 64%) and lower specificity (25% versus 48%) for patients who required hospitalization. The triage system successfully predicted the Admit decisions made in the ED whereas the emergency medicine specialist decisions could not predict the ED disposition. Both triage system and emergency medicine specialist properly disposed 56% of cases, however, the emergency medicine specialist in this study under-disposed more patients than the triage system considering Admit disposition (p=0.004) while he appropriately discharged more patients compared to the triage system (p=0.017). CONCLUSION: The triage system studied here shows promise as a triage decision support tool to be used for telephone triage and triage in the emergency departments. This technology may also be useful to the patients as a self-triage tool. However, the efficiency of this particular application of this technology is unclear.     

Quantitative evaluation of clinical software, exemplified by decision support systems

To ensure the correctness of publicity material (‘truth in labelling’) and to inform their licensing decisions, agencies certifying or regulating any clinical computer system will need information about the system's structure, performance and likely impact on users and the environment in which they work. This information must be reliable and complete, so it needs to be collected in a structured, rigorous evaluation programme. Clinical decision support systems are generally more complex and their effects less easy to predict than most other clinical software, so pose the greatest challenge to evaluators. They are therefore the focus of this paper.     

Evacuation support system for improved medical documentation and information flow in the field

PURPOSE: Documentation of medical treatment and observation of patients during evacuation from the point of injury to definitive treatment is important both for optimizing patient treatment and managing the evacuation process. The current practice in military medical field documentation uses paper forms and voice communication. There are many shortcomings associated with this approach, especially with respect to information capture and sharing processes. Current research addresses the use of new technology for civilian ambulance-to-hospital communication. The research work presented in this article addresses information capture and sharing in extreme military conditions by evaluating a targeted computerized information system called EvacSys during a military exercise in northern Norway in December 2003. METHODS AND MATERIALS: EvacSys was designed and implemented in close cooperation with military medical personnel in both Norway and the USA. The system was evaluated and compared to the traditional paper-based documentation method during a military exercise. The on-site evaluation was conducted in a military medical platoon in the Norwegian Armed Forces, using questionnaires, semi-structured interviews, observation and video recording to capture the users' system acceptance. RESULTS: A prototype software system running on a commercial off-the-shelf hardware platform was successfully developed. The evaluation of this system shows that the usability of digital information capturing and sharing are perceived to be at least as good as the traditional paper-based method. The medics found the new digital method to be more viable than the old one. No technical problems were encountered. CONCLUSIONS: Our research shows that it is feasible to utilize digital information systems for medical documentation in extreme outdoor environments. The usability concern is of utmost importance, and more research should be put into the design and alignment with existing workflow. Successful digitalization of information at the point of care will provide accurate and timely information for the management of resources during disaster response.     

A tale of two studies: the importance of setting, subjects and context in two randomized, controlled trials of a web-based decision support for perimenopausal and postmenopausal health decisions

OBJECTIVE: Knowledge relevant to women's peri- and postmenopausal health decisions has been evolving rapidly. Web-based decision supports can be rapidly updated and have the potential to improve the quality of patients' decisions. We developed and tested a web-based decision support for peri- and postmenopausal health decisionmaking. METHODS: We recruited 409 women aged 45-75 for one randomized, controlled trial and 54 women with an upcoming clinic appointment for a subsequent trial. Women were randomized to use the web-based decision support versus a printed brochure (first trial) and usual care (second trial). Outcomes were changes in decisional satisfaction, decisional conflict, and knowledge, both within each trial and compared across the trials. RESULTS: Intervention subjects had greater increases in decisional satisfaction in the second trial and knowledge in both trials. A high dropout rate among women randomized to the website in the first trial effectively negated benefits in that trial, but not in the second. CONCLUSIONS: The utility of this web-based decision support in two trials depended on a number of factors that appear related to the urgency of making a decision. PRACTICE IMPLICATIONS: Decision aids should be targeted to patients actively trying to make a decision.     

Development and field testing of a decision support tool to facilitate shared decision making in contraceptive counseling

ObjectiveWe developed and formatively evaluated a tablet-based decision support tool for use by women prior to a contraceptive counseling visit to help them engage in shared decision making regarding method selection.MethodsDrawing upon formative work around women’s preferences for contraceptive counseling and conceptual understanding of health care decision making, we iteratively developed a storyboard and then digital prototypes, based on best practices for decision support tool development. Pilot testing using both quantitative and qualitative data and cognitive testing was conducted. We obtained feedback from patient and provider advisory groups throughout the development process.ResultsNinety-six percent of women who used the tool in pilot testing reported that it helped them choose a method, and qualitative interviews indicated acceptability of the tool’s content and presentation. Compared to the control group, women who used the tool demonstrated trends toward increased likelihood of complete satisfaction with their method. Participant responses to cognitive testing were used in tool refinement.ConclusionOur decision support tool appears acceptable to women in the family planning setting.Practice implicationsFormative evaluation of the tool supports its utility among patients making contraceptive decisions, which can be further evaluated in a randomized controlled trial.     

Evaluation of a proximity card authentication system for health care settings

BackgroundMultiple users access computer workstations in busy clinical settings, requiring many logins throughout the day as users switch from one computer to another. This can lead to workflow inefficiencies as well as security concerns resulting from users sharing login sessions to save time. Proximity cards and readers have the potential to improve efficiency and security by allowing users to access clinical workstations simply by bringing the card near the reader, without the need for manual entry of a username and password.ObjectivesTo assess the perceived impact of proximity cards and readers for rapid user authentication to clinical workstations in the setting of an existing electronic health record with single sign-on software already installed.MethodsQuestionnaires were administered to clinical faculty and staff five months before and three months after the installation of proximity card readers in an inpatient birthing center and an outpatient obstetrics clinic. Open-ended feedback was also collected and qualitatively analyzed.ResultsThere were 71 and 33 responses to the pre- and post-implementation surveys, respectively. There was a significant increase in the perceived speed of login with the proximity cards, and a significant decrease in the self-reported occurrence of shared login sessions between users. Feedback regarding the system was mostly positive, although several caveats were noted, including minimal benefit when used with an obstetric application that did not support single sign-on.ConclusionsProximity cards and readers, along with single sign-on software, have the potential to enhance workflow efficiency by allowing for faster login times and diminish security concerns by reducing shared logins on clinical workstations. The positive feedback was used by our health system leadership to support the expanded implementation of the proximity card readers throughout the clinical setting.     

Comparison of Electronic Health Record System Functionalities to support the patient recruitment process in clinical trials

ObjectivesReusing data from electronic health records for clinical and translational research and especially for patient recruitment has been tackled in a broader manner since about a decade. Most projects found in the literature however focus on standalone systems and proprietary implementations at one particular institution often for only one singular trial and no generic evaluation of EHR systems for their applicability to support the patient recruitment process does yet exist. Thus we sought to assess whether the current generation of EHR systems in Germany provides modules/tools, which can readily be applied for IT-supported patient recruitment scenarios.MethodsWe first analysed the EHR portfolio implemented at German University Hospitals and then selected 5 sites with five different EHR implementations covering all major commercial systems applied in German University Hospitals. Further, major functionalities required for patient recruitment support have been defined and the five sample EHRs and their standard tools have been compared to the major functionalities.ResultsIn our analysis of the site's hospital information system environments (with four commercial EHR systems and one self-developed system) we found that – even though no dedicated module for patient recruitment has been provided – most EHR products comprise generic tools such as workflow engines, querying capabilities, report generators and direct SQL-based database access which can be applied as query modules, screening lists and notification components for patient recruitment support. A major limitation of all current EHR products however is that they provide no dedicated data structures and functionalities for implementing and maintaining a local trial registry.ConclusionsAt the five sites with standard EHR tools the typical functionalities of the patient recruitment process could be mostly implemented. However, no EHR component is yet directly dedicated to support research requirements such as patient recruitment. We recommend for future developments that EHR customers and vendors focus much more on the provision of dedicated patient recruitment modules.     

A systematic review of menopausal symptom management decision aid trials

Objective

To systematically review the literature regarding the effects of menopausal symptom management decision aids.

Methods

Using pre-designated inclusion and exclusion criteria, relevant articles were located using the PubMed.gov online search engine and reviewing reference lists of relevant articles. Full-text, English-language, peer-reviewed articles relevant to testing decision aids in uncontrolled trials (UCT) and randomized controlled trials (RCT) were reviewed.

Results

The 18 articles represented 15 trials focused on natural health products decision aids (1 UCT, 1 RCT) or hormone therapy decision aids (1 UCT, 12 RCT). Whereas the natural health products aid was intended for women deciding about menopausal symptom management strategies, decision aids for hormone therapy were intended for a broader group of menopausal women and included indications for symptom management, prevention of heart disease, and prevention of osteoporosis. Many trials occurred prior to two pivotal events: the 2002 announcement of the Women's Health Initiative findings and the 2006 publication of the International Patient Decision Aids Standards. Study limitations may help explain contradictory findings for outcomes such as decisional conflict, decisional confidence, decisional satisfaction, knowledge and values, and decisions.

Conclusions

There is a relatively scant contemporary literature related to menopausal symptom management decision aids. Additional methodologically sound studies are needed to develop and subsequently test decision aids that are based on (a) contemporary knowledge regarding the wide array of available therapies and (b) international standards for decision aids that include consideration of women's values and preferences.     

A fusion-based clinical decision support for disease diagnosis from endoscopic images

This paper presents an intelligent decision support system designed on a decision fusion framework coupled with a priori knowledge base for abnormality detection from endoscopic images. Sub-decisions are made based on associated component feature sets derived from the endoscopic images and predefined algorithms, and subsequently fused to classify the patient state. Bayesian probability computations are employed to evaluate the accuracies of sub-decisions, which are utilized in estimating the probability of the fused decision. The overall detectability of abnormalities by using the proposed fusion approach is improved in terms of detection of true positive and true negative conditions when compared with corresponding results from individual methods.     

Clinicians' perceptions of clinical decision support integrated into computerized provider order entry

Computerized provider order entry systems can improve patient care by integrating clinical decision support. Decision support is most effective when its content and delivery are acceptable. The authors report the results of a multifaceted survey of the attitudes of housestaff and medical student users of a provider order entry system with integrated decision support at an academic medical center. The survey contained 16 items covering four themes: efficiency, quality of care, results reporting and embedded guidelines. Responses were captured using a five point Likert scale, and were compared using ANOVA and Bartlett's testing. Out of 491 housestaff and 128 medical students, response rates were 47 and 29%, respectively. Among respondents, 72% agreed or strongly agreed that the provider order entry system improves the quality of care that they provide, 54% that the decision support usually help them to provide quality patient care, and 62% that it improves the efficiency of order entry. Respondents were least likely to agree that the display of prior laboratory results influenced their decision to order a subsequent test. There were no significant differences between subspecialties and by advancing years of training among housestaff. Respondents agreed that the integrated clinical decision support enhanced their medical training.