How to derive and validate clinical prediction models for use in intensive care medicine

Background

Clinical prediction models are formal combinations of historical, physical examination and laboratory or radiographic test data elements designed to accurately estimate the probability that a specific illness is present (diagnostic model), will respond to a form of treatment (therapeutic model) or will have a well-defined outcome (prognostic model) in an individual patient. They are derived and validated using empirical data and used to assist physicians in their clinical decision-making that requires a quantitative assessment of diagnostic, therapeutic or prognostic probabilities at the bedside.

Purpose

To provide intensivists with a comprehensive overview of the empirical development and testing phases that a clinical prediction model must satisfy before its implementation into clinical practice.

Results

The development of a clinical prediction model encompasses three consecutive phases, namely derivation, (external) validation and impact analysis. The derivation phase consists of building a multivariable model, estimating its apparent predictive performance in terms of both calibration and discrimination, and assessing the potential for statistical over-fitting using internal validation techniques (i.e. split-sampling, cross-validation or bootstrapping). External validation consists of testing the predictive performance of a model by assessing its calibration and discrimination in different but plausibly related patients. Impact analysis involves comparative research [i.e. (cluster) randomized trials] to determine whether clinical use of a prediction model affects physician practices, patient outcomes or the cost of healthcare delivery.

Conclusions

This narrative review introduces a checklist of 19 items designed to help intensivists develop and transparently report valid clinical prediction models.     

The emergency medicine resident as paramedic: a prehospital in-field rotation

Prehospital care has undergone a significant evolution during the past two decades and has been transformed from a transportation service into an advanced life support (ALS) delivery system. Crucial to the quality of such a program is physician knowledge and medical control. We describe a formal, one-month prehospital rotation for emergency medicine residents. The resident physician is exposed to a number of varying emergency medical services (EMS) systems, administrative experiences, and most uniquely, functions as a paramedic within our own ALS EMS system. In this manner, we believe the resident best obtains an understanding of the environment, attitudes, and behavior of prehospital personnel.     

International emergency medicine: How to train for it

This article provides background information about the emerging field of international emergency medicine (IEM) and how emergency physicians in Australasia can participate in its practice and development. It reviews the seven key areas of knowledge and skills involved in the practice of IEM as put forward by US fellowship programmes: (i) Emergency Medicine Systems Development; (ii) Humanitarian Relief; (iii) Disaster Management; (iv) Public Health; (v) Travel and Field Medicine; (vi) Programme Administration; and (vii) Academic Skills. Current obstacles to the development of similar programmes in Australasia are explored and identified as primarily financial. Means by which individuals can fund and engage in IEM activities are proposed. This article provides a reference of domestic and international IEM training resources that can be obtained by Australasian emergency physicians and trainees today.     

Social care's impact on emergency medicine: a model to test

Mainly in response to the policy drive to avoid unnecessary acute hospital admissions and delayed discharge on social grounds, there has been a gradual development of social work services attached to emergency departments (EDs) in the UK. In the absence of a clearly articulated evidence base or debate about the roles of ED attached social workers, a model of ED based social work practice and indicative supporting evidence is presented. It is argued that social workers may be able to contribute to the efficiency and effectiveness of hospital services while providing a key point of access to social care services. A number of obstacles remain to the implementation of this model of service, including the narrow focus of current social care practice, the hours that a social work service is normally provided, chronic under-funding, and continuing perverse incentives in the health and social care system. More systematic evidence in the UK context is needed to support the case for change.     

Education research: a primer for educators in emergency medicine

As medical educators strive to adopt an evidence-based, outcomes-driven approach to teaching, education research in emergency medicine (EM) is burgeoning. Many educational challenges prompt specific research questions that are well suited to investigative study, but educators face numerous barriers to translating exciting ideas into research publications. This primer, intended for educators in EM, provides a brief overview of the current scope and essential elements of education research. We present an approach to identifying research problems and conceptual frameworks and defining specific research questions. A common approach to curricular development is reviewed, as well as a fundamental overview of qualitative and quantitative methods that can be applied to educational research questions. Finally, suggestions for disseminating results and overcoming common barriers to conducting research are discussed.     

Review article: A primer for clinical researchers in the emergency department: Part VII. Considering a research higher degree in emergency medicine: How does it work,where to start,what to consider

In this series we address important topics for clinicians who participate in research or are considering research as part of their career path in emergency medicine. While much emergency research is successfully done by clinicians without a research higher degree (RHD), undertaking a master's degree or doctorate allows a research topic to be pursued in greater depth. It also provides a solid basis for a future research career in terms of research quality, advanced skills, academic progression and track record, as well as eligibility for grants and RHD supervision. The decision to undertake a RHD is not an easy one, and requires consideration of the time and cost involved, as well as the impact on a clinician's life plans. However, the expertise provided through a RHD often ultimately complements clinical training and establishes an excellent foundation for future research and career. This article provides an overview of RHDs and what to consider before embarking on one.     

Compensation models in emergency medicine: An ethical perspective

There is considerable diversity in compensation models in the specialty of Emergency Medicine (EM). We review different compensation models and examine moral consequences possibly associated with the use of various models. The article will consider how different models may promote or undermine health care's quadruple aim of providing quality care, improving population health, reducing health care costs, and improving the work-life balance of health care professionals. It will also assess how different models may promote or undermine the basic bioethical principles of beneficence, non-maleficence, respect for autonomy, and justice.     

Expertise in prehospital endotracheal intubation by emergency medicine physicians-Comparing 'proficient performers' and 'experts'

Background

Training requirements to perform safe prehospital endotracheal intubation (ETI) are not clearly known. This study aimed to determine differences in ETI performance between ‘proficient performers’ and ‘experts’ according to the Dreyfus &; Dreyfus framework of expertise. As a model for ‘proficient performers’ EMS physicians with a clinical background in internal medicine were compared to EMS physicians with a background in anaesthesiology as a model for ‘experts’.

Methods

Over a one-year period all ETIs performed by the EMS physicians of our institution were prospectively evaluated. ‘Proficient performers’ and ‘experts’ were compared regarding incidence of difficult ETI, ability to predict difficult ETI, and decision for ETI.

Results

Mean years of professional experience were similar between the physician groups, but the median ETI experience differed significantly with 18/year for ‘proficients’ and 304/year for ‘experts’ (p < 0.001). ‘Proficient performers’ intubated 130 of their 2170 treated patients (6.0%), while ‘experts’ did so in 146 of 1809 cases (8.1%, p = 0.01 for difference). The incidence of difficult ETI was 17.7% for ‘proficient performers’, and 8.9% for ‘experts’ (p < 0.05). In 4 cases ETI was impossible, all managed by ‘proficient performers’, but all patients could be ventilated sufficiently. Unexpected difficult ETI occurred in 6.1% for ‘proficient performers’, and 2.0% for ‘experts’ (p = 0.08).

Conclusions

In a prehospital setting ‘expert’ status was associated with a significantly lower incidence of ‘difficult ETI’ and a higher proportion of ETI decisions. In addition, ability to predict difficult ETI was higher, although non-significant. There was no difference in the incidence of impossible ventilation.     

Review article: A primer for clinical researchers in the emergency department: Part IX. How to conduct a systematic review in the field of emergency medicine

In this series we address important topics for emergency clinicians who either participate in research as part of their work, or use the knowledge generated by research studies. Emergency clinicians are routinely in the position of applying new evidence in clinical practice. With an ever‐increasing volume of evidence generated, this can be problematic when studies are conducted in different settings, and include different patient groups, different interventions and different outcomes. This is made even more difficult when the results of primary research studies do not agree. Systematic reviews are becoming increasingly valuable as they appraise and synthesise research findings using a clear methodology, and summarise the results of primary studies. As such, systematic reviews help translate research findings into clinical practice. This paper provides a practical starting point for understanding the steps involved in conducting a systematic review in emergency medicine and will help readers appraise the findings of systematic reviews.     

The business of emergency medicine: a model for success

Today's EPOs and their physicians face an array of daunting challenges.Falling reimbursement, rising malpractice costs. ED and hospital crowding,and demands for improving ED efficiency and patient satisfaction all contribute to the challenging and sometimes threatening environment of EM practice. The EP involved in a busy and often hectic ED shift may feel unduly and unnecessarily distracted when required to continuously acknowledge and address the business aspect of the practice. Nevertheless,regardless of the size and structure, fiscal viability ultimately determines the EPO's ability to continue to offer access to care. This article contends that a comprehensive business strategy drives superior financial performance and supports the organization's mission. The business strategy must identify financial and non-financial determinants of the EPO's success and provide a mechanism for understanding how the organization's resources are converted to value for customers. The section offers a framework for developing this strategy, for identifying possible gaps or deficiencies, and for measuring and monitoring progress in achieving strategic objectives and ultimately, the EPO's mission.The importance of the mission and the dynamic EM environment require that the strategy development process be more than an annual exercise for the leadership of the organization. Though key leaders in any size EPO--set the course for the organization, the entire organization must be aware and understand the strategy before they commit themselves and adopt actions and behaviors that promote it. The model presented here provides a graphic display that lends itself well to consistent communication of a comprehensive strategy in a concise way throughout the organization.Furthermore, the balance of the model, across four perspectives, recognizes the value of balanced organizational objectives and lends itself well to the creation of a measurement system that supports cause and effect relation-ships through leading and lagging indicators. This can prevent the imposition of unproven, arbitrary metrics and expectations.EPOs that expend the time and resources required to develop and implement a comprehensive, well balanced strategy make a long-term investment in their patients and themselves. EPOs that go on to develop credible, efficient measurement systems are not only able to drive more commitment and accountability in the organization, they also have a competitive advantage. They can demonstrate performance history and targeted future performance to differentiate themselves from competitors and to stem efforts to impose unbalanced metrics that do not serve well the hospital, the EPO, and ultimately, patients.The power of such measurement systems, internal and external to the organization, strongly point to the need for consistently defined measures across the industry. Ultimately when faced with changes in the environment,the entire industry will be better equipped to respond in specific ways and to communicate its progress to communities or government authorities.     

Unit-based research forums: a model for the clinical nurse specialist to promote clinical research

Although nurse researchers have generated research findings, practitioners in clinical practice have not consistently utilized them. Evaluation and modification of clinical practice through research is a key component of the role of the clinical nurse specialist (CNS). In this paper, an innovative model is described in which the Clinical Nurse Specialist coordinates specialty-based clinical research through the use of unit-based research forums, an outgrowth of unit-based quality assurance. Such a model allows staff at the unit level to engage in problem-solving dialogues, resulting in increased use of research findings and generation of original research.