Development of the Engage with Impact Toolkit: A comprehensive resource to support the evaluation of patient,family and caregiver engagement in health systems

Introduction

Recent shifts in the patient, family and caregiver engagement field have focused greater attention on measurement and evaluation, including the impacts of engagement efforts. Current evaluation tools offer limited support to organizations seeking to reorient their efforts in this way. We addressed this gap through the development of an impact measurement framework and accompanying evaluation toolkit—the Engage with Impact Toolkit.

Methods

The measurement framework and toolkit were co-designed with the Evaluating Patient Engagement Working Group, a multidisciplinary group of patient, family and caregiver partners, engagement specialists, researchers and government personnel. Project activities occurred over four phases: (1) project scoping and literature review; (2) modified concept mapping; (3) working group deliberations and (4) toolkit web design.

Results

The project scope was to develop a measurement framework and an evaluation toolkit for patient engagement in health systems that were practical, accessible, menu-driven and aligned with current system priorities. Concept mapping yielded 237 impact statements that were sorted, discussed and combined into 81 unique items. A shorter list of 50 items (rated 8.0 or higher out of 10) was further consolidated to generate a final list of 35 items mapped across 8 conceptual domains of impact: (1) knowledge and skills; (2) confidence and trust; (3) equity and inclusivity; (4) priorities and decisions; (5) effectiveness and efficiency; (6) patient-centredness; (7) culture change and (8) patient outcomes and experience. Working Group members rated the final list for importance (1–5) and identified a core set of 33 items (one for each of the 8 domains and 25 supplementary items). Two domains (priorities and decisions; and culture change) yielded the highest overall importance ratings (4.8). A web-based toolkit ( www.evaluateengagement.ca ) hosts the measurement framework and related evaluation supports.

Conclusion

The Engage with Impact Toolkit builds on existing engagement evaluation tools but brings a more explicit focus to supporting organizations to assess the impacts of their engagement work.

Patient Contribution

Patient, family and caregiver partners led the early conceptualization of this work and were involved at all stages and in all aspects of the work. As end-users of the toolkit, their perspectives, knowledge and opinions were critical.     

Terms of engagement for working with patients in a person-centred partnership: A secondary analysis of qualitative data

Evidence is emerging of the potential of person-centred approaches to create partnerships between professionals and patients while also containing healthcare costs. This is important for enhancing outcomes in individuals with complex needs, who consistently report poor experiences with care. The shift towards person-centred care (PCC) is, however, a radical departure from the norm, with increased expectations of both professional and patient. Although there have been studies on the ways in which health care professionals can modify practice to enhance PCC, not all patients welcome changes to their care delivery or understand the aim of the new approach. Without engagement and understanding from the patient, a PCC approach will fail to initiate. Few studies explore how, why and in what circumstances patients become more involved in their care and what professionals can do to enhance participation. We conducted a secondary analysis of qualitative data to examine this issue. Data were collected between 2014 and 2018 from primary care-based PCC projects across the southwest of England. Supported by people with experience (practitioners and those receiving treatment), theory building workshops developed an explanatory framework that identified contextual factors and mechanisms likely to contribute to effective engagement. Our results show that engagement in a care partnership is achieved through trust and a patient's sense of candidacy. Shared understanding of purpose, clarity of expectations and power sharing were found to facilitate trusted relationships between professional and patient and encourage candidacy. Only then is it possible to develop goals that are meaningful to the patient. Our theory of engagement applies to professionals and patients alike but places the initial burden of responsibility on those who hold the most power: the professional and the system. This theory has the potential to explain patient engagement in PCC and a range of other service interventions, treatments and intervention research.     

Patients as partners in health research: A scoping review

BackgroundThe role of patient involvement in health research has evolved over the past decade. Despite efforts to engage patients as partners, the role is not well understood. We undertook this review to understand the engagement practices of patients who assume roles as partners in health research.MethodsUsing a recognized methodological approach, two academic databases (MEDLINE and EMBASE) and grey literature sources were searched. Findings were organized into one of the three higher levels of engagement, described by the Patient and Researcher Engagement framework developed by Manafo. We examined and quantified the supportive strategies used during involvement, used thematic analysis as described by Braun and Clarke and themed the purpose of engagement, and categorized the reported outcomes according to the CIHR Engagement Framework.ResultsOut of 6621 records, 119 sources were included in the review. Thematic analysis of the purpose of engagement revealed five themes: documenting and advancing PPI, relevance of research, co‐building, capacity building and impact on research. Improved research design was the most common reported outcome and the most common role for patient partners was as members of the research team, and the most commonly used strategy to support involvement was by meetings.ConclusionThe evidence collected during this review advanced our understanding of the engagement of patients as research partners. As patient involvement becomes more mainstream, this knowledge will aid researchers and policy‐makers in the development of approaches and tools to support engagement.Patient/User InvolvementPatients led and conducted the grey literature search, including the synthesis and interpretation of the findings.     

Youth and family engagement in childhood disability evidence syntheses: A scoping review

Within the last decade, stakeholder engagement in research has become increasingly popular in childhood disability research; however, literature on the engagement of youth with neurodisabilities and their families in evidence syntheses is underdeveloped. Involving patients as partners in research has the potential to improve applicability and relevance of the research and benefit patient partners (e.g. enhanced self-esteem, increased research knowledge and skills); however, the methods, challenges, outcomes and recommendations of engaging youth with neurodisabilities and their families in evidence syntheses are unknown. Two parents of youth with complex disability needs were engaged as partners throughout this review. Following methods outlined by Arksey and O'Malley (2005), the primary research question in this scoping review is twofold: (i) what activities have youth with neurodisabilities and their families been engaged in as part of evidence syntheses and (ii) what were the outcomes of that engagement? After full text review of 369 articles, nine articles were included. Youth and families were engaged prior to the evidence synthesis and at every stage in the project, most often during data analysis where they contextualized the findings. Youth and family engagement were not formally evaluated; however, positive outcomes were reported by parents and researchers. Challenges such as increased time, sustaining engagement, and parents' dissatisfaction with their level of involvement were reported. Recommendations centred around providing partners with information, building relationships via social media, and openly communicating about roles, feedback and logistics. Childhood disability researchers should be aware of how they can increase engagement opportunities at all stages of evidence syntheses and how they might improve accessibility for youth with neurodisabilities and their families. Further research is needed to solidify a unified framework for conduct and reporting of youth and family engagement in evidence syntheses.     

Value to Whom? The Patient Voice in the Value Discussion

Background

Professional societies and other organizations have recently taken a visible role trying to define treatment value via value frameworks and assessments, providing payer or provider recommendations, and potentially impacting patient access. Patient perspectives routinely differ from those of other stakeholders. Yet, it is not always apparent that patients were engaged in value framework development or assessment.

Emerging Good Practices for Transforming Value Assessment: Patients’ Voices,Patients’ Values

Background

Patient engagement is a transformative strategy for improving value assessment. US value framework developers have increased engagement activities, but more needs to be learned about how to best achieve meaningful patient engagement in value assessment. The objective was to glean good practices in patient engagement emerging from patient community experiences, to be used in value assessment.

Optimizing patient involvement in quality improvement

Introduction

Patient and public involvement in healthcare planning, service development and health‐related research has received significant attention. However, evidence about the role of patient involvement in quality improvement work is more limited. We aimed to characterize patient involvement in three improvement projects and to identify strengths and weaknesses of contrasting approaches.

Methods

Three case study quality improvement projects were purposively sampled from a broader programme. We used an ethnographic approach involving 126 in‐depth interviews, 12 weeks of non‐participant observations and documentary analysis. Data analysis was based on the constant comparative method.

Results

The three projects differed in the ways they involved patients in their quality improvement work, including their rationales for including patients. We characterized three very different models of patient involvement, which were each influenced by project context. Patients played distinctive roles across the three projects, acting in some cases as intermediaries between the wider patient community and clinicians, and sometimes undertaking persuasive work to convince clinicians of the need for change. We identified specific strategies that can be used to help ensure that patient involvement works most effectively and that the enthusiasm of patients to make a difference is not dissipated.

Conclusion

Patient involvement in quality improvement work needs careful management to realize its full potential.     

Creating opportunities for patient participation in managing medications across transitions of care through formal and informal modes of communication

BackgroundCommunicating about medications across transitions of care is important in older patients who frequently move between health care settings. While there is increasing interest in understanding patient communication across transitions of care, little is known about older patients'' involvement in formal and informal modes of communication regarding managing medications.ObjectiveThe aim of this paper was to explore how older patients participated in managing their medications across transitions of care through formal and informal modes of communication.MethodsThe study was conducted across two metropolitan hospitals: an acute hospital and a geriatric rehabilitation hospital in metropolitan Melbourne, Australia. A focused ethnographic design was used involving semi‐structured interviews (n = 50), observations (203 h) and individual interviews or focus groups (n = 25). Following thematic analysis, data were analysed using Fairclough''s Critical Discourse Analysis.ResultsData analysis revealed two major discursive practices, which comprised of an interplay between formal and informal communication and environmental influences on formal and informal communication. Self‐created patient notes were used by older patients to initiate informal discussion with health professionals about medication decisions, which challenged traditional unequal power relations between health professionals and patients. Formal prompts on electronic medication administration records facilitated the continuous information discourse about patients'' medications across transitions of care and encouraged health professionals to seek out older patients'' preferences through informal bedside interactions. Environmental influences on communication comprised health professionals'' physical movements across private and public spaces in the ward, their distance from older patients at the bedside and utilization of the computer systems during patient encounters.ConclusionOlder patients'' self‐created medication notes enabled them to take on a more active role in formal and informal medication communication across transitions of care. Older patients and family members did not have continuous access to information about medication changes during their hospital stay and systems often failed to address older patients'' key concerns about their medications, which hindered their active involvement in formal and informal communication.Patient or Public ContributionOlder adults, family members and health professionals volunteered to be interviewed and observed.     

The effects of a mobile application for patient participation to improve patient safety

BackgroundPatient participation in patient safety activities in care processes is a fundamental element of safer care. Patients play an important role in preventing patient safety incidents and improving health outcomes. Therefore, healthcare providers need to develop and provide educational materials and actionable tools for patient participation.ObjectivesThis study aimed to develop a mobile application for health consumers'' participation and evaluate the effect of the mobile application on improving health consumers'' participation in patient safety.MethodsA quasi‐experimental design was adopted. We developed a mobile application on the basis of a needs assessment, literature review, compilation of patient safety topics, and validity testing of the application. The target population included Korean adults aged between 30 and 65 years who had visited a medical institution more than once within the most recent 6 months. The intervention group received patient participation training by using the mobile application, Application for Patient Participation in Safety Enhancement, for 2 months. The primary outcome variables were patient safety knowledge, self‐efficacy of participation, willingness to participate and experience of patient participation in patient safety activities. End‐user satisfaction was assessed using a questionnaire. To assess participants'' experiences with the intervention, qualitative data were collected through a focus group interview and open‐ended responses to an end‐user satisfaction survey.ResultsThe intervention group (n = 60) had significantly higher overall average scores than the control group (n = 37) with regard to patient safety knowledge (p < .001), self‐efficacy of participation (p = .001), willingness to participate (p = .010) and experience of participation (p = .038) in the post‐survey. The total mean end‐user satisfaction score was 3.56 ± 0.60. The participants expressed the realization that patients could play an important role in improving patient safety.ConclusionsThis study demonstrated that educating health consumers through a mobile application with useful information improves patient participation in patient safety activities. Educational materials and patient participation tools could motivate health consumers'' health‐related behaviours.Patient or Public ContributionPatients were involved during the programme development and evaluation.     

Defining Patient Engagement in Research: Results of a Systematic Review and Analysis: Report of the ISPOR Patient-Centered Special Interest Group

ObjectivesLack of clarity on the definition of “patient engagement” has been highlighted as a barrier to fully implementing patient engagement in research. This study identified themes within existing definitions related to patient engagement and proposes a consensus definition of “patient engagement in research.”MethodsA systematic review was conducted to identify definitions of patient engagement and related terms in published literature (2006-2018). Definitions were extracted and qualitatively analyzed to identify themes and characteristics. A multistakeholder approach, including academia, industry, and patient representation, was taken at all stages. A proposed definition is offered based on a synthesis of the findings.ResultsOf 1821 abstracts identified and screened for eligibility, 317 were selected for full-text review. Of these, 169 articles met inclusion criteria, from which 244 distinct definitions were extracted for analysis. The most frequently defined terms were: “patient-centered” (30.5%), “patient engagement” (15.5%), and “patient participation” (13.4%). The majority of definitions were specific to the healthcare delivery setting (70.5%); 11.9% were specific to research. Among the definitions of “patient engagement,” the most common themes were “active process,” “patient involvement,” and “patient as participant.” In the research setting, the top themes were “patient as partner,” “patient involvement,” and “active process”; these did not appear in the top 3 themes of nonresearch definitions.ConclusionDistinct themes are associated with the term “patient engagement” and with engagement in the “research” setting. Based on an analysis of existing literature and review by patient, industry, and academic stakeholders, we propose a scalable consensus definition of “patient engagement in research.”     

An empirically based conceptual framework for fostering meaningful patient engagement in research

Background

Patient engagement in research (PEIR) is promoted to improve the relevance and quality of health research, but has little conceptualization derived from empirical data.

Objective

To address this issue, we sought to develop an empirically based conceptual framework for meaningful PEIR founded on a patient perspective.

Methods

We conducted a qualitative secondary analysis of in‐depth interviews with 18 patient research partners from a research centre‐affiliated patient advisory board. Data analysis involved three phases: identifying the themes, developing a framework and confirming the framework. We coded and organized the data, and abstracted, illustrated, described and explored the emergent themes using thematic analysis. Directed content analysis was conducted to derive concepts from 18 publications related to PEIR to supplement, confirm or refute, and extend the emergent conceptual framework. The framework was reviewed by four patient research partners on our research team.

Results

Participants’ experiences of working with researchers were generally positive. Eight themes emerged: procedural requirements, convenience, contributions, support, team interaction, research environment, feel valued and benefits. These themes were interconnected and formed a conceptual framework to explain the phenomenon of meaningful PEIR from a patient perspective. This framework, the PEIR Framework, was endorsed by the patient research partners on our team.

Conclusions

The PEIR Framework provides guidance on aspects of PEIR to address for meaningful PEIR. It could be particularly useful when patient‐researcher partnerships are led by researchers with little experience of engaging patients in research.     

Revealing and resolving patient safety defects: the impact of leadership WalkRounds on frontline caregiver assessments of patient safety

Objective. To evaluate the impact of rigorous WalkRounds on frontline caregiver assessments of safety climate, and to clarify the steps and implementation of rigorous WalkRounds. Data Sources/Study Setting. Primary outcome variables were baseline and post WalkRounds safety climate scores from the Safety Attitudes Questionnaire (SAQ). Secondary outcomes were safety issues elicited through WalkRounds. Study period was August 2002 to April 2005; seven hospitals in Massachusetts agreed to participate; and the project was implemented in all patient care areas. Study Design. Prospective study of the impact of rigorously applied WalkRounds on frontline caregivers assessments of safety climate in their patient care area. WalkRounds were conducted weekly and according to the seven‐step WalkRounds Guide. The SAQ was administered at baseline and approximately 18 months post‐WalkRounds implementation to all caregivers in patient care areas. Results. Two of seven hospitals complied with the rigorous WalkRounds approach; hospital A was an academic teaching center and hospital B a community teaching hospital. Of 21 patient care areas, SAQ surveys were received from 62 percent of respondents at baseline and 60 percent post WalkRounds. At baseline, 10 of 21 care areas (48 percent) had safety climate scores below 60 percent, whereas post‐WalkRounds three care areas (14 percent) had safety climate scores below 60 percent without improving by 10 points or more. Safety climate scale scores in hospital A were 62 percent at baseline and 77 percent post‐WalkRounds (t=2.67, p=.03), and in hospital B were 46 percent at baseline and 56 percent post WalkRounds (t=2.06, p=.06). Main safety issues by category were equipment/facility (A [26 percent] and B [33 percent]) and communication (A [24 percent] and B [18 percent]). Conclusions. WalkRounds implementation requires significant organizational will; sustainability requires outstanding project management and leadership engagement. In the patient care areas that rigorously implemented WalkRounds, frontline caregiver assessments of patient safety increased. SAQ results such as safety climate scores facilitate the triage of quality improvement efforts, and provide consensus assessments of frontline caregivers that identify themes for improvement.