Evaluating Various Areas of Process Improvement in an Effort to Improve Clinical Research: Discussions from the 2012 Clinical Translational Science Award (CTSA) Clinical Research Management Workshop

Emphasis has been placed on assessing the efficiency of clinical and translational research as part of the National Institutes of Health (NIH) goal to “improve human health.” Improvements identified and implemented by individual organizations cannot address the research infrastructure needs of all clinical and translational research conducted. NIH''s National Center for Advancing Translational Sciences (NCATS) has brought together 61 Clinical and Translational Science Award (CTSA) sites creating a virtual national laboratory that reflects the diversity and breadth of academic medical centers to collectively improve clinical and translational science. The annual Clinical Research Management workshop is organized by the CTSA consortium with participation from CTSA awardees, NIH, and others with an interest in clinical research management. The primary objective of the workshop is to disseminate information that improves clinical research management although the specific objectives of each workshop evolve within the consortium. The fifth annual workshop entitled “Learning by doing; applying evidence‐based tools to re‐engineer clinical research management” took place in June 2012. The primary objective of the 2012 workshop was to utilize data to evaluate, modify, and improve clinical research management. This report provides a brief summary of the workshop proceedings and the major themes discussed among the participants.     

Alignment of Emergency Medicine Research Efforts with Clinical and Translational Science Awards

The Clinical and Translational Science Awards (CTSA) represent a major new funding pathway for health science investigators seeking National Institutes of Health (NIH) funds. This new pathway provides institutional-level support for clinical and translational research and is not tied to one organ system or disease process, fitting well with emergency medicine (EM) research needs. These awards open unique opportunities for advancing EM research. The CTSA mechanism provides institutional support from the NIH to promote both clinical and translational science. Of the 60 expected awards, 38 sites are currently funded. EM investigators can benefit the institutions applying for these awards and simultaneously gain from involvement. Some opportunities for participation provided by the CTSA include research training programs, joining multidisciplinary research teams, seed grant funding, and use of the CTSA-developed research infrastructure. Involvement of EM can benefit institutions by enhancing acute care research collaboration both within and among institutions. Emergency medicine researchers at institutions either planning to submit a CTSA application or with funded CTSA grants are encouraged to become actively involved in CTSA-related research programs.     

Using AI to Understand Key Success Features in Evolving CTSAs

A vital role for Clinical and Translational Science Award (CTSA) evaluators is to first identify and then articulate the necessary change processes that support the research infrastructures and achieve synergies needed to improve health through research. The use of qualitative evaluation strategies to compliment quantitative tracking measures (e.g., number of grants/publications) is an essential but under‐utilized approach in CTSA evaluations. The Clinical and Translational Science Institute of Southeast Wisconsin implemented a qualitative evaluation approach using appreciative inquiry (AI) that has revealed three critical features associated with CTSA infrastructure transformation success: developing open communication, creating opportunities for proactive collaboration, and ongoing attainment of milestones at the key function group level. These findings are consistent with Bolman & Deal''s four interacting hallmarks of successful organizations: structural (infrastructure), political (power distribution; organizational politics), human resource (facilitating change among humans necessary for continued success), and symbolic (visions and aspirations). Data gathered through this longitudinal AI approach illuminates how these change features progress over time as CTSA funded organizations successfully create the multiinstitutional infrastructures to connect laboratory discoveries with the diagnosis and treatment of human disease.     

Instruction in the Responsible Conduct of Research: An Inventory of Programs and Materials within CTSAs

The National Institutes of Health (NIH) require instruction in the responsible conduct of research (RCR) as a component of any Clinical and Translational Science Award (CTSA). The Educational Materials Group of the NIH CTSA Consortium''s Clinical Research Ethics Key Function Committee (CRE‐KFC) conducted a survey of the 38 institutions that held CTSA funding as of January 2009 to determine how they satisfy RCR training requirements. An 8‐item questionnaire was sent by email to directors of the Clinical Research Ethics, the Educational and Career Development, and the Regulatory Knowledge cores. We received 78 completed surveys from 38 CTSAs (100%). We found that there is no unified approach to RCR training across CTSAs, many programs lack a coherent plan for RCR instruction, and most CTSAs have not developed unique instructional materials tailored to the needs of clinical and translational scientists. We recommend collaboration among CTSAs and across CTSA key function committees to address these weaknesses. We also requested that institutions send electronic copies of original RCR training materials to share among CTSAs via the CTSpedia website. Twenty institutions submitted at least one educational product. The CTSpedia now contains more than 90 RCR resources.     

Community Engagement Strategies in the Original and Renewal Applications for CTSA Grant Funding

Purpose

The Clinical and Translational Science Award (CTSA) program has highlighted community engagement in research by requiring institutions to develop bidirectional relationships with communities to obtain funding. Little is known about how institutions have operationalized that requirement. This study aimed to describe the strategies proposed by the first institutions to receive CTSA funding and to undergo the CTSA renewal process.

Methods

The authors conducted a qualitative document analysis of the community engagement section of 12 original and 10 renewal grant applications of the 12 institutions awarded CTSA funding in 2006 and renewed in 2010.

Results

Institutions employed ‐ and research engagement strategies. Capacity‐building strategies included education, pilot grants, connecting potential partners, and community research centers. Research engagement strategies ranged from those that allowed for very little input from communities, such as announcements, to those that allowed for a high amount of input from communities, such as community‐researcher teams.

Conclusion

CTSA funding has supported capacity‐building for institutions and communities to partner. Engagement strategies employed by the institutions are largely known from prior community‐engaged research. Based on the grants from these institutions, the CTSA funding has largely enabled institutions to develop capacity to engage.     

The Quantitative Evaluation of the Clinical and Translational Science Awards (CTSA) Program Based on Science Mapping and Scientometric Analysis

The Clinical and Translational Science Awards (CTSA) program is one of the most important initiatives in translational medical funding. The quantitative evaluation of the efficiency and performance of the CTSA program has a significant referential meaning for the decision making of global translational medical funding. Using science mapping and scientometric analytic tools, this study quantitatively analyzed the scientific articles funded by the CTSA program. The results of the study showed that the quantitative productivities of the CTSA program had a stable increase since 2008. In addition, the emerging trends of the research funded by the CTSA program covered clinical and basic medical research fields. The academic benefits from the CTSA program were assisting its members to build a robust academic home for the Clinical and Translational Science and to attract other financial support. This study provided a quantitative evaluation of the CTSA program based on science mapping and scientometric analysis. Further research is required to compare and optimize other quantitative methods and to integrate various research results.     

Models of Interinstitutional Partnerships between Research Intensive Universities and Minority Serving Institutions (MSI) across the Clinical Translational Science Award (CTSA) Consortium

Health disparities are an immense challenge to American society. Clinical and Translational Science Awards (CTSAs) housed within the National Center for Advancing Translational Science (NCATS) are designed to accelerate the translation of experimental findings into clinically meaningful practices and bring new therapies to the doorsteps of all patients. Research Centers at Minority Institutions (RCMI) program at the National Institute on Minority Health and Health Disparities (NIMHD) are designed to build capacity for biomedical research and training at minority serving institutions. The CTSA created a mechanism fostering formal collaborations between research intensive universities and minority serving institutions (MSI) supported by the RCMI program. These consortium‐level collaborations activate unique translational research approaches to reduce health disparities with credence to each academic institutions history and unique characteristics. Five formal partnerships between research intensive universities and MSI have formed as a result of the CTSA and RCMI programs. These partnerships present a multifocal approach; shifting cultural change and consciousness toward addressing health disparities, and training the next generation of minority scientists. This collaborative model is based on the respective strengths and contributions of the partnering institutions, allowing bidirectional interchange and leveraging NIH and institutional investments providing measurable benchmarks toward the elimination of health disparities.     

Social Network Analysis to Assess the Impact of the CTSA on Biomedical Research Grant Collaboration

Success of the Clinical Translational Science Award (CTSA) program implicitly demands team science efforts and well‐orchestrated collaboration across the translational silos (T1–T4). Networks have proven to be useful abstractions of research collaborations. Networks provide novel system‐level insights and exhibit marked changes in response to external interventions, making them potential evaluation tools that complement more traditional approaches. This study is part of our ongoing efforts to assess the impact of the CTSA on Biomedical Research Grant Collaboration (BRGC). Collaborative research grants are a complex undertaking and an outcome of sustained interaction among researchers. In this report, BRGC networks representing collaborations among CTSA‐affiliated investigators constructed from grants management system data at the University of Kentucky across a period of six years (2007–2012) corresponding to pre‐ and post‐CTSA are investigated. Overlapping community structure detection algorithms, in conjunction with surrogate testing, revealed the presence of intricate research communities rejecting random graphs as generative mechanisms. The deviation from randomness was especially pronounced post‐CTSA, reflecting an increasing trend in collaborations and team‐science efforts potentially as a result of CTSA. Intercommunity cross talk was especially pronounced post‐CTSA.     

Community Engagement in the CTSA Program: Stakeholder Responses from a National Delphi Process

In response to the Institute of Medicine (IOM) Committee''s December 2012 public request for stakeholder input on the Clinical and Translational Science Award (CTSA) program, two nonprofit organizations, the Center for Community Health Education Research and Service, Inc. (CCHERS) and Community‐Campus Partnerships for Health (CCPH), solicited feedback from CTSA stakeholders using the Delphi method. Academic and community stakeholders were invited to participate in the Delphi, which is an exploratory method used for group consensus building. Six questions posed by the IOM Committee to an invited panel on community engagement were electronically sent to stakeholders. In Round 1 stakeholder responses were coded thematically and then tallied. Round 2 asked stakeholders to state their level of agreement with each of the themes using a Likert scale. Finally, in Round 3 the group was asked to rank the Round 2 based on potential impact for the CTSA program and implementation feasibility. The benefits of community engagement in clinical and translational research as well as the need to integrate community engagement across all components of the CTSA program were common themes. Respondents expressed skepticism as to the feasibility of strengthening CTSA community engagement.     

CTSA Consortium Consensus Scientific Review Committee (SRC) Working Group Report on the SRC Processes

Human research projects must have a scientifically valid study design, analytic plan, and be operationally feasible in order to be successfully completed and thus to have translational impact. To ensure this, institutions that conduct clinical research should have a scientific review process prior to submission to the Institutional Review Committee (IRB). This paper reports the Clinical and Translational Science Award (CTSA) Consortium Scientific Review Committee (SRC) Consensus Working Group''s proposed framework for a SRC process. Recommendations are provided for institutional support and roles of CTSAs, multisite research, criteria for selection of protocols that should be reviewed, roles of committee members, application process, and committee process. Additionally, to support the SCR process effectively, and to ensure efficiency, the Working Group recommends information technology infrastructures and evaluation metrics to determine outcomes are provided.     

Developing the Translational Research Workforce: A Pilot Study of Common Metrics for Evaluating the Clinical and Translational Award KL2 Program

Purpose

This pilot study describes the career development programs (i.e., NIH KL2 awards) across five Clinical and Translational Science Award (CTSA) institutions within the University of California (UC) system, and examines the feasibility of a set of common metrics for evaluating early outcomes.

Methods

A survey of program administrators provided data related to the institutional environment within which each KL2 program was implemented. Application and progress report data yielded a combined data set that characterized KL2 awardees, their initial productivity, and early career outcomes.

Results

The pilot project demonstrated the feasibility of aggregating common metrics data across multiple institutions. The data indicated that KL2 awardees were an accomplished set of investigators, both before and after the award period, representing a wide variety of disciplines. Awardees that had completed their trainee period overwhelmingly remained active in translational research conducted within an academic setting. Early indications also suggest high rates of success with obtaining research funding subsequent to the KL2 award.

Conclusion

This project offers a model for how to collect and analyze common metrics related to the education and training function of the CTSA Consortium. Next steps call for expanding participation to other CTSA sites outside of the University of California system.     

The CTSA as an Exemplar Framework for Developing Multidisciplinary Translational Teams

Translational science requires that scientists from multiple disciplines work together to improve the prevention, diagnosis, and treatment of human disease. Although a literature exists on the design and management of multidisciplinary teams, little has been written on multidisciplinary translational teams (MTTs). MTTs are distinct hybrid entities, with goals taken from both industry and academic models. We identified 30 design factors in 10 domains from a literature survey relevant to our MTT model: specific goals, structures, and processes. These dimensions were adapted to our own institutional environment in the selection and management of 11 MTTs that exploited resources of University of Texas Medical Branch (UTMB) Clinical and Translational Sciences Awards (CTSA). Case illustrations of two specific MTTs illustrate some of the challenges encountered and opportunities realized in terms of education and scientific advances. Network depiction of disciplinarity indicated that CTSA KRs and CTSA leadership contributed to discipline diversity especially in small (or nascent) MTTs. A separate depiction of MTT‐KR utilization indicated that data analysis, translational technologies, and novel methods were heavily utilized by MTTs, whereas other KRs contributed significant effort to infrastructure development. We conclude that the CTSA can provide a rich
infrastructural framework and scientific environment for the development of successful MTTs. Clin Trans Sci 2013; Volume 6: 60–71     

Evaluation Guidelines for the Clinical and Translational Science Awards (CTSAs)

The National Center for Advancing Translational Sciences (NCATS), a part of the National Institutes of Health, currently funds the Clinical and Translational Science Awards (CTSAs), a national consortium of 61 medical research institutions in 30 states and the District of Columbia. The program seeks to transform the way biomedical research is conducted, speed the translation of laboratory discoveries into treatments for patients, engage communities in clinical research efforts, and train a new generation of clinical and translational researchers. An endeavor as ambitious and complex as the CTSA program requires high‐quality evaluations in order to show that the program is well implemented, efficiently managed, and demonstrably effective. In this paper, the Evaluation Key Function Committee of the CTSA Consortium presents an overall framework for evaluating the CTSA program and offers policies to guide the evaluation work. The guidelines set forth are designed to serve as a tool for education within the CTSA community by illuminating key issues and practices that should be considered during evaluation planning, implementation, and utilization. Additionally, these guidelines can provide a basis for ongoing discussions about how the principles articulated in this paper can most effectively be translated into operational reality.     

Enhancing Dissemination,Implementation, and Improvement Science in CTSAs through Regional Partnerships

Background and Importance

Challenges in healthcare policy and practice have stimulated interest in dissemination and implementation science. The Institute of Medicine Committee on the Clinical Translational Science Award (CTSA) program recommended expanding the CTSA program''s investment and activity in this domain. Guidance is needed to facilitate successful growth of DII science infrastructure, activity and impacts.

Objectives

Several CTSAs in Southern California collaborated to identify and respond to local challenges and opportunities to expand dissemination, implementation and improvement research by strengthening capacity and relationships between DII researchers and community, health system, and population health partners.

Main outcomes

Planning and outreach by the Southern California CTSAs increased awareness and interest in DII research and generated recommendations for growth. Recommendations include: increasing strong partnerships with healthcare and population health systems to guide policy research agendas and collaborative DII science; promoting multi‐sector partnerships that involve researchers and delivery systems throughout DII processes; bringing together multiple disciplines; and addressing national and international barriers as well as opportunities in DII science.

Implications

CTSAs through regional collaboration can increase their contributions to improved community health via skill‐building, partnership development and enhanced outreach to local healthcare and public health agencies and delivery systems.     

The CTSA Mentored Career Development Program: Supporting the Careers of Child Health Investigators

Training translational scientists is a priority of the Clinical and Translational Science Award (CTSA) consortium. Objectives: 1) Describe the landscape of CTSA Mentored Research Career Development Awards (CDA) and 2) evaluate participation and outcomes of child health investigators in these programs. Design: Survey of the CTSA Child Health Oversight Committee (CC‐CHOC) and review of nonresponders'' CTSA Websites. Results: Thirty‐two of 53 CC‐CHOC members (60%) responded and all nonresponder Websites were reviewed. Institutions supported 1,166 CDA positions from 2006 to 2011, with 134 awarded to child health investigators (11.5%). Respondents reported a mean of 29.8 KL2 positions (95% CI 17.5–42.2) during their award period, with a mean of 2.8 (95% CI 1.8–3.8) awarded to child health investigators. The proportion of child health awardees varied from 0% to 50% across institutions. We identified 45 subsequent National Institutes of Health (NIH) awards to the 134 child health investigators (34%). Conclusions: The CTSA program contributes substantially to training the next generation of translational investigators. One‐third of child health investigators obtained subsequent NIH awards in the short follow‐up period demonstrating success of the CTSA CDA programs. Child health investigators are represented variably across the consortium. Pediatric institutions can partner with the CTSA program to further support training child health investigators.     

The CTSA Consortium's Catalog of Assets for Translational and Clinical Health Research (CATCHR)

The 61 CTSA Consortium sites are home to valuable programs and infrastructure supporting translational science and all are charged with ensuring that such investments translate quickly to improved clinical care. Catalog of Assets for Translational and Clinical Health Research (CATCHR) is the Consortium''s effort to collect and make available information on programs and resources to maximize efficiency and facilitate collaborations. By capturing information on a broad range of assets supporting the entire clinical and translational research spectrum, CATCHR aims to provide the necessary infrastructure and processes to establish and maintain an open‐access, searchable database of consortium resources to support multisite clinical and translational research studies. Data are collected using rigorous, defined methods, with the resulting information made visible through an integrated, searchable Web‐based tool. Additional easy‐to‐use Web tools assist resource owners in validating and updating resource information over time. In this paper, we discuss the design and scope of the project, data collection methods, current results, and future plans for development and sustainability. With increasing pressure on research programs to avoid redundancy, CATCHR aims to make available information on programs and core facilities to maximize efficient use of resources.     

Preliminary Competencies for Comparative Effectiveness Research

The Clinical and Translational Science Award (CTSA) Workgroup for Comparative Effectiveness Research (CER) Education, Training, and Workforce Development identified a need to delineate the competencies that practitioners and users of CER for patient‐centered outcomes research, should acquire. With input from CTSA representatives and collaborators, we began by describing the workforce. We recognize the workforce that conducts CER and the end users who use CER to improve the health of individuals and communities. We generated a preliminary set of competencies and solicited feedback from the CER representatives at each member site of the CTSA consortium. We distinguished applied competencies (i.e., skills needed by individuals who conduct CER) from foundational competencies that are needed by the entire CER workforce, including end users of CER. Key competency categories of relevance to both practitioners and users of CER were: (1) asking relevant research questions; (2) recognizing or designing ideal CER studies; (3) executing or using CER studies; (4) using appropriate statistical analyses for CER; and (5) communicating and disseminating CER study results to improve health. Although CER is particularly broad concept, we anticipate that these preliminary, relatively generic competencies will be used in tailoring curricula to individual learners from a variety of programmatic perspectives. Clin Trans Sci 2012; Volume 5: 476–479     

The Critical Need for Academic Health Centers to Assess the Training,Support, and Career Development Requirements of Clinical Research Coordinators: Recommendations from the Clinical and Translational Science Award Research Coordinator Taskforce

Clinical Research Coordinators (CRCs) are a vital component of the clinical research enterprise providing a pivotal role in human subject protection through the numerous activities and responsibilities assigned to them. In 2006, the National Institutes of Health’s National Center for Research resources (NCRR) implemented the Clinical and Translational Science Awards program (CTSA) to advance biomedical research. As a part of this endeavor, many workgroups were formed among the Consortium to support translational research. The Research Coordinator Taskforce was created as part of the Regulatory Knowledge group of the Clinical Research Innovation Key Function Committee, and focuses on enhancing CTSA capabilities to provide support and training for CRCs. In the spring of 2008, this taskforce conducted two surveys of the then 24 CTSA Consortium members to better understand the current expectations and responsibilities of research coordinators in addition to the mechanism for providing education, training, and support in order for CRCs to successfully meet the study responsibilities placed upon them. The results of these surveys are summarized in this article and provide context to the recommendations of the Research Coordinator Taskforce for institutional considerations, approaches, and best practices for providing education, training, and support the expanding role of CRCs in fulfilling their responsibilities delegated to them by investigators. Clin Trans Sci 2012; Volume 5: 470–475     

Faculty Perceptions of How Community‐Engaged Research Is Valued in Tenure,Promotion, and Retention Decisions

Purpose

We assessed the perceptions of community core faculty in academic medical center institutions that received Clinical and Translational Science Awards (CTSA) about how these institutions consider community‐engaged scholarship (CES) when tenure, promotion, and retention decisions are made.

Method

An assessment tool was adapted to create an 18‐item survey that was sent during November and December 2011 via the Internet to the 369 members of the community‐engagement core mailing list of the CTSA.

Results

Fifty‐nine responses were received which represented 37 of the possible 60 different funded institutions. The mean score was 48.14 (SD = 11.18); range of 23–74; and Cronbach''s alpha was .91 About half reported that support for CES and its inclusion in the academic decision process increased since the institution was awarded a CTSA. Open‐ended responses indicated some confusion with terminology although a definition of CES had been provided in the instrument instructions.

Conclusion

Respondents overall agreed there was moderate support for CES in tenure, promotion, and retention decisions which may have been influenced by the CTSA application requirements. This survey could be used to identify if there are differences in institutional and departments and measure changes over time.