Measuring the payback of research activities: a feasible ex-post evaluation methodology in epidemiology and public health

Most ex-post evaluations of research funding programs are based on bibliometric methods and, although this approach has been widely used, it only examines one facet of the project's impact, that is, scientific productivity. More comprehensive models of payback assessment of research activities are designed for large-scale projects with extensive funding. The purpose of this study was to design and implement a methodology for the ex-post evaluation of small-scale projects that would take into account both the fulfillment of projects' stated objectives as well as other wider benefits to society as payback measures. We used a two-phase ex-post approach to appraise impact for 173 small-scale projects funded in 2007 and 2008 by a Spanish network center for research in epidemiology and public health. In the internal phase we used a questionnaire to query the principal investigator (PI) on the outcomes as well as actual and potential impact of each project; in the external phase we sent a second questionnaire to external reviewers with the aim of assessing (by peer-review) the performance of each individual project. Overall, 43% of the projects were rated as having completed their objectives "totally", and 40% "considerably". The research activities funded were reported by PIs as socially beneficial their greatest impact being on research capacity (50% of payback to society) and on knowledge translation (above 11%). The method proposed showed a good discriminating ability that makes it possible to measure, reliably, the extent to which a project's objectives were met as well as the degree to which the project contributed to enhance the group's scientific performance and of its social payback.     

An efficient strategy allowed English-speaking reviewers to identify foreign-language articles eligible for a systematic review

ObjectiveTo assess English-speaking reviewers' accuracy in determining the eligibility of foreign-language articles for a systematic review.Study Design and SettingsSystematic review of randomized controlled trials of therapy for fibromyalgia. Guided by 10 questions, English-speaking reviewers screened non–English-language articles for eligibility. Teams of two native-language speakers provided reference standard judgments of eligibility.ResultsOf 15,466 potentially eligible articles, we retrieved 763 in full text, of which 133 were published in 19 non-English languages; 53 trials published in 11 languages other than English proved eligible. Of the 53 eligible articles, English-language reviewers guided by the 10 questions mistakenly judged 6 as ineligible; of the 80 ineligible articles, 8 were incorrectly judged eligible by English-language reviewers (sensitivity = 0.89; specificity = 0.90). Use of a simple three-step rule (excluding languages with less than three articles, reviewing titles and abstracts for clear indications of eligibility, and noting the lack of a clearly reported statistical analysis unless the word “random” appears) led to accurate classification of 51 of 53 articles (sensitivity = 0.96; specificity = 0.70).ConclusionOur findings show promise for limiting the need for non–English-language review teams in systematic reviews with large numbers of potentially eligible non–English-language articles.     

The central question and the scope of nursing research

As nursing continues to develop as a professional discipline, it is important for nurses to have a central question to guide their research. Since the 1800s, nursing practice and research have covered a wide scope in cooperation with other disciplines. This wide area of nursing practice and research has led to the proposal that the central question be: How can the well‐being of a person, family, community, or population be improved? The proposed question must remain flexible and open to revision because nurses will continue to adapt to the changing needs of their patients and populations and to their complex and evolving work environments.     

Community-based participatory research (CBPR) in South Africa: engaging multiple constituents to shape the research question

Community engagement is an on-going, arduous, and necessary process for developing effective health promotion programs. The challenges are amplified when the particular health issue or research question is not prominent in the consciousness of the targeted community. In this paper, we explore the community-based participatory research (CBPR) model as a means to negotiate a mutual agenda between communities and researchers. The paper is focused on the (perceived) need for cervical cancer screening in an under-resourced community in Cape Town, South Africa. Cervical cancer is a significant health problem in this community and elsewhere in South Africa. Unlike HIV-AIDS, however, many Black South Africans have not been educated about cervical cancer and the importance of obtaining screening. Many may not consider screening a priority in their lives. Our research included extensive consultations and informal interviews with diverse community and regional stakeholders. Following these, we conducted 27 focus groups and 106 demographic surveys with randomly selected youth, parents, local health care personnel, educators and school staff. Focus group data were summarized and analyzed cross-sectionally. Community stakeholders were involved throughout this research. Our consultations, interviews, and focus group data were key in identifying the concerns and priorities of the community. By engaging community stakeholders, we developed a research framework that incorporated the community's concerns and priorities, and stressed the intersecting roles of poverty, violence, and other cultural forces in shaping community members' health and wellbeing. Community members helped to refocus our research from cervical cancer to 'cervical health,' a concept that acknowledged the impact on women's bodies and lives of HIV-AIDS and STDs, sexual violence, poverty, and multiple social problems. We conclude that the research agenda and questions in community-based health research should not be considered immutable. They need to be open to negotiation, creativity, and constant reinvention.     

Using interactive workshops to prompt knowledge exchange: a realist evaluation of a knowledge to action initiative

Introduction

Interactive workshops are often the default mechanism for sharing knowledge across professional and sector boundaries; yet we understand little about if, and how, they work. Between 2009 and 2011, the Research to Reality programme in North East England ran eight stand-alone facilitated multi-agency workshops focused on priority public health issues. Local authorities, the health service, and academe collaborated on the programme to share latest evidence and best practice

Methods

A realist evaluation asked the overarching question ‘what worked where, for whom, and under what conditions’ regarding the knowledge exchange (KE) mechanisms underpinning any changes. Data were collected from fifty-one interviews, six observations, and analysis of programme documentation.

Results

191 delegates attended (local authority 46%, NHS 24%, academia 22%, third sector 6%, other 2%). The programme theory was that awareness raising and critical discussion would facilitate ownership and evidence uptake. KE activity included: research digests, academic and senior practitioner presentations, and facilitated round-table discussions. Joint action planning was used to prompt informed follow-up action. Participants valued the digests, expert input, opportunities for discussion, networking and ‘space to think’. However, within a few months, sustainability was lost. There was no evidence of direct changes to practice. Multiple barriers to research utilization emerged.

Discussion

The findings suggest that in pressured contexts exacerbated by structural reform providing evidence summaries, input from academic and practice experts, conversational spaces and personal action planning are necessary to create enthusiasm on the day, but are insufficient to prompt practice change in the medium term. The findings question makes assumptions about the instrumental, linear use of knowledge and of change focused on individuals as a driver for organizational change. Delegates' views of ‘what would work’ are shared. Mechanisms that would enhance interactive formats are discussed.     

On how to define the concept of health: A loose comparative approach

Medicine, Health Care and Philosophy -     

A third of systematic reviews changed or did not specify the primary outcome: a PROSPERO register study

ObjectivesTo examine outcome reporting bias of systematic reviews registered in PROSPERO.Study Design and SettingRetrospective cohort study. The primary outcomes from systematic review publications were compared with those reported in the corresponding PROSPERO records; discrepancies in the primary outcomes were assessed as upgrades, additions, omissions, or downgrades. Relative risks (RRs) and 95% confidence intervals (CI) were calculated to determine the likelihood of having a change in primary outcome when the meta-analysis result was favorable and statistically significant.ResultsNinety-six systematic reviews were published. A discrepancy in the primary outcome occurred in 32% of the included reviews and 39% of the reviews did not explicitly specify a primary outcome(s); 6% of the primary outcomes were omitted. There was no significant increased risk of adding/upgrading (RR, 2.14; 95% CI: 0.53, 8.63) or decreased risk of downgrading (RR, 0.76; 95% CI: 0.27, 2.17) an outcome when the meta-analysis result was favorable and statistically significant. As well, there was no significant increased risk of adding/upgrading (RR, 0.89; 95% CI: 0.31, 2.53) or decreased risk of downgrading (RR, 0.56; 95% CI: 0.29, 1.08) an outcome when the conclusion was positive.ConclusionsWe recommend review authors carefully consider primary outcome selection, and journals are encouraged to focus acceptance on registered systematic reviews.     

Evaluation of the results of a randomized controlled trial: how to define changes between baseline and follow-up

OBJECTIVE: The most common way to evaluate the effect of an intervention is to compare the intervention and nonintervention groups regarding the change in the outcome variable between baseline and follow-up; however, there are many different ways to define "changes." The purpose of this article is to demonstrate how different definitions of "change" used in the analysis can influence the results of a study. STUDY DESIGN AND SETTING: Two different randomized controlled trials were used as examples. RESULTS: The results of the analyses showed that for continuous outcome variables, analysis of covariance seems to be the most appropriate because it corrects for the phenomenon of regression to the mean. For dichotomous outcome variables, multinomial logistic regression analysis with all possible changes over time as outcome seems to be the most appropriate, especially because of its straightforward interpretation. CONCLUSION: A different definition of "change" can lead to different results in the evaluation of the effect of an intervention.     

Measuring the health and health behaviours of adolescents through cross-national survey research: recent developments in the Health Behaviour in School-aged Children (HBSC) study

Cross-national surveys have the potential to make a significant contribution to the study of adolescent health. The Health Behaviour in School-aged Children (HBSC) study was among the first international studies established in three countries in 1983 and growing to more than 40 countries for the seventh wave of fieldwork in 2005/06. The original aim of the study has remained largely the same since its inception, to increase understanding of adolescent health behaviours, health and well being in their social context and to collect high quality comparable data to achieve this. The challenges to producing valid and reliable data from cross-national, school-based research were recognised from the outset and reflected in the methodological development of the study. The paper sets out how these challenges were addressed, examining key aspects of the methodology, including study design, questionnaire content, data collection and file preparation. These methods are still in place, but HBSC has had to recognise the social and political change of recent years. The challenges that were recognised 20 years ago are magnified today, with the study embracing a variety of cultures across Europe and North America. As demand for HBSC data has grown from the scientific and policy communities, greater attention has been paid to scrutiny of the data produced, matched by a sharper focus on continuous improvement in data quality. Key developments of recent years are summarised in the paper, focusing on study organisation, review of the international Research Protocol, strengthening support for sampling, greater attention to translation and improvements in data processing and documentation. It is concluded that the HBSC study has evolved over the last 20 years and continues to do so, recognising the importance of data quality, but also the constraints of cross-national survey research. Looking to the future, some outstanding issues for consideration are touched upon, including the opportunities and challenges for expanding our knowledge on the possibilities for gathering cross-national and cross-cultural data, with the HBSC study being used to build capacity in understanding the health needs of young people in other regions of the world.     

Rebutting the suggestion that Anthony Giddens's Structuration Theory offers a useful framework for sociological nursing research: a critique based upon Margaret Archer's Realist Social Theory

Abstract  A recent paper in this journal by Hardcastle et al . in 2005 argued that Anthony Giddens's Structuration Theory (ST) might usefully inform sociological nursing research. In response, a critique of ST based upon the Realist Social Theory of Margaret Archer is presented. Archer maintains that ST is fatally flawed and, in consequence, it has little to offer nursing research. Following an analysis of the concepts epiphenomenalism and elisionism, it is suggested that emergentist Realist Social Theory captures or describes a more coherent explanatory vision of social reality than other perspectives and nurse researchers are advised to consider its potential.     

变革理论在卫生政策与体系研究领域的应用

变革理论是主流的项目理论之一,其描述了干预措施在特定背景下产生预期效果的因果逻辑,为干预设计、实施和评价以及经验推广提供理论基础,有利于提高政策干预的有效性和系统性。然而目前国内卫生政策与体系领域应用变革理论的研究相对较少,本文探讨了变革理论的基本概念,并对国内外相关研究进行了综述,促进变革理论在我国卫生政策与体系研究领域中的应用。     

OddSocks at the Melbourne Fringe Festival: a methods paper for using an arts installation in promoting public health

Installation art involves using everyday objects and spaces in an unusual way to gain attention and encourage interaction. While arts-based projects have the ability to inform and provoke interest, few have focused on public health. Oddsocks was conceived as a public health installation as part of the annual Melbourne Fringe Festival, with a primary aim to raise awareness concerning the importance of exercise and foot health. Promoting such underserved public health issues creates specific challenges which have been typically under-reported. This methods paper focuses on how these challenges can be met through arts-based initiatives and arts-based inquiry.     

'A day in the life': advancing a methodology for the cultural study of development and learning in early childhood

This paper explores the methodology of an ecological investigation of aspects of culture in the interactional construction of early childhood in diverse global communities: Peru, Italy, Canada, Thailand, and the United Kingdom. Regarding culture as a dynamic dimension of the child's socialisation, the approach taken was to film a 'day in the life' of a two-and-a-half-year-old girl in each location. The principal investigators viewed these five 'days' and selected clips were made into a compilation tape, to be interrogated and interpreted by the local investigators and the child's family. These latter reflections were also taped and then applied to a growing appreciation of the child in cultural context. Other inter-researcher techniques were used to elucidate and explore events and values further. Reflexive concerns as to the interplay between aims and methods in interpretive research are critical components of this endeavour to develop new cultural understandings of the girls in context.     

Effect on Trend Estimates of the Difference between Survey Respondents and Non-respondents: Results from 27 Populations in the WHO MONICA Project

Introduction: In the World Health Organization (WHO) MONICA (multinational MONItoring of trends and determinants in CArdiovascular disease) Project considerable effort was made to obtain basic data on non-respondents to community based surveys of cardiovascular risk factors. The first purpose of this paper is to examine differences in socio-economic and health profiles among respondents and non-respondents. The second purpose is to investigate the effect of non-response on estimates of trends. Methods:Socio-economic and health profile between respondents and non-respondents in the WHO MONICA Project final survey were compared. The potential effect of non-response on the trend estimates between the initial survey and final survey approximately ten years later was investigated using both MONICA data and hypothetical data. Results: In most of the populations, non-respondents were more likely to be single, less well educated, and had poorer lifestyles and health profiles than respondents. As an example of the consequences, temporal trends in prevalence of daily smokers are shown to be overestimated in most populations if they were based only on data from respondents. Conclusions: The socio-economic and health profiles of respondents and non-respondents differed fairly consistently across 27 populations. Hence, the estimators of population trends based on respondent data are likely to be biased. Declining response rates therefore pose a threat to the accuracy of estimates of risk factor trends in many countries.     

基于空间点模式分析的疾病分布状态的量化指标研究

目的 从空间点模式分析的角度探讨疾病分布状态的量化指标,为疾病的预防控制提供新的统计学方法.方法 基于病例间的距离系统地总结了空间点模式分析中反映疾病分布状态的定量统计指标--G函数、F函数、J函数和K函数.在介绍其基本原理的基础上,研究距离取0到3000 m,间隔为50 m,使用4个指标对贵池区的急性血吸虫病病例资料进行分析,并在SaTScan软件中用空间移动扫描圆形窗口法进行分析验证.结果 贵池区6年共发生83例急性血吸虫病病例,空间点模式分布图显示病例主要分布在长江和秋浦河附近.获得了4个定量统计指标的计算方法,用其对上述急性血吸虫病资料分析发现病例分布具有空间聚集性(G函数和K函数位于95%可信区间的上方,F函数和J函数位于95%可信区间的下方),与空间移动扫描圆形窗口法的结果一致,后者并发现1个最可能的聚集区域,圆心坐标为(30.65 N,117.44 E),圆半径为2.69 km,相对危险度为12.78(对数似然比=32.80,P=0.0001).结论 量化统计指标的应用不仅解决了传统标点地图不能量化分析的缺点,而且为深入的空间聚集性分析提供了理论依据.     

Implementation of the nursing process in a health area: models and assessment structures used

OBJECTIVE:

to analyze what nursing models and nursing assessment structures have been used in the implementation of the nursing process at the public and private centers in the health area Gipuzkoa (Basque Country).

METHOD:

a retrospective study was undertaken, based on the analysis of the nursing records used at the 158 centers studied.

RESULTS:

the Henderson model, Carpenito''s bifocal structure, Gordon''s assessment structure and the Resident Assessment Instrument Nursing Home 2.0 have been used as nursing models and assessment structures to implement the nursing process. At some centers, the selected model or assessment structure has varied over time.

CONCLUSION:

Henderson''s model has been the most used to implement the nursing process. Furthermore, the trend is observed to complement or replace Henderson''s model by nursing assessment structures.