Updating the Evidence on Functional Capacity Evaluation Methods: A Systematic Review

Objectives To synthesize the evidence on the psychometrics functional capacity evaluation (FCE) methods. Methods A systematic literature search in nine databases. The resulting articles were screened based on predefined in- and exclusion criteria. Two reviewers independently performed this screening. Included studies were appraised based on their methodological quality. Results The search resulted in 20 eligible studies about nine different FCE methods. The Baltimore Therapeutic Equipment work simulator showed a moderate predictive validity. The Ergo-Kit (EK) showed moderate variability and high inter- and intra-rater reliability. Low discriminative abilities and high convergent validity were found for the EK. Concurrent validity of the EK and the ERGOS Work Simulator was low to moderate. Moderate to high test–retest, inter- and intra-reliability was found in the Isernhagen Work-Systems (IWS) FCE. The predictive validity of the IWS was low. The physical work performance evaluation (PWPE) showed moderate test–retest reliability and moderate to high inter-rater reliability. Low internal and external responsiveness were found for the PWPE, predictive validity was high. The predictive validity of the short-form FCE was also high but need to be further examined on several psychometric properties. Low discriminative and convergent validity were found for the work disability functional assessment battery. The WorkHab showed moderate to high test–retest, inter- and intra-rater reliability. Conclusion Well-known FCE methods have been rigorously studied, but some of the research indicates weaknesses in their reliability and validity. Future research should address how these weaknesses can be overcome.     

Smoking, nutrition, alcohol and physical activity interventions targeting Indigenous Australians: rigorous evaluations and new directions needed

Objective: To describe and critique methodological aspects of interventions targeting reductions in smoking, poor nutrition, alcohol misuse and physical inactivity (SNAP risk factors) among Indigenous Australians. Methods: An electronic search of eight databases and a manual search of reference lists of literature reviews and reference libraries for Indigenous‐specific intervention studies published in peer‐reviewed journals (January 1990 to August 2007) were undertaken. Alcohol, smoking, nutrition or physical activity needed to be the primary focus of the study and the intervention needed to specifically target Indigenous Australians. Results: Twenty studies were selected for inclusion in the review. Methodologically, few studies employed randomisation or a control group, most omitted important details (e.g. costs), some did not report process measures (e.g. attrition rates), and some did not use validated measures. Two‐thirds of interventions were implemented at the community level and employed multiple strategies. Conclusion: There is a need for more rigorous evaluations of interventions targeting reductions in SNAP risk factors among Indigenous Australians, and to establish the reliability and validity of measures to quantify their effect. Implications: It may be beneficial for future Indigenous‐specific intervention research to focus on the evaluation of secondary prevention to complement the current concentration of effort targeting primary prevention. Community‐wide interventions, combining strategies of greater intensity for high risk individuals with those of less intensity targeting lower risk individuals, might also offer considerable promise.     

Test–Retest Reliability of the Isernhagen Work Systems Functional Capacity Evaluation in Patients with Chronic Low Back Pain

The aim of this study was to investigate test–retest reliability of the Isernhagen Work System Functional Capacity Evaluation (IWS FCE) in a sample of patients (n = 30) suffering from Chronic Low Back Pain (CLBP) and selected for rehabilitation treatment. The IWS FCE consists of 28 tests that reflect work-related activities like lifting, carrying, bending, etc. In this study, a slightly modified IWS FCE was used. Patients were included in the study if they were still at work or were less than 1 year out of work because of CLBP. Participants' mean age was 40 years, the duration of low back pain ranged between 5 and 10 years. Fifteen patients (50%) were out of work for a mean of 17 weeks, and they all received financial compensation. Two FCE sessions were held with a 2-week interval in between. Means per session, 95% confidence intervals of the mean difference, one-way random Intra Class Correlations (ICC), limits of agreement, Cohen's kappa and percentage of absolute agreement were calculated where appropriate. An ICC of 0.75 or more, a kappa value of more than 0.60 and a percentage of absolute agreement of 80% were considered as an acceptable reliability. Tests of the IWC FCE were divided into tests with and tests without an acceptable test–retest reliability on the basis of the kappa values, the percentage of absolute agreement and the ICC values. Fifteen tests (79%) showed an acceptable test–retest reliability based on Kappa values and percentage of absolute agreement. Eleven tests (61%) showed an acceptable test–retest reliability based on ICC values.     

Development and Validation of a Short-Form Functional Capacity Evaluation for Use in Claimants with Low Back Disorders

Objectives: Functional Capacity Evaluations (FCE) are used for making return-to-work decisions, yet FCE's modest predictive ability is currently outweighed by the administrative burden of testing. We attempted to develop a short-form FCE while maintaining comparable predictive ability. Methods: Three databases previously created for evaluating FCE predictive validity were used. Subjects were compensation claimants with low back disorders. FCE measures included items in the Isernhagen Work Systems’ FCE. Days until benefit suspension served as an indicator of return-to-work. Analysis included Cox regression. Results: Three items, floor-to-waist lift, crouching, and standing, were maintained in the short-form FCE. The short-form FCE was found to predict comparably to the entire FCE protocol in two validation cohorts (R ² difference<3%). Subjects meeting job demands on all three items consistently experienced faster benefit suspension. Conclusion: A short-form FCE for determining future work status in claimants with low back disorders was developed. A substantially abbreviated FCE may offer an efficient alternative.     

Upper Lifting Performance of Healthy Young Adults in Functional Capacity Evaluations: A Comparison of Two Protocols

The objectives of this study were to explore the concurrent validity of test results of upper lifting tasks of the Ergo-Kit FCE and the Isernhagen Work Systems (IWS) FCE. Seventy-one healthy young adults performed 5 upper lifting tests with at least 5 min of rest in between. The lifting tests included 3 standard protocols and 2 modified protocols. Three criteria for concurrent validity were established: 1) Pearson correlation higher than .75, 2) nonsignificant two-tailed t test, and 3) mean difference smaller than 5 kg. The results showed that none of the criteria were met for the standard protocols. For the modified protocols criteria 2 and 3 were not met. Individual differences larger than 10 kg were found for both standard and modified protocols. It was concluded that the standard protocols for upper lifting tasks of the Ergo-Kit FCE and the IWS FCE do not meet the criteria for concurrent validity and can, therefore, not be used interchangeably.     

A Review of Methods to Assess Parental Feeding Practices and Preschool Children's Eating Behavior: The Need for Further Development of Tools

We reviewed tools developed to measure parental feeding practices and eating behavior and food intake or preferences of children aged 0 to 5 years. Two electronic literature databases (Medline and Psycinfo) were used to search for both observational and experimental studies in human beings. The articles selected for review were those presenting tools with data on internal consistency and/or test–retest reliability and/or construct validity. A total of 3,445 articles were retrieved, and further searching of reference lists and contact with experts produced an additional 18 articles. We identified three tools on the qualitative dimension of children's eating behavior, two tools on food intake or preferences, and one tool on parental feeding practices with rigorous testing of internal consistency, construct validity, and test–retest reliability. All other tools presented in this review need further evaluation of their validity or reliability. Because major gaps exist, we highlight the need for more tools on parental attention to children's hunger and satiety cues, and the need to evaluate the degree of control allowed to children younger than age 2 years in feeding events. Food avoidance (ie, behaviors or strategies to take away and to reject food) and food approach (ie, attractiveness for food stimuli) have not been assessed in children aged 12 to 24 months. Food preference tests based on sensory aspects rather than nutritional quality may be worth investigating. We identified a need for further evaluation of quality, especially test–retest reliability and construct validity, for most tools developed for use in studying children aged 0 to 5 years.     

A critical review of children's culinary nutrition interventions,the methodologies used and their impact on dietary,psychosocial and wellbeing outcomes

Diet quality has been associated with numerous health outcomes, resulting in nutrition education to improve children's diet quality. Culinary nutrition interventions have been emphasised as a promising approach for enhancing children's food preferences and behaviours. Recently, there has been an increase in such interventions, and it is essential to understand their effectiveness and the specific methods used. Therefore, this review aimed to critically investigate methodological approaches in a range of children's culinary nutrition interventions and experiments. A secondary aim was to investigate the impact of these interventions on dietary, psychosocial and wellbeing outcomes. A systematic and pragmatic search strategy was developed and implemented using two electronic databases. Data extraction of the relevant content of eligible studies and a narrative synthesis were conducted. A total of 12 312 articles were identified from the search and 38 studies on children's culinary nutrition interventions or experiments were included. Most studies (n = 25) were conducted in North America. Only two studies had an RCT design. Less than half the studies (n = 16) used an underpinning theory, model or framework. Only four studies conducted sample size calculations. Some validated measurement tools were used. Despite the methodological concerns, most studies found some positive changes in dietary and/or psychosocial outcomes, while only two studies assessed wellbeing. Therefore, the area warrants further in-depth research anchored in methodological rigor to strengthen the validity of the research. The strengthening of the evidence in children's culinary nutrition could have a significant beneficial impact on public health if it resulted in widespread interventions and, in the long-term, reduce the impact on health systems.     

Psychosocial Measures Used to Assess the Effectiveness of School-based Nutrition Education Programs: Review and Analysis of Self-report Instruments for Children 8 to 12 Years Old

ObjectiveTo identify the psychometric properties of evaluation instruments that measure mediators of dietary behaviors in school-aged children.DesignSystematic search of scientific databases limited to 1999–2010.Main Outcome MeasuresPsychometric properties related to development and testing of self-report instruments for children 8–12 years old.AnalysisSystematic search of 189 articles and review of 15 instruments (20 associated articles) meeting the inclusion criteria. Search terms used included children, school, nutrition, diet, nutrition education, and evaluation.ResultsFourteen studies used a theoretical framework to guide the instrument's development. Knowledge and self-efficacy were the most commonly used psychosocial measures. Twelve instruments focused on specific nutrition-related behaviors. Eight instruments included over 40 items and used age-appropriate response formats. Acceptable reliability properties were most commonly reported for attitude and self-efficacy measures. Although most of the instruments were reviewed by experts (n = 8) and/or pilot-tested (n = 9), only 7 were tested using both rigorous types of validity and with low-income youth.Conclusions and ImplicationsResults from this review suggest that additional research is needed to develop more robust psychosocial measures for dietary behaviors, for low-income youth audiences.     

Measurement properties of questionnaires assessing participation in children and adolescents with a disability: a systematic review

Purpose

To critically appraise the measurement properties of questionnaires measuring participation in children and adolescents (0–18 years) with a disability.

Methods

Bibliographic databases were searched for studies evaluating the measurement properties of self-report or parent-report questionnaires measuring participation in children and adolescents (0–18 years) with a disability. The methodological quality of the included studies and the results of the measurement properties were evaluated using a checklist developed on consensus-based standards.

Results

The search strategy identified 3,977 unique publications, of which 22 were selected; these articles evaluated the development and measurement properties of eight different questionnaires. The Child and Adolescent Scale of Participation was evaluated most extensively, generally showing moderate positive results on content validity, internal consistency, reliability and construct validity. The remaining questionnaires also demonstrated positive results. However, at least 50 % of the measurement properties per questionnaire were not (or only poorly) assessed.

Conclusions

Studies of high methodological quality, using modern statistical methods, are needed to accurately assess the measurement properties of currently available questionnaires. Moreover, consensus is required on the definition of the construct ‘participation’ to determine content validity and to enable meaningful interpretation of outcomes.     

The utility of Functional Capacity Evaluation: the opinion of physicians and other experts in the field of return to work and disability claims

Objectives: This qualitative study explored how Dutch experts perceive the utility of functional capacity evaluation (FCE) for return to work (RTW) and disability claim (DC) assessment purposes. Methods: Twenty-one RTW case managers and 29 DC experts were interviewed by telephone using a semi-structured interview schedule. Results: The RTW case managers valued the utility of FCE on a scale of 0–10. Their mean valuation was 6.5 (SD 1.5). The average valuation for DC experts was 4.8 (SD 2.2). Arguments in favor of FCE were (1) its ability to confirm own opinions and (2) the objectivity of its measurement method. Arguments against FCE were (1) the redundancy of the information it provides and (2) the lack of objectivity. Indications for FCE were musculoskeletal disorders, a positive patient self-perception of ability to work, and the presence of an actual job. Contraindications for FCE were medically unexplained disorders, a negative patient self-perception of ability to work, and the existence of disputes and legal procedures. Conclusions: The responding RTW case managers perceived FCE to be more useful than the responding DC experts. The question of whether the arguments presented for and against the utility of FCE are valid is one that should be addressed in a future study.     

Systematic review of early intervention programmes for children from birth to nine years who have a physical disability

Aim: To systematically review the literature on the effectiveness of early intervention programmes for children with physical disabilities. Methods: Twelve electronic databases were searched for articles published between 1990 and April 2008. The quality of articles was appraised using an adapted version of the Checklist for the Evaluation of Research Articles and the Physiotherapy Evidence Database (PEDro) scale. Results: Ten studies were included in the review. Cross‐sectional and interrupted time‐series studies were of moderate methodological quality, whereas the non‐randomised control trial was of moderate‐to‐high methodological quality. Studies differed considerably with respect to participants, types of intervention and outcomes measured. Conclusions: Positive outcomes for both children and families have resulted from early intervention. However, methodological limitations hamper a more rigorous analysis of findings across studies.     

A systematic review of tools used to assess the quality of observational studies that examine incidence or prevalence and risk factors for diseases

ObjectiveTo create a comprehensive evaluation of checklists and scales used to evaluate observational studies that examine incidence or prevalence and risk factors for diseases.Study DesignWe did a literature search of several databases to abstract format, content, development, and validation of the tools.ResultsWe identified 46 scales and 51 checklists. Forty-seven of these tools were created for therapeutic studies, 48 for risk factors, and 5 for incidence studies. Forty-seven percent were modifications of previously published peer-reviewed appraisals, 18% were developed based on methodological standards, and 35% did not report development. Twenty-two percent reported reliability and 10% the validation procedure. Tools did not discriminate poor reporting vs. methodological quality of studies or external vs. internal validity; 35% categorize quality by the presence of predefined major flaws in design or by total score from the scale. Level of evidence was proposed in 22% of the tools by criteria of causality or internal validity of the studies. Evaluation required different degrees of subjectivity.ConclusionsFormat, length, and content varied substantially across available checklists and scales. Development, validation, and reliability were not consistently reported. Transparent objective quality assessments should be developed in the future.     

药物流行病学中处方序列对称分析研究的系统综述

目的 系统评价和分析国内外处方序列对称分析（PSSA）的原始研究,总结研究进展和方法学细节,为我国今后开展同类研究提供参考。方法 以PSSA相关的中英文关键词,分别检索中国知网、万方数据知识服务平台、维普数据库及PubMed、Embase、Cochrane,筛选截至2020年6月30日发表的PSSA原始研究。采用Excel软件构建信息提取表,采用Stata软件进行统计学分析。结果 共纳入45篇文献,从2013年开始文献发表数量显著上升;研究地区主要分布在日本（11篇,24.44%）、中国（10篇,22.22%）、丹麦（9篇,20.00%）、澳大利亚（8篇,17.78%）,大部分研究使用的数据库为医疗保险理赔数据库。纳入的PSSA研究共涉及16类药物,其中精神类药物和他汀类药物的研究数量最多（8篇,17.78%）,人体各系统的不良反应均有涉及。方法学上,明确给出了洗脱期和间隔期的研究数量分别为35篇（77.78%）和43篇（95.56%）,其中相应有14篇（31.11%）和9篇（20.00%）给出了确定的方法或理由。另外分别有16篇（35.56%）和18篇（40.00%）文献报告了敏感性分析和亚组分析。结论 PSSA作为有效的安全性信号挖掘方法之一发展迅速,但方法学处理和结果报告还需提高。我国PSSA研究仍处于起步阶段,需注重研究质量,推进方法学探索。     

A Systematic Review of Health Care Efficiency Measures

Objective. To review and characterize existing health care efficiency measures in order to facilitate a common understanding about the adequacy of these methods.
Data Sources. Review of the MedLine and EconLit databases for articles published from 1990 to 2008, as well as search of the "gray" literature for additional measures developed by private organizations.
Study Design. We performed a systematic review for existing efficiency measures. We classified the efficiency measures by perspective, outputs, inputs, methods used, and reporting of scientific soundness.
Principal Findings. We identified 265 measures in the peer-reviewed literature and eight measures in the gray literature, with little overlap between the two sets of measures. Almost all of the measures did not explicitly consider the quality of care. Thus, if quality varies substantially across groups, which is likely in some cases, the measures reflect only the costs of care, not efficiency. Evidence on the measures' scientific soundness was mostly lacking: evidence on reliability or validity was reported for six measures (2.3 percent) and sensitivity analyses were reported for 67 measures (25.3 percent).
Conclusions. Efficiency measures have been subjected to few rigorous evaluations of reliability and validity, and methods of accounting for quality of care in efficiency measurement are not well developed at this time. Use of these measures without greater understanding of these issues is likely to engender resistance from providers and could lead to unintended consequences.     

A Systematic Search and Review of Questionnaires Measuring Individual psychosocial Factors Predicting Return to Work After Musculoskeletal and Common Mental Disorders

Purpose Individual psychosocial factors are crucial in the return to work (RTW) process of workers with musculoskeletal disorders (MSDs) and common mental disorders (CMDs). However, the quality and validity of the questionnaires used to measure these factors have rarely been investigated. The present systematic search and literature review aims at identifying, categorizing, and evaluating the questionnaires (measurement tools) used to measure individual psychosocial factors related to the perception of the personal condition and motivation to RTW that are predictive of successful RTW among workers with MSDs or CMDs. Methods Through a systematic search on PubMed, Web of Science, and PsycINFO library databases and grey literature, we identified the individual psychosocial factors predictive of successful RTW among these workers. Then, we retrieved the questionnaires used to measure these factors. Finally, we searched for articles validating these questionnaires to describe them exhaustively from a psychometric and practical point of view. Results: The review included 76 studies from an initial pool of 2263 articles. Three common significant predictors of RTW after MSDs and CMDs emerged (i.e., RTW expectations, RTW self-efficacy, and work ability), two significant predictors of RTW after MSDs only (i.e., work involvement and the self-perceived connection between health and job), and two significant predictors of RTW after CMDs only (i.e., optimism and pessimism). We analyzed 30 questionnaires, including eight multiple-item scales and 22 single-item measures. Based on their psychometric and practical properties, we evaluated one of the eight multiple-item scales as questionable and five as excellent. Conclusions: With some exceptions (i.e., self-efficacy), the tools used to measure individual psychosocial factors show moderate to considerable room for improvement.

     

Construct Validity of a Kinesiophysical Functional Capacity Evaluation Administered Within a Worker's Compensation Environment

The construct validity of a kinesiophysical Functional Capacity Evaluation (FCE) administered within a worker's compensation context was examined. A cross-sectional study design was employed. Clinical and demographic information on workers' compensation claimants was extracted from a rehabilitation facility's database. Measures of interest were the Isernhagen Work Systems' (Duluth, MN) FCE, the Pain Disability Index (PDI), and a pain visual analogue scale (VAS). A multitrait Pearson correlation matrix was created to observe the pattern of relationships between variables. The sample consisted of 321 subjects with work-related, medically stable low back pain of median duration of 307 days. FCE performance was moderately correlated with the PDI (r = –0.44–0.52) and with the pain VAS (r = 0.34–0.45). Pain intensity was correlated highly with the PDI (r = 0.79). The moderate relationship between FCE and the PDI supports the construct validity of FCE as a functional measure. However, kinesiophysical FCE performance was not unrelated to pain severity ratings as purported.     

The design, implementation, and evaluation of online credit nutrition courses: a systematic review

ObjectiveTo assess how postsecondary online nutrition education courses (ONEC) are delivered, determine ONEC effectiveness, identify theoretical models used, and identify future research needs.DesignSystematic search of database literature.SettingPostsecondary education.ParticipantsNine research articles evaluating postsecondary ONEC.Main Outcome MeasuresKnowledge/performance outcomes and student satisfaction, motivation, or perceptions.AnalysisSystematic search of 922 articles and review of 9 articles meeting search criteria.ResultsLittle research regarding ONEC marketing/management existed. Studies primarily evaluated introductory courses using email/websites (before 2000), or course management systems (after 2002). None used true experimental designs; just 3 addressed validity or reliability of measures or pilot-tested instruments. Three articles used theoretical models in course design; few used theories to guide evaluations. Four quasi-experimental studies indicated no differences in nutrition knowledge/performance between online and face-to-face learners. Results were inconclusive regarding student satisfaction, motivation, or perceptions.Conclusions and ImplicationsStudents can gain knowledge in online as well as in face-to-face nutrition courses, but satisfaction was mixed. More up-to-date investigations on effective practices are warranted, using theories to identify factors that enhance student outcomes, addressing emerging technologies, and documenting ONEC marketing, management, and delivery. Adequate training/support for faculty is needed to improve student experiences and faculty time management.     

Functional Capacity Evaluation Research: Report from the Second International Functional Capacity Evaluation Research Meeting

Introduction Functional capacity evaluations are an important component of many occupational rehabilitation programs and can play a role in facilitating reintegration to work thus improving health and disability outcomes. The field of functional capacity evaluation (FCE) research has continued to develop over recent years, with growing evidence on the reliability, validity and clinical utility of FCE within different patient and healthy worker groups. The second International FCE Research Conference was held in Toronto, Canada on October 2nd 2014 adjacent to the 2014 Work Disability Prevention Integration conference. This paper describes the outcomes of the conference. Report Fifty-four participants from nine countries attended the conference where eleven research projects and three workshops were presented. The conference provided an opportunity to discuss FCE practice, present new research and provide a forum for discourse around the issues pertinent to FCE use. Conference presentations covered aspects of FCE use including the ICF–FCE interface, aspects of reliability and validity, consideration of specific injury populations, comparisons of FCE components and a lively debate on the merits of ‘Man versus Machine’ in FCE’s. Future directions Researchers, clinicians, and other professionals in the FCE area have a common desire to improve the content and quality of FCE research and to collaborate to further develop research across systems, cultures and countries.