Reliable Classification of Functional Profiles and Movement Disorders of Children with Cerebral Palsy

ABSTRACT

Objective: To examine the inter-rater reliability of the Communication Function Classification System (CFCS), Bimanual Fine Motor Function (BFMF), Surveillance of Cerebral Palsy in Europe (SCPE) classification tree, and Gross Motor Function Classification System (GMFCS) in children with cerebral palsy (CP) and periventricular white matter injury (PWMI) aged 4–11 years. Method: Twenty children were assessed by two raters using the four tools, in addition parents undertook ratings on the Manual Ability Classification System (MACS). Kappa statistics were used to calculate the level of agreement between raters’ classifications. Results: Participants comprised 12 males and 8 females with CP and PWMI, mean age 8 years 1 month (standard deviation 2 years 3 months). Inter-rater reliability across the four tools was 0.98 (CFCS, BFMF, and GMFCS) and 0.84 (SCPE). Implications: These findings suggest that these four tools are reasonably robust to inter-rater variability supporting their routine use along with the MACS in clinical and research applications.     

Evaluation of the Reliability of the Challenge when used to Measure Advanced Motor Skills of Children with Cerebral Palsy

Aims: The Challenge was designed as an extension to the GMFM-66 to assess advanced motor skills of children with cerebral palsy (CP) who walk/run independently. This study evaluated the Challenge's inter-rater and test–retest reliability. Methods: Thirty children with CP (GMFCS level I [n = 24] and II [n = 6]) completed the Challenge, with re-testing one to two weeks later. Seven physiotherapist assessors passed the Challenge criterion test pre-administration. A single assessor administered and scored test and retest sessions (test–retest reliability). A second assessor independently scored one of these sessions (inter-rater reliability). Results: Inter-rater reliability was excellent (ICC = 0.97, 95%CI 0.94–0.99, CoV < 10%), with no bias (Bland-Altman plot). Test–retest ICC was excellent (ICC = 0.94, 95% CI 0.88–0.97. CoV < 10%, and Minimum Detectable Change (MDC₉₀) was 4.47 points. Many participants indicated practising at home pre-retest session. Conclusions: There was strong rating consistency between assessors. While test–retest ICC estimates were also high, Challenge scores were higher at retest. The MDC₉₀ was still in a range (>4.5 points) that seems clinically viable for change detection. Test–retest reliability could be reassessed with children instructed not to practice between assessments to determine the extent to which between-session practice influenced scores.     

Reliability of Neurobehavioral Assessments from Birth to Term Equivalent Age in Preterm and Term Born Infants

Neurobehavioral assessments provide insight into the functional integrity of the developing brain and help guide early intervention for preterm (<37 weeks’ gestation) infants. In the context of shorter hospital stays, clinicians often need to assess preterm infants prior to term equivalent age. Few neurobehavioral assessments used in the preterm period have established interrater reliability. Aim: To evaluate the interrater reliability of the Hammersmith Neonatal Neurological Examination (HNNE) and the NICU Network Neurobehavioral Scale (NNNS), when used both preterm and at term (>36 weeks). Methods: Thirty-five preterm infants and 11 term controls were recruited. Five assessors double-scored the HNNE and NNNS administered either preterm or at term. A one-way random effects, absolute, single-measures interclass correlation coefficient (ICC) was calculated to determine interrater reliability. Results: Interrater reliability for the HNNE was excellent (ICC > 0.74) for optimality scores, and good (ICC 0.60–0.74) to excellent for subtotal scores, except for ‘Tone Patterns’ (ICC 0.54). On the NNNS, interrater reliability was predominantly excellent for all items. Interrater agreement was generally excellent at both time points. Conclusions: Overall, the HNNE and NNNS neurobehavioral assessments demonstrated mostly excellent interrater reliability when used prior to term and at term.     

Inter-Rater and Test–Retest Reliability of the German Pediatric Evaluation of Disability Inventory (PEDI-G)

ABSTRACT

Activities of daily living (ADL) of children are widely assessed with the Pediatric Evaluation Disability Inventory (PEDI). This study examined test–retest and inter-rater reliability of the German PEDI (PEDI-G). During the adaptation of the PEDI nine items were added. In total, 117 parents of 53 children without and 64 children with a diagnosed physical disability from Austria, Germany, and Switzerland participated. Reliability was examined by intraclass correlation coefficient (ICC), standard error of measurement (SEM) and smallest detectable difference (SDD) for the Functional Skill Scale with and without added items and the Caregiver Assistance Scale. Cohen`s Kappa was used to calculate the reliability of the Modification Scale. All ICC's for test–retest and inter-rater reliability were above 0.75, indicating good to very good reliability. The SDD varied from 0.83–5.58 across PEDI domains and scales. For the Modification Scale, Cohen's weighted kappa varied from 0.25 to 1.00 indicating sufficient reliability for some but not all items. Our findings indicate that the Functional Skill Scale and the Caregiver Assistance Scale of the PEDI-G are reliable scales that can be used to evaluate ADLs of children with and without physical disability.     

Development of an Age Band on the ManuVis for 3-Year-Old Children with Visual Impairments

Aim: To compare fine motor performance of 3-year-old children with visual impairment with peers having normal vision, to provide reference scores for 3-year-old children with visual impairment on the ManuVis, and to assess inter-rater reliability. Method: 26 children with visual impairment (mean age: 3 years 7 months (SD 3 months); 17 boys) and 28 children with normal vision (mean age: 3 years 7 months (SD 4 months); 14 boys) participated in the study. The ManuVis age band for 3-year-old children comprised two one-handed tasks, two two-handed tasks, and a pre-writing task. Results: Children with visual impairment needed more time on all tasks (p < .01) and performed the pre-writing task less accurately than children with normal vision (p < .001). Children aged 42–47 months performed significantly faster on two tasks and had better total scores than children aged 36–41 months (p < .05). Inter-rater reliability was excellent (Intra-class Correlation Coefficient = 0.96–0.99). Conclusions: The ManuVis age band for 3-year-old children is appropriate to assess fine motor skills, and is sensitive to differences between children with visual impairment and normal vision and between half-year age groups. Reference scores are provided for 3-year-old children with visual impairment to identify delayed fine motor development.     

Concurrent Validity of the Bayley-III and the Peabody Developmental Motor Scales-2 at 18 Months

Aim: To determine concurrent validity between the Bayley Scales of Infant and Toddler Development, 3^rd edition (Bayley-III) and the Peabody Developmental Motor Scales, 2^nd edition (PDMS-2). Methods: Both assessments were administered to 184 preterm children at 18?months corrected age; standard scores for total score, gross motor, and fine motor were calculated for each child. Cross-tabulation and Pearson correlation coefficient (r) determined concurrent validity between the Bayley-III and the PDMS-2 motor domains. Results: High correlations were found between total motor (r?=?0.88), gross motor (r?=?0.88), and fine motor scores (r?=?0.79). Both assessments had 93% agreement on classification for motor impairment; 23 children were identified by both assessments as having motor impairments, but 12 children were identified differently on each assessment (7 as impaired on PDMS-2 but average on Bayley-III; 5 as impaired on Bayley-III but average on PDMS-2). Most children with motor impairments were identified as 1SD below the mean on the PDMS-2 (27/30) and Bayley-III (18/28); however, the Bayley-III identified more children 2SD below the mean (10/28) compared to the PDMS-2 (3/30). Conclusions: Both the Bayley-III and PDMS-2 identify motor delays in children; however, clinicians should be aware of the concurrent validity as each assessment may lead to differing results.     

北极星移动评价量表在Duchenne型肌营养不良患儿中的信度研究

目的　了解北极星移动评价量表（North Star Ambulatory Assessment，NSAA）应用在Duchenne型肌营养不良（Duchenne muscular dystrophy，DMD）患儿中的组间信度及组内信度，并探讨通过录像法评测运动功能的可行性。方法　分析2017年7月至2017年11月北京大学第一医院使用NSAA对40例DMD患儿进行运动功能评估的资料。40例DMD患儿年龄4.3～13.4岁，均为男性；分为3个年龄亚组，＜5岁的10例，5～9岁的18例，＞9岁的12例。由3名治疗师A、 B、 C进行评估。在组间信度的研究中，评估者A应用NSAA对DMD患儿进行现场运动评估并摄像，评估者B、 C分别通过录像进行独立评分，通过三者对同一患儿的评分计算组间信度。在组内信度的研究中，与第1次评估至少间隔1个月以上，评估者分别根据录像对每个患儿进行第2次评估，根据评估者B、 C对同一患儿两次评估的评分计算组内信度。根据评估者A对同一患儿两次评估的评分，计算录像评分与现场评分的一致性。通过计算相关系数检测信度。结果　在组间信度的研究中，总体相关系数为0.990，各年龄亚组相关系数为0.971～0.992； 在组内信度的研究中，总体相关系数为0.987～0.988； 录像评分与现场评分一致性相关系数为0.980。结论　NSAA在评估DMD患儿运动功能水平时具有较高的信度，且现场评分和对录像评分一致性好，可能适用于远程康复评估。     

Reliability of Inclinometer-Derived Passive Range of Motion Measures in Youth with Cerebral Palsy

Aims: This study evaluated, for youth with cerebral palsy (CP), the reliability of passive range of motion (PROM) measures taken with an inclinometer, a device that may be simpler to use than a goniometer. Methods: The PROM for elbow and wrist extension, ankle dorsal flexion (knee flexed 90° and fully extended), and the knee popliteal angle of 30 youth with CP (18 boys, 12 girls, 7.0?±?3.8 years old, classified in Gross Motor Function Classification levels I–V) was measured using an inclinometer. Two physical therapists took the measures during two different sessions, a maximum of 1 week apart. Results: Good mean intra-rater inter-session, inter-rater intra-session, and inter-rater inter-session reliability (ICC = 0.75–0.89), was found for the elbow, ankle, and knee sites. Absolute reliability for these sites and conditions was 7–14° (90% confident) and 10–16° (95% confident). Reliability values for wrist extension were comparable, albeit slight lower. Conclusions: Similar to published values for goniometry, inclinometery yields reliable upper and lower limb PROM measures from ambulatory and non-ambulatory youth with CP whether measures are carried out by different evaluators within or across sessions or whether measures are performed by the same evaluator across sessions.     

Test–retest Reproducibility of the Assessment of Motor and Process Skills in Children with Unilateral Cerebral Palsy

Aims: To examine test–retest reproducibility of the Assessment of Motor and Process Skills (AMPS) in children aged 8–16 years with unilateral cerebral palsy (UCP). Methods: Thirty children with mild to moderate UCP (mean age = 11y 7m, SD 2y 4m; males = 18; Manual Ability Classification System level I = 10, II = 20; Gross Motor Function Classification System level I = 9, II = 21) enrolled in a large randomized controlled trial were recruited via consecutive series sampling. Children carried out two AMPS tasks over two consecutive days according to standardized AMPS administration procedures. The standard error of measurement (SEM), smallest detectable change (SDC), 95% limits of agreement using the Bland–Altman method, and intraclass correlation coefficients (ICC; 2,1) were calculated. Results: The SDC was 0.23 logits for the AMPS motor scale and 0.30 logits for the AMPS process scale. Test–retest reliability was excellent for both the AMPS motor scale (ICC = 0.93) and the AMPS process scale (ICC = 0.86). Intra-rater reliability (n = 10) was excellent for AMPS motor scale (ICC = 0.96) and AMPS process scale (ICC = 0.98). Conclusions: The AMPS can be used by therapists with 8- to 16-year-old children with UCP as an outcome measure with changes in scores reflecting real changes in performance or capacity.     

Reproducibility of Muscle Strength Testing for Children with Spina Bifida

Aims: Examine the inter-rater reliability of the Daniels and Worthingham (D&;W) manual muscle test (MMT) protocol for children with spina bifida between experienced and novice physiotherapists. Methods: Thirteen children with spina bifida (two males) aged 6 to 18 years were recruited from a spina bifida service in a state-wide tertiary children's hospital. Each child had the muscle strength of 19, and lower limb muscles were measured bilaterally by one experienced and one novice physiotherapist using the D&;W MMT protocol. Examiner and limb order were counterbalanced. Standard training and recording sheets were used. Agreement between raters was examined with quadratic weighted kappa (κ_w2), percentage exact agreement (%EA), and the Bland–Altmann Limits of Agreement (LoA). Minimal detectable change (MDC) was calculated. Results: Inter-rater agreement between the experienced and novice raters was excellent (κ_w2 = 0.95; 95% confidence interval: 0.94–0.96); %EA was good (72.3%); and clinically appropriate LoA levels were ?1.30–1.40. The MDC was 1.11 points on a six-point MMT scale. Conclusions: The D&;W MMT protocol, when used after standard training and with standard recording sheets, was reproducible for children with spina bifida, aged 6–18 years.     

Test-retest and inter-rater reliability for the Comprehensive Developmental Inventory for Infants and Toddlers diagnostic and screening tests

BACKGROUND: Reliability information for the Comprehensive Developmental Inventory for Infants and Toddlers diagnostic (CDIITDT) and screening tests (CDIITST) is inadequate. AIM: To assess the test-retest and inter-rater reliability of the CDIITDT and CDIITST. STUDY DESIGN: A repeated measures design was selected. SUBJECTS: Non-disabled term (n=15; mean age 8.4+/-1.6 months) and preterm infants (n=16; mean age 9.3+/-2.9 months), and children with developmental disabilities (n=15; mean age 24.7+/-11.8 months) were recruited. A single rater assessed the children twice in 3 days to examine the test-retest reliability; and a second rater observed and scored performance while the same rater conducted the first assessment for the inter-rater reliability analysis. OUTCOME MEASURES: The raw score, developmental age (DA) and developmental quotient (DQ)/Z score for the six subtests, two motor subdomains and the whole test were used as outcome measures for the CDIITDT and CDIITST. RESULTS: The test-retest reliabilities for the CDIITDT were rated good for the three pediatric groups (ICC 0.76-1.00), with the exception of moderate ratings for the self-help subtest for the term infants and for the social, self-help and fine-motor DQs for the preterm group. The CDIITDT inter-rater reliabilities were good for the three groups (ICC 0.76-1.00), with the exception of only moderate reliability for the cognitive DQs for the preterm infants. The reliabilities for the whole CDIITST for the three groups were high (ICC 0.93-1.00). CONCLUSION: The reliabilities for the whole CDIITDT and its various subtests and the whole CDIITST are acceptable for clinical use.     

The Development of an Instrument for Faculty to Assess Resident-Led Large Group Teaching

ObjectiveThe Accreditation Council on Graduate Medical Education requires residents to teach and many residency programs assess resident teaching competency. While much formal resident-led teaching is for large groups, no corresponding published assessment instrument with validity evidence exists. We developed an instrument for faculty to assess pediatric resident-led large group teaching and gathered preliminary validity evidence.MethodsLiterature review and our experience leading resident-as-teacher curricula informed initial instrument content. Resident focus groups from 3 northeastern pediatric residency programs provided stakeholder input. A modified Delphi panel of international experts provided iterative feedback. Three investigators piloted the instrument in 2018; each assessed 8 video recordings of resident-led teaching. We calculated Cronbach's alpha for internal consistency and intraclass correlation (ICC) for inter-rater reliability.ResultsThe instrument has 6 elements: learning climate, goals/objectives, content, promotion of understanding/retention, session management, and closure. Each element contains behavioral subelements. Cronbach's alpha was .844. ICC was excellent for 6 subelements, good for 1, fair for 1, and poor for 3.ConclusionsWe developed an instrument for faculty assessment of resident-led large group teaching. Pilot data showed assessed behaviors had good internal consistency, but inconsistent interrater reliability. With further development, this instrument has potential to assess resident teaching competency.     

Validity of the Fine Motor Area of the 12-Month Ages and Stages Questionnaire in Infants Following Major Surgery

ABSTRACT

The Ages and Stages Questionnaires (ASQ) are parent-report screening tools to identify infants at risk of developmental difficulties. The purpose of this study was to examine validity and internal reliability of the fine motor developmental area of the ASQ, 2nd edition (ASQ2-FM) for screening 12-month-old infants following major surgery. The ASQ2-FM was completed by caregivers of 74 infants who had cardiac surgery in the first 90 days of life, 104 infants who had noncardiac surgery in the first 90 days of life, and a control group of 154 infants. The Rasch item response analysis revealed that the ASQ2-FM had poor ability to discriminate among levels of fine motor ability. Sensitivity was poor (20%) and specificity was good (98%) when compared with the scores for the fine motor subscale of the Bayley Scales of Infant and Toddler Development. The ASQ2-FM under-identified infants at risk for fine motor delay; internal reliability and construct validity do not support use as a screening tool of fine motor development of infants aged 12 months who have undergone major surgery.     

Observer reliability in grading nephrocalcinosis on ultrasound examinations in children

Background. Nephrocalcinosis is often associated with a variety of hypercalcemic conditions. Diagnostic ultrasound is often used for assessing nephrocalcinosis in children, but its reliability has not been proven. Objective. To determine the reliability of expert interpretation of sonographic films with a grading scale of severity for nephrocalcinosis. Materials and methods. Fifty-eight ultrasonographic films of 30 children with Williams syndrome and other conditions know to be associated with nephrocalcinosis were assessed. We used a blinded randomized design to assess intra- and interobserver reliability. Results. Grades I, II, and III nephrocalcinosis were noted in 13 %, 19 %, and 27 % of the examinations, respectively. The weighted kappa coefficient was 0.80 (standard error 0.12; 95 % confidence interval 0.68–0.92) for intraobserver agreement and 0.76 (standard error 0.13; 95 % confidence interval 0.63 to 0.89) for interobserver agreement. Reliability in assessing change from one examination to the next, with independently graded films, was fair with an unweighted kappa coefficient of 0.68 (95 % confidence interval 0.38–0.96) and 0.51 (95 % confidence interval 0.21–0.80) for intra- and interobserver reliability, respectively. Conclusion. The severity of nephrocalcinosis can be reliably interpreted with an ultrasonography grading scale. Received: 30 October 1997 Accepted: 4 August 1998     

A Novel Scale to Communicate Perceived Likelihood of Child Sexual Abuse

Background and ObjectiveChild abuse pediatricians (CAPs) are often asked to determine the likelihood that a particular child has been sexually abused. These determinations affect medical and legal interventions, and are important for multisite research. No widely accepted scale is available to communicate perceived sexual abuse likelihood. In this study, we measure intra- and inter-rater reliability of a 5-point scale to communicate child sexual abuse likelihood.MethodsWe developed a 5-point scale of perceived likelihood of child sexual abuse with example cases and medical-legal language for each risk category. We then surveyed CAPs who regularly perform sexual abuse evaluations using the abstracted facts of 15 actual cases with concern for sexual abuse. A subset of participants rated the same vignettes again, 1 month later.ResultsOf 512 invited participants, 240 (46.7%) responded, with 145 (28.3%) indicating that they regularly perform sexual abuse evaluations, 116 initially completing all 15 vignettes, and 36 completing repeat ratings at least 1 month later. The scale showed consistent stepwise increase in mean perceived likelihood of abuse and intention to report for each increase in scale rating. Inter-rater agreement was substantial (Fleiss’ weighted kappa 0.64) and test-retest reliability among 36 participants was almost perfect (Cohen's kappa = 0.81).ConclusionsWe introduce a scale of perceived sexual abuse likelihood that appears to reflect CAPs’ perceptions and intention to report. This scale may be a reasonable metric for use in multicenter studies. CAPs demonstrated substantial inter- and intrarater reliability when evaluating sexual abuse likelihood in case vignettes. While this scale may improve communication of sexual abuse likelihood among experts, its examples should not be used as a legal standard or a clinical criterion for sexual abuse diagnosis.     

复旦中文版脑瘫幼儿手功能分级系统的信度和效度研究

摘要 目的：建立复旦中文版脑瘫幼儿手功能分级系统（Mini-MACS)的信度和效度。方法：以2017年10月至2017年11月在复旦大学附属儿科医院（我院）康复中心以及上海儿童康复合作群（8家单位）接受康复干预的1~4岁的脑瘫（CP）儿童为研究对象,排除有严重视觉和听觉障碍的CP儿童。通过分析现场操作评价和录像评价结果之间的关系确定Mini-MACS的重测信度;分析不同评估者间［主要照顾者、作业治疗师（OT师）、物理治疗师（PT师）］的评价结果确定Mini-MACS的评估者间信度;以脑瘫儿童精细运动能力测试（FMFM）的精细运动能力分值为效标确定Mini-MACS的平行效度。结果：76例CP儿童进入本文分析,其中我院康复中心18例,上海儿童康复合作群单位共58例;男46例,女30例;平均年龄（2.5±0.9）个月,1~2岁组25例,~3岁组23例,~4岁组28例;痉挛型四肢瘫29例,双瘫11例,偏瘫27例,手足徐动型2例,共济失调型2例,无法分类5例;粗大运动功能分级（GMFCS）Ⅰ级7例,Ⅱ级27例,Ⅲ级16例,Ⅳ级16例,Ⅴ级10例。OT师现场评价与录像评价结果间重测信度ICC值为0.96（95%CI：0.94~0.98）;OT师与PT师间的ICC值为0.97（95%CI：0.94~0.98）,OT师与主要照顾者ICC值为0.92（95%CI：0.86~0.95）;1~2岁组、~3岁组和~4岁组重测信度ICC值0.95~1.00,~4岁组评价者间信度低于1~2岁组和~3岁组,尤其是OT师与主要照顾者间的信度明显低于其他两组（ICC=0.64）。Mini-MACS与精细运动能力分值之间平行效度Spearman相关系数为-0.76。结论：中文版Mini-MACS具有良好的评价者间信度和平行效度,适用于对国内脑瘫幼儿进行手功能分级。     

Validity and reliability of a Swedish version of the Non‐Communicating Children’s Pain Checklist – Postoperative Version

Aim: To test the validity and reliability of a Swedish version of the Non‐Communicating Children’s Pain Checklist – Postoperative Version (NCCPC‐PV). Methods: Thirty‐two consecutive children/adolescents (2–20 years of age) with cognitive impairment and no verbal communication from four habilitation centres were admitted to the study. Each child’s behaviour was observed by a parent or a caregiver and by a physiotherapist in two calm and two painful situations within the child’s everyday life. The raters independently assessed and graded the child’s behaviour during 5 min according to the translated Swedish version of the NCCPC‐PV. The intrarater and interrater reliability were determined, and the construct validity was examined. Results: The results from 202 assessments showed that the construct validity was good: children’s behavioural signs differed significantly between situations of pain and situations of calm (p < 0.001). Repeated assessments showed poor agreement both within and between raters [intraclass correlation coefficient (ICC) 0.51–0.65]. The agreement for pain was good (ICC 0.83). Conclusion: The Swedish version of the NCCPC‐PV can be used for pain assessment in children with cognitive impairments who lack verbal communication. Aspects of reliability need to be further analysed.