The influence of validity criteria on Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) test–retest reliability among high school athletes

Objective: Research findings to date on the stability of Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) Composite scores have been inconsistent, requiring further investigation. The use of test validity criteria across these studies also has been inconsistent. Using multiple measures of stability, we examined test–retest reliability of repeated ImPACT baseline assessments in high school athletes across various validity criteria reported in previous studies. Method: A total of 1146 high school athletes completed baseline cognitive testing using the online ImPACT test battery at two time periods of approximately two-year intervals. No participant sustained a concussion between assessments. Five forms of validity criteria used in previous test–retest studies were applied to the data, and differences in reliability were compared. Results: Intraclass correlation coefficients (ICCs) ranged in composite scores from .47 (95% confidence interval, CI .38, .54]) to .83 (95% CI .81, .85]) and showed little change across a two-year interval for all five sets of validity criteria. Regression based methods (RBMs) examining the test–retest stability demonstrated a lack of significant change in composite scores across the two-year interval for all forms of validity criteria, with no cases falling outside the expected range of 90% confidence intervals. Conclusion: The application of more stringent validity criteria does not alter test–retest reliability, nor does it account for some of the variation observed across previously performed studies. As such, use of the ImPACT manual validity criteria should be utilized in the determination of test validity and in the individualized approach to concussion management. Potential future efforts to improve test–retest reliability are discussed.