Consequences of ignoring the response-shift and measure non-invariant items in sleep studies: an empirical data based simulation of the treatment effect of CBT-I on dysfunctional sleep beliefs

IntroductionPrevious studies have shown that psychological interventions do not only improve patients' mental symptoms (i.e., true change) but may also change the internal standards patients use to evaluate their symptoms (i.e., response shifts). Although the response shifts could reflect patients’ cognitive changes toward their disorders as the interventions aim to achieve, failing to differentiate them from the true change during data analyses could bias the research conclusions. Considering this issue is seldom discussed in sleep studies, this study thus examined the impacts of response-shift items in an intervention study of cognitive behavioral therapy for insomnia (CBT-I) via empirical-data based simulations.MethodWe used longitudinal measurement invariance tests to identify the items in an abbreviated version of the Dysfunctional Beliefs and Attitudes about Sleep Scale that are non-invariant (response shifted) against CBT-I based on data from 114 insomnia patients. The partial invariance model built accordingly was then used as a population model for simulations to examine the impacts of the response-shift items on follow-up paired t-tests.ResultsInvariance tests indicate CBT-I would lift the intercept of one item in DBAS-10 and cause non-uniform calibrations in three items. The following up simulations showed that failing to exclude the intercept-lifted item from the calculations of the subscale scores would lower the probability of using paired t-test to correctly detect the treatment effect by up to 53%.ConclusionsWe recommend sleep researchers to consider the issues of response-shift when assessing sleep-related constructs in interventional studies for insomnia.