The development and testing of a module on child functioning for identifying children with disabilities on surveys. II: Question development and pretesting

Background

This paper is part of a series of articles documenting the development of a module on child functioning by UNICEF in collaboration with the Washington Group on Disability Statistics (WG). This paper documents the contribution of Cognitive Interview (CI) question evaluation methods to the development of the final module.

Objective

The overall goal of this project was to develop a cross-nationally comparable module to measure child functioning and disability. Specifically, the goals of the question evaluation study were to investigate question interpretation, sources of error and bias and to use the results iteratively in the development of the final module.

Methods

As is standard in CI studies, data were gathered through one-one-one, in-depth interviews. A total of four rounds of testing, comprising 385 Cognitive Interviews, were conducted across six countries. Qualitative data analysis methods were used to identify patterns of question interpretation and areas of potential error and bias among sub-groups of respondents.

Results

Through an iterative process of testing and revision, analytic findings from these interviews were used to guide decisions on question inclusion, revision and deletion. Four types of revisions were made: 1) changing, deleting or adding specific words; 2) moving, deleting or adding clarifying phrases; 3) revising or deleting items for conceptual clarity; and 4) adding examples.

Conclusions

These efforts to reduce error and bias resulted in a validated module that can provide cross-nationally comparable measures of child functioning.