Threats to validity in the longitudinal study of psychological effects: the case of short stature

In all studies of health-related problems and their effects on well-being, research design issues threaten to compromise the validity of findings. This is particularly so in a longitudinal study, essentially stemming from the tension between maintaining participant compliance and retaining investigator objectivity. Such a tension may be exacerbated where measures of dependent variables such as self-esteem are used alongside the collection of physical data which is essential to the study, as in research into the psychological effects of short stature on children and young people. In this paper one particular project, the Wessex Growth Study, is used to illustrate the common threats to validity, both internal and external, of such research, and to consider future improvements in design. The Wessex Growth Study, set up in 1986, was designed to overcome some of the methodological problems found in earlier research with short stature children. It is following the growth and psychological development through their school years of a cohort of short children (below third centile for height when first identified) and case-matched controls (10th-90th centiles) recruited at school entry (ages 5/6). Findings have generally found only small differences between short and average height children. Though these results so far have mainly been presented cross-sectionally, the young people involved are followed up at 6-monthly intervals for height and other data to be collected, and thus to some extent the study also has the advantages and problems of a longitudinal research design. Using Campbell and Stanley's criteria this article makes clear the strain on both internal and external validity in the study, but argues that these problems are to some extent inherent in all longitudinal psychological research, and are outweighed in the present research by the collection of data on short stature which would not otherwise be available. Future data collection within the study will introduce further improvements in design.     

Concurrent validity of the alcohol use disorders identification test (AUDIT)

AIMS: To determine cut-offs for the Alcohol Use Disorders Identification Test (AUDIT) 10-item questionnaire, differentiating hazardous drinking (HZD) and alcohol use disorders (AUD) from low risk drinking (LRD), and AUD from HZD and LRD among married men in a Sri Lankan sample. METHODS: Using 62 low risk drinkers and 88 each from hazardous drinkers and AUD, the AUDIT instrument was compared with adapted and translated versions of a beverage-specific, quantity-frequency questionnaire, and the alcohol use module of the Composite International Diagnostic Interview (CIDI), and two receiver operating characteristic (ROC) curves were plotted. RESULTS: The area under the ROC curves to differentiate HZD + AUD from LRD and AUD from HZD + LRD were 0.96 (95% CI: 0.94-0.99) and 0.97 (95% CI: 0.95-0.99) respectively. The cut-off values of 7 and 16 were observed to have the best trade-offs between sensitivity, specificity, the ratio of positive likelihood to negative likelihood ratios, and positive predictive values. CONCLUSION: The AUDIT could be used to screen for LRD, HZD, and AUD among Sinhalese married men in Sri Lanka.     

Threats to the validity of locally developed multiple-choice tests in medical education: construct-irrelevant variance and construct underrepresentation

Construct-irrelevant variance (CIV) - the erroneous inflation or deflation of test scores due to certain types of uncontrolled or systematic measurement error - and construct underrepresentation (CUR) - the under-sampling of the achievement domain - are discussed as threats to the meaningful interpretation of scores from objective tests developed for local medical education use. Several sources of CIV and CUR are discussed and remedies are suggested. Test score inflation or deflation, due to the systematic measurement error introduced by CIV, may result from poorly crafted test questions, insecure test questions and other types of test irregularities, testwiseness, guessing, and test item bias. Using indefensible passing standards can interact with test scores to produce CIV. Sources of content under representation are associated with tests that are too short to support legitimate inferences to the domain and which are composed of trivial questions written at low-levels of the cognitive domain. ``Teaching to the test' is another frequent contributor to CUR in examinations used in medical education. Most sources of CIV and CUR can be controlled or eliminated from the tests used at all levels of medical education, given proper training and support of the faculty who create these important examinations. This revised version was published online in June 2006 with corrections to the Cover Date.     

Threats to human health in the use of pesticides (a review)

The paper presents current trends in the use of pesticides and detection of their residues in the environmental objects. Pesticides are shown to be a serious health risk factor for different groups of the population when they are irrationally used and inadequately stored. A package of measures to prevent intoxication and to reduce a risk for pesticide entry into the body is provided.     

Threats to internal validity in worksite health promotion program research: common problems and possible solutions

Reviews of the research on worksite health promotion programs reveal that most studies are limited in their ability to draw clear inferences about program effects because the studies employed flawed research designs and/or analyses. Conclusions are often drawn about program effectiveness with little consideration given to alternative explanations for the findings. In an effort to promote improved research, this article uses the Cook and Campbell delineation of threats to valid causal inference to illustrate how the threats can operate in worksite health promotion program research as well as how they can be examined and controlled. Researchers, even those attempting to conduct true experiments, must consider the existence of all plausible threats to validity and control or rule them out before valid causal inferences can be drawn. The theoretical and design issues involved in worksite health promotion program research are presented, followed by a discussion of threats to internal validity.     

Constructing arguments for the interpretation and use of patient-reported outcome measures in research: an application of modern validity theory

Quality of Life Research - The past 100 years have witnessed an evolution of the meaning of validity and validation within the fields of education and psychology. Validity was once viewed...