Poor Performance Validity Predicts Clinical Characteristics and Cognitive Test Performance of OEF/OIF/OND Veterans in a Research Setting

This study examined the performance of 198 Veteran research participants deployed during Operation Enduring Freedom, Operation Iraqi Freedom, and/or Operation New Dawn (OEF/OIF/OND) on four measures of performance validity: the Medical Symptom Validity Test (MSVT), California Verbal Learning Test: Forced Choice Recognition (FCR), Reliable Digit Span (RDS), and TOVA Symptom Exaggeration Index (SEI). Failure on these performance validity tests (PVTs) ranged from 4% to 9%. The overall base rate of poor performance validity, as measured by failure of the MSVT in conjunction with an embedded PVT (FCR, RDS, SEI), was 5.6%. Regression analyses revealed that poor performance validity predicted cognitive test performance and self-reported psychological symptom severity. Furthermore, a greater prevalence of traumatic brain injury (TBI), Post-Traumatic Stress Disorder (PTSD), co-morbid TBI/PTSD, and other Axis I diagnoses, was observed among participants with poor effort. Although poor performance validity is relatively uncommon in a research setting, these findings demonstrate that clinicians should be cautious when interpreting psychological symptoms and neuropsychological test performance of Veteran participants who fail effort measures.     

Young Child With Severe Brain Volume Loss Easily Passes the Word Memory Test and Medical Symptom Validity Test: Implications for Mild TBI

The Word Memory Test (WMT) and Medical Symptom Validity Test (MSVT) are two commonly used free-standing measures of test-taking effort. The use of any test as a measure of effort is enhanced when evidence shows that it can be easily passed by patients with severe neurological conditions. The opportunity arose to administer the WMT and MSVT to a 9-year-old girl (referred to as CJ) with severe congenital bilateral brain tissue loss (shown via a compelling brain MRI image), chronic epilepsy, an extremely low Full Scale IQ, extremely low adaptive functioning, developmental delays, numerous severe cognitive impairments, and treatment with multiple high-dose benzodiazepines. She received extensive early intervention services and numerous academic accommodations. Despite this set of problems, CJ passed the WMT and MSVT at perfect to near perfect levels. Implications for failure on these tests among patients with known or alleged mild traumatic brain injury are discussed.     

Confronting Patients About Insufficient Effort: The Impact on Subsequent Symptom Validity and Memory Performance

Symptom validity tests (SVTs) are commonly used to assess effort in neuropsychological evaluations. However, no empirical research or official guidelines exist about how clinicians should proceed if a patient produces a non-valid SVT result. The purpose of this study was to examine whether confronting patients immediately after scoring in a non-valid range on a SVT would have an impact on subsequent symptom validity and memory tests performance. Archival patient data for 507 adults with clinically definite multiple sclerosis (MS) (ages 18–76) were examined. All patients completed the Victoria Symptom Validity Test (VSVT), the Wechsler Memory Scale, 3rd edition (WMS III), and the Beck Depression Inventory, 2nd edition (BDI II). Although the majority (89%) of patients produced valid VSVT scores (the Valid group), 56 patients produced non-valid VSVT scores. Due to a change in clinical procedure, 28 of the 56 were confronted regarding their non-valid VSVT performances and were asked to complete the test a second time (the CONF group), while the remaining 28 proceeded with testing as usual following a non-valid score (the N-CONF group). Results showed that 68% of the CONF group produced valid VSVT scores on re-administration, as well as memory performances that were comparable to those of the Valid group. In contrast the N-CONF group produced memory scores that were significantly below the Valid group. This is the first study to provide empirical support for the effectiveness of intervention when patients exhibit inadequate effort on SVTs in clinical, non-forensic settings.     

Intent to Fail: Significance Testing of Forced Choice Test Results

A score that is significantly below the level of chance on a forced choice (FC) performance validity test results from the deliberate production of wrong answers. In order to increase the power of significance testing of a below chance result on standardized FC tests with empirically derived cutoff scores, we recommend using one-tailed tests of significance and selecting probability levels greater than .05 (.20 for most standardized FC tests with empirically derived cutoff scores). Under certain circumstances, we also recommend combining scores from different sections of the same FC test and combining scores across different FC tests. These recommendations require modifications when applied to non-standardized FC tests that lack empirically derived cutoff scores or to FC tests with a non-random topographical distribution of correct and incorrect answers.     

Rates of Below-Chance Performance in Forced-Choice Symptom Validity Tests

The rates of significantly below-chance results on three neuropsychological symptom validity tests (SVTs) including the Portland Digit Recognition Test (PDRT), Test of Memory Malingering (TOMM), and Word Memory Test (WMT) were compared in a private practice forensic sample of 1032 examinees with alleged mild traumatic brain injury, moderate to severe traumatic brain injury, alleged toxic exposure, and reported chronic pain. The PDRT and WMT were equivalent to one another in the rates of below-chance results, with both yielding more frequent below-chance results than the TOMM. Seemingly more difficult sections of the PDRT and WMT had higher yields than seemingly easier sections. Multiple SVTs were more likely to yield below-chance results than a single test, supporting the use of multiple SVTs in forensic neuropsychological evaluations.     

Effort Test Failure: Toward a Predictive Model

Predictors of effort test failure were examined in an archival sample of 555 traumatically brain-injured (TBI) adults. Logistic regression models were used to examine whether compensation-seeking, injury-related, psychological, demographic, and cultural factors predicted effort test failure (ETF). ETF was significantly associated with compensation-seeking (OR?=?3.51, 95% CI [1.25, 9.79]), low education (OR:. 83 [.74, . 94]), self-reported mood disorder (OR: 5.53 [3.10, 9.85]), exaggerated displays of behavior (OR: 5.84 [2.15, 15.84]), psychotic illness (OR: 12.86 [3.21, 51.44]), being foreign-born (OR: 5.10 [2.35, 11.06]), having sustained a workplace accident (OR: 4.60 [2.40, 8.81]), and mild traumatic brain injury severity compared with very severe traumatic brain injury severity (OR: 0.37 [0.13, 0.995]). ETF was associated with a broader range of statistical predictors than has previously been identified and the relative importance of psychological and behavioral predictors of ETF was evident in the logistic regression model. Variables that might potentially extend the model of ETF are identified for future research efforts.     

Test of Memory Malingering: Cutoff Scores for Psychometrically Defined Malingering Groups in a Military Sample

This research examined cutoffs for the Test of Memory Malingering (TOMM) in a military sample composed primarily of mTBI patients. The results are consistent with previous research and provide additional evidence that cutoffs higher than those originally recommended for the TOMM can produce excellent classification and diagnostic statistics when a psychometrically defined non-malingering group is compared with three psychometrically defined malingering groups: Probable, Probable to Definite, and Definite Malingering. The groups were formed based on the number of symptom and performance validity tests passed or failed. Cutoffs that were 4–5 points higher for Trial 2 and the Retention Trial than originally recommended produced very low false positive rates (.0 to .06) and excellent positive predictive values (.75 to 1.00) for a base rate of malingering commonly found in TBI patients. Positive likelihood ratios were all above 10 for these two trials indicating excellent ability to rule in malingering. A range of cutoffs for Trial 1 were also examined, and classification and diagnostic statistics are presented for cutoffs ranging from 40 to 44 with results similar to the other TOMM trials.     

Interpreting the Meaning of Multiple Symptom Validity Test Failure

While it is recommended that judgments regarding the credibility of test performance be based on the results of more than one effort indicator, and recent efforts have been made to improve interpretation of multiple effort test failure, the field currently lacks adequate guidelines for using multiple measures of effort in concert with one another. A total of 103 patients were referred for outpatient neuropsychological evaluation, which included multiple measures of negative response bias embedded in standard test batteries. Using any pairwise failure combination to predict diagnostic classification was superior (sensitivity = 83.8%, specificity = 93.9%, overall hit rate = 90.3%) to using any one test by itself and to using any three-test failure combination. Further, the results were comparable to the results of logistical regression analyses using the embedded indicators as continuous predictors. Given its parsimony and clinical utility, the pairwise failure model is therefore a recommended criterion for identifying non-credible performance; however, there are of course other important contextual factors and influences to consider, which are also discussed.     

Lorazepam Effects on Word Memory Test Performance: A Randomized,Double-Blind,Placebo-Controlled,Crossover Trial

The Word Memory Test (WMT) is a common measure of symptom validity. To investigate the effects of acute benzodiazepines on WMT scores, oral lorazepam 2?mg (LOR) and placebo were administered 1 week apart in a randomized, double-blind, placebo-controlled, crossover study. A total of 28 participants completed the study and were administered the WMT during each drug condition. Within-participant comparisons of LOR vs placebo revealed significant LOR effects for Immediate Recognition (p?=?.007) and Consistency (p?=?.019), but not Delayed Recognition (p?=?.085). Significant LOR effects were present for Reaction Time Measures (Immediate Recognition RT, p?=?.013; Delayed Recognition RT, p?=?.001; Multiple Choice RT, p?=?.011) and Delayed Memory scores (Multiple Choice, p?=?.007; Paired Associates, p?=?.029; Free Recall, p?=?.001). A pattern similar to crossover results was detected for LOR vs placebo between-group differences for initial test assessment scores. When examined using publisher recommended cut scores for the principal WMT measures, there were six participants failing the WMT during initial LOR testing; all six subsequently performed in the normal range upon retesting with placebo. One participant failed WMT during placebo and obtained passing scores during LOR. These data indicate that multiple WMT measures may be affected by acute LOR dosing, and provide additional evidence that potential latent variables and their effects on both SVT performance and cognitive function should be part of the clinical decision-making process.     

Comparison of Credible Patients of Very Low Intelligence and Non-Credible Patients on Neurocognitive Performance Validity Indicators

The purpose of this archival study was to identify performance validity tests (PVTs) and standard IQ and neurocognitive test scores, which singly or in combination, differentiate credible patients of low IQ (FSIQ ≤ 75; n = 55) from non-credible patients. We compared the credible participants against a sample of 74 non-credible patients who appeared to have been attempting to feign low intelligence specifically (FSIQ ≤ 75), as well as a larger non-credible sample (n = 383) unselected for IQ. The entire non-credible group scored significantly higher than the credible participants on measures of verbal crystallized intelligence/semantic memory and manipulation of overlearned information, while the credible group performed significantly better on many processing speed and memory tests. Additionally, credible women showed faster finger-tapping speeds than non-credible women. The credible group also scored significantly higher than the non-credible subgroup with low IQ scores on measures of attention, visual perceptual/spatial tasks, processing speed, verbal learning/list learning, and visual memory, and credible women continued to outperform non-credible women on finger tapping. When cut-offs were selected to maintain approximately 90% specificity in the credible group, sensitivity rates were highest for verbal and visual memory measures (i.e., TOMM trials 1 and 2; Warrington Words correct and time; Rey Word Recognition Test total; RAVLT Effort Equation, Trial 5, total across learning trials, short delay, recognition, and RAVLT/RO discriminant function; and Digit Symbol recognition), followed by select attentional PVT scores (i.e., b Test omissions and time to recite four digits forward). When failure rates were tabulated across seven most sensitive scores, a cut-off of ≥ 2 failures was associated with 85.4% specificity and 85.7% sensitivity, while a cut-off of ≥ 3 failures resulted in 95.1% specificity and 66.0% sensitivity. Results are discussed in light of extant literature and directions for future research.     

Assessing Effort: Differentiating Performance and Symptom Validity

The current study aimed to clarify the relationship among the constructs involved in neuropsychological assessment, including cognitive performance, symptom self-report, performance validity, and symptom validity. Participants consisted of 120 consecutively evaluated individuals from a veteran’s hospital with mixed referral sources. Measures included the Wechsler Adult Intelligence Scale-Fourth Edition Full Scale IQ (WAIS-IV FSIQ), California Verbal Learning Test-Second Edition (CVLT-II), Trail Making Test Part B (TMT-B), Test of Memory Malingering (TOMM), Medical Symptom Validity Test (MSVT), WAIS-IV Reliable Digit Span (RDS), Post-traumatic Check List-Military Version (PCL-M), MMPI-2 F scale, MMPI-2 Symptom Validity Scale (FBS), MMPI-2 Response Bias Scale (RBS), and the Postconcussive Symptom Questionnaire (PCSQ). Six different models were tested using confirmatory factor analysis (CFA) to determine the factor model describing the relationships between cognitive performance, symptom self-report, performance validity, and symptom validity. The strongest and most parsimonious model was a three-factor model in which cognitive performance, performance validity, and self-reported symptoms (including both standard and symptom validity measures) were separate factors. The findings suggest failure in one validity domain does not necessarily invalidate the other domain. Thus, performance validity and symptom validity should be evaluated separately.     

Validity Testing in Dually Diagnosed Post-Traumatic Stress Disorder and Mild Closed Head Injury

Prospects for the coexistence of post-traumatic stress syndrome (PTSS) and mild traumatic brain injury (mTBI) rely exclusively on subjective evidence, increasing the risk of response bias in a compensatable social context. Using a priori specificities derived from genuine brain disorder groups, we examined validity failure rates in three domains (symptom, cognitive, motor) in 799 persons reporting persistent subjective disability long after mild neurological injury. Validity tests included the Test of Memory Malingering, MMPI-2 Fake Bad Scale, and Infrequency (F) scales, reliable digit span, and Halstead-Reitan finger tapping. Analyses showed invalidity signs in large excess of actuarial expectations, with rising invalidity risk conditional on post-traumatic complexity; the highest failure rates were produced by the 95 persons reporting both neurogenic amnesia and re-experiencing symptoms. We propose an “over-endorsement continuum” hypothesis: The more complex the post-traumatic presentation after mild neurological injury, the stronger the association with response bias. Late-appearing dual diagnosis is a litigation phenomenon so intertwined with secondary gain as to be a byproduct of it.     

Malingering of PTSD: forensic and diagnostic considerations, characteristics of malingerers and clinical presentations

OBJECTIVE: This paper aims to study the detection of individuals malingering posttraumatic stress disorder (PTSD) in criminal and civil situations. METHOD: A brief history of PTSD and its rise to prominence in legal circles are discussed. The characteristics of individuals who malinger and particularly those who fake PTSD are discussed. Diagnostic dilemmas inherent to the condition, such as the definition of a traumatic exposure, what constitutes a PTSD flashback and the potential for normal symptom exaggeration, are explored. RESULTS: The typical presentation of malingered symptoms is presented to help clinicians detect commonly seen malingering patterns. Suggestions for interview techniques, Minnesota Multiphasic Personality Inventory test values and sources of collateral information to help detect malingering are reviewed. CONCLUSION: The paper concludes with a review of the typical presentations of malingered PTSD symptoms and a reminder that physicians need to distinguish legitimate symptoms from faked or embellished presentations.     

Cumulative False Positive Rates Given Multiple Performance Validity Tests: Commentary on Davis and Millis (2014) and Larrabee (2014)

Controversy has arisen over interpretation of performance validity tests (PVTs) when multiple PVTs are given. Some papers state that more stringent criteria are needed to judge overall performance as invalid, while others argue that concerns about the number of PVTs are overstated and that widely used criteria are appropriate. We examine theoretical models and assumptions, and analyze published data to determine the magnitude of effects implied by theory and observed in practice. Assertions advanced in the primary papers are examined for consistency with the empirical data. Existing theoretical models do not account well for the diverse empirical data, substantial empirical effects remain poorly understood, and the primary papers include assertions that are not empirically supported. The results indicate that: (a) neuropsychology lacks solid theoretical bases for estimating PVT failure rates given various combinations of PVTs, and thus needs to rely on empirical data; (b) existing empirical data fail to support the application of any uniform criteria across the broad range of scenarios involving multiple PVTs; and (c) practice should rely on empirical studies involving combinations of PVTs that have been studied together, in samples clearly appropriate to the individual case, using experimental designs germane to the questions under consideration.     

I Just Want My Research Credit: Frequency of Suboptimal Effort in a Non-Clinical Healthy Undergraduate Sample

Although performance validity testing is becoming fairly routine in clinical settings, research protocols involving neuropsychological tests infrequently include assessments of performance validity. The current study utilized an embedded measure of effort over two administrations of CNS Vital Signs to determine the frequency of poor effort in non-clinical healthy undergraduate students participating in a research study for course credit. Results indicate that more than 1 in 10 college students participating in a cognitive test battery for research showed test scores consistent with inadequate effort, which was associated with poor performance on testing across many domains. This conclusion was supported by poor performance on many other subtests. Healthy college students with suboptimal effort (n = 11) had an overall score in the 15th percentile on average compared to the 48th percentile in the rest of the students (n = 66). Those who failed validity indicators on the baseline administration were more likely to fail validity indicators on the repeat administration. Those who were tested in the morning were also more likely to fail validity indicators. The current study provides evidence for the potential limitations of conducting research using neuropsychological tests with healthy college student volunteers in the absence of performance validity testing. Revised college-level cutoffs are proposed.     

Using Symptom Validity Tests to Detect Malingered ADHD in College Students

Recently there has been growing concern that college students may feign symptoms of ADHD in order to obtain academic accommodations and stimulant medication. Unfortunately research has only begun to validate detection tools for malingered ADHD. The present study cross-validated the results of Sollman, Ranseen, and Berry (2010) on the efficacy of several symptom validity tests for detection of simulated ADHD among college students. Undergraduates with a history of diagnosed ADHD were randomly assigned either to respond honestly or exaggerate symptoms, and were compared to undergraduates with no history of ADHD or other psychiatric disorders who were also randomly assigned to respond honestly or feign symptoms of ADHD. Similar to Sollman et al. (2010) and other recent research on feigned ADHD, several symptom validity tests, including the Test of Memory Malingering (TOMM), Letter Memory Test (LMT), Digit Memory Test (DMT), Nonverbal Medical Symptom Validity Test (NV-MSVT), and the b Test were reasonably successful at discriminating feigned and genuine ADHD. When considered as a group, the criterion of failure of 2 or more of these SVTs had a sensitivity of. 475 and a specificity of 1.00.     

Derivation of a Composite Measure of Embedded Symptom Validity Indices

Assessment of effort on cognitive testing has become a suggested standard in both forensic and clinical neuropsychological assessment. Both stand-alone and empirically derived embedded measures have been developed, however guidelines for combining several measures are not always available. This study used logistic regression analysis to derive a multivariable composite to detect suboptimal effort using scores extracted from commonly administered neuropsychological tests. A model predicting suboptimal effort, as defined by performance on the Medical Symptom Validity Test, was created with 124 participants using variables from the California Verbal Learning test 2nd edition, Rey Complex Figure Test and the Wechsler Memory Scale 3rd edition. The model reliably predicted suboptimal effort (χ^2?=?44.37, p?<?.001) with excellent discrimination (AUC?=?.84).     

Relationships Among Victoria Symptom Validity Test Indices and Personality Assessment Inventory Validity Scales in a Large Clinical Sample

The purpose of this study was twofold: (1) to examine the relationships among measures of cognitive symptom exaggeration (i.e., response accuracy and response latency) and (2) to examine the relationship between measures of cognitive and psychopathological symptom exaggeration. It was expected that Victoria Symptom Validity Test (VSVT) accuracy and latency measures would be significantly correlated, with invalid responders demonstrating longer response latencies. VSVT scores were also expected to correlate significantly with the Negative Impression Management (NIM) and Infrequency (INF) subscales of the Personality Assessment Inventory (PAI). VSVT and PAI data were collected from 300 patients during routine clinical neuropsychological evaluations. Results indicated that VSVT accuracy and latency measures were significantly and moderately correlated, and both types of VSVT scores were significantly, but modestly, related to NIM, but not INF. These findings suggest that VSVT response latencies may supplement accuracy scores in identifying patients who are exerting suboptimal effort on cognitive measures. These findings further suggest that measures of cognitive symptom validity only partially overlap with measures of psychopathological symptom exaggeration.     

Official Position of the American Academy of Clinical Neuropsychology Social Security Administration Policy on Validity Testing: Guidance and Recommendations for Change

The milestone publication by Slick, Sherman, and Iverson (1999) of criteria for determining malingered neurocognitive dysfunction led to extensive research on validity testing. Position statements by the National Academy of Neuropsychology and the American Academy of Clinical Neuropsychology (AACN) recommended routine validity testing in neuropsychological evaluations. Despite this widespread scientific and professional support, the Social Security Administration (SSA) continued to discourage validity testing, a stance that led to a congressional initiative for SSA to reevaluate their position. In response, SSA commissioned the Institute of Medicine (IOM) to evaluate the science concerning the validation of psychological testing. The IOM concluded that validity assessment was necessary in psychological and neuropsychological examinations (IOM, 2015 Institute of Medicine. (2015). Psychological testing in the service of disability determination. Washington, DC: Institute of Medicine. Retrieved from http://iom.nationalacademies.org/Reports/2015/PsychTesting.aspx [Google Scholar]). Objective: The AACN sought to provide independent expert guidance and recommendations concerning the use of validity testing in disability determinations. Method: A panel of contributors to the science of validity testing and its application to the disability process was charged with describing why the disability process for SSA needs improvement, and indicating the necessity for validity testing in disability exams. Results: This work showed how the determination of malingering is a probability proposition, described how different types of validity tests are appropriate, provided evidence concerning non-credible findings in children and low-functioning individuals, and discussed the appropriate evaluation of pain disorders typically seen outside of mental consultations. Conclusions: A scientific plan for validity assessment that additionally protects test security is needed in disability determinations and in research on classification accuracy of disability decisions.