Effectiveness of the Comalli Stroop Test as a Measure of Negative Response Bias

Practice guidelines recommend the use of multiple performance validity tests (PVTs) to detect noncredible performance during neuropsychological evaluations, and PVTs embedded in standard cognitive tests achieve this goal most efficiently. The present study examined the utility of the Comalli version of the Stroop Test as a measure of response bias in a large sample of “real world” noncredible patients (n = 129) as compared with credible neuropsychology clinic patients (n=233). The credible group performed significantly better than the noncredible group on all trials, but particularly on word-reading (Stroop A) and color-naming (Stroop B); cut-scores for Stroop A and Stroop B trials were associated with moderate sensitivity (49–53%) as compared to the low sensitivity found for the color interference trial (29%). Some types of diagnoses (including learning disability, severe traumatic brain injury, psychosis, and depression), very advanced age (?80), and lowered IQ were associated with increased rates of false positive identifications, suggesting the need for some adjustments to cut-offs in these subgroups. Despite some previous reports of an inverted Stroop effect (i.e., color-naming worse than color interference) in noncredible subjects, individual Stroop word reading and color naming trials were much more effective in identifying response bias.     

Classification Accuracy and Predictive Ability of The Medical Symptom Validity Test's Dementia Profile and General Memory Impairment Profile

When assessing symptom validity in patients with dementia, traditional approaches may be inappropriate because neurological factors may cause altered performance. The Medical Symptom Validity Test has a Dementia Profile that explicitly recognizes this fact. We prospectively evaluated classification accuracy of the Dementia Profile for 52 referrals to a memory disorders clinic. The Dementia Profile correctly classified 36/52 patients. Sensitivity was 54.8%, specificity was 90.5%, positive predictive value was 89.5%, negative predictive value was 60.0%, and the likelihood ratio was 5.77. Of 31 patients with dementia, 11 did not fail symptom validity indices. When only considering patients who failed symptom validity indices, sensitivity of the Dementia Profile was 85%. Classification accuracy statistics are also provided for the Genuine Memory Impairment Profile.     

Assessing Effort: Differentiating Performance and Symptom Validity

The current study aimed to clarify the relationship among the constructs involved in neuropsychological assessment, including cognitive performance, symptom self-report, performance validity, and symptom validity. Participants consisted of 120 consecutively evaluated individuals from a veteran’s hospital with mixed referral sources. Measures included the Wechsler Adult Intelligence Scale-Fourth Edition Full Scale IQ (WAIS-IV FSIQ), California Verbal Learning Test-Second Edition (CVLT-II), Trail Making Test Part B (TMT-B), Test of Memory Malingering (TOMM), Medical Symptom Validity Test (MSVT), WAIS-IV Reliable Digit Span (RDS), Post-traumatic Check List-Military Version (PCL-M), MMPI-2 F scale, MMPI-2 Symptom Validity Scale (FBS), MMPI-2 Response Bias Scale (RBS), and the Postconcussive Symptom Questionnaire (PCSQ). Six different models were tested using confirmatory factor analysis (CFA) to determine the factor model describing the relationships between cognitive performance, symptom self-report, performance validity, and symptom validity. The strongest and most parsimonious model was a three-factor model in which cognitive performance, performance validity, and self-reported symptoms (including both standard and symptom validity measures) were separate factors. The findings suggest failure in one validity domain does not necessarily invalidate the other domain. Thus, performance validity and symptom validity should be evaluated separately.     

Confronting Patients About Insufficient Effort: The Impact on Subsequent Symptom Validity and Memory Performance

Symptom validity tests (SVTs) are commonly used to assess effort in neuropsychological evaluations. However, no empirical research or official guidelines exist about how clinicians should proceed if a patient produces a non-valid SVT result. The purpose of this study was to examine whether confronting patients immediately after scoring in a non-valid range on a SVT would have an impact on subsequent symptom validity and memory tests performance. Archival patient data for 507 adults with clinically definite multiple sclerosis (MS) (ages 18–76) were examined. All patients completed the Victoria Symptom Validity Test (VSVT), the Wechsler Memory Scale, 3rd edition (WMS III), and the Beck Depression Inventory, 2nd edition (BDI II). Although the majority (89%) of patients produced valid VSVT scores (the Valid group), 56 patients produced non-valid VSVT scores. Due to a change in clinical procedure, 28 of the 56 were confronted regarding their non-valid VSVT performances and were asked to complete the test a second time (the CONF group), while the remaining 28 proceeded with testing as usual following a non-valid score (the N-CONF group). Results showed that 68% of the CONF group produced valid VSVT scores on re-administration, as well as memory performances that were comparable to those of the Valid group. In contrast the N-CONF group produced memory scores that were significantly below the Valid group. This is the first study to provide empirical support for the effectiveness of intervention when patients exhibit inadequate effort on SVTs in clinical, non-forensic settings.     

Effort Test Failure: Toward a Predictive Model

Predictors of effort test failure were examined in an archival sample of 555 traumatically brain-injured (TBI) adults. Logistic regression models were used to examine whether compensation-seeking, injury-related, psychological, demographic, and cultural factors predicted effort test failure (ETF). ETF was significantly associated with compensation-seeking (OR?=?3.51, 95% CI [1.25, 9.79]), low education (OR:. 83 [.74, . 94]), self-reported mood disorder (OR: 5.53 [3.10, 9.85]), exaggerated displays of behavior (OR: 5.84 [2.15, 15.84]), psychotic illness (OR: 12.86 [3.21, 51.44]), being foreign-born (OR: 5.10 [2.35, 11.06]), having sustained a workplace accident (OR: 4.60 [2.40, 8.81]), and mild traumatic brain injury severity compared with very severe traumatic brain injury severity (OR: 0.37 [0.13, 0.995]). ETF was associated with a broader range of statistical predictors than has previously been identified and the relative importance of psychological and behavioral predictors of ETF was evident in the logistic regression model. Variables that might potentially extend the model of ETF are identified for future research efforts.     

Validation Of The Sentence Repetition Test As A Measure Of Suspect Effort

It has been suggested that the Sentence Repetition Test (SRT) could serve as an adequate embedded symptom valid ity measure identifying suspect effort during neuropsychological testing. However, very little research has examined sensitivity and specificity rates when using this measure in a variety of clinical settings. The SRT was administered to 1031 patients referred for neuropsychological assessment in outpatient, inpatient, and independent medical evaluation settings. These patients were diagnosed with a wide range of psychiatric, developmental, and neurological disorders. The results of this study reveal that the SRT is a valid measure of suspect effort for the vast majority of these patients (sensitivity = 56.8% and specificity = 95.8% in combined clinical settings). However, analyses also indicate that the SRT is not a valid effort measure for individuals with mental retardation or dementia due to specificity rates falling well below 90% for both groups. Furthermore, the validity of the SRT as an effort measure is questionable for individuals with English as a second language, with a verbal learning disability, with a left cerebrovascular accident, or with an expressive-receptive language disorder due to small sample sizes or borderline specificity rates. Sensitivity, specificity, positive predictive accuracy, and negative predictive accuracy rates are provided for varying cutoff scores in inpatient, outpatient, and IME settings. The results of this extensive study confirm that the SRT can be a useful measure in detecting suspect effort in neuropsychological testing while also providing valuable clinical information.     

Lorazepam Effects on Word Memory Test Performance: A Randomized,Double-Blind,Placebo-Controlled,Crossover Trial

The Word Memory Test (WMT) is a common measure of symptom validity. To investigate the effects of acute benzodiazepines on WMT scores, oral lorazepam 2?mg (LOR) and placebo were administered 1 week apart in a randomized, double-blind, placebo-controlled, crossover study. A total of 28 participants completed the study and were administered the WMT during each drug condition. Within-participant comparisons of LOR vs placebo revealed significant LOR effects for Immediate Recognition (p?=?.007) and Consistency (p?=?.019), but not Delayed Recognition (p?=?.085). Significant LOR effects were present for Reaction Time Measures (Immediate Recognition RT, p?=?.013; Delayed Recognition RT, p?=?.001; Multiple Choice RT, p?=?.011) and Delayed Memory scores (Multiple Choice, p?=?.007; Paired Associates, p?=?.029; Free Recall, p?=?.001). A pattern similar to crossover results was detected for LOR vs placebo between-group differences for initial test assessment scores. When examined using publisher recommended cut scores for the principal WMT measures, there were six participants failing the WMT during initial LOR testing; all six subsequently performed in the normal range upon retesting with placebo. One participant failed WMT during placebo and obtained passing scores during LOR. These data indicate that multiple WMT measures may be affected by acute LOR dosing, and provide additional evidence that potential latent variables and their effects on both SVT performance and cognitive function should be part of the clinical decision-making process.     

Test of Memory Malingering: Cutoff Scores for Psychometrically Defined Malingering Groups in a Military Sample

This research examined cutoffs for the Test of Memory Malingering (TOMM) in a military sample composed primarily of mTBI patients. The results are consistent with previous research and provide additional evidence that cutoffs higher than those originally recommended for the TOMM can produce excellent classification and diagnostic statistics when a psychometrically defined non-malingering group is compared with three psychometrically defined malingering groups: Probable, Probable to Definite, and Definite Malingering. The groups were formed based on the number of symptom and performance validity tests passed or failed. Cutoffs that were 4–5 points higher for Trial 2 and the Retention Trial than originally recommended produced very low false positive rates (.0 to .06) and excellent positive predictive values (.75 to 1.00) for a base rate of malingering commonly found in TBI patients. Positive likelihood ratios were all above 10 for these two trials indicating excellent ability to rule in malingering. A range of cutoffs for Trial 1 were also examined, and classification and diagnostic statistics are presented for cutoffs ranging from 40 to 44 with results similar to the other TOMM trials.     

TOMM Trial 1 as a performance validity indicator in a criminal forensic sample

Objective: To determine the effectiveness of the Test of Memory Malingering Trial 1 (TOMM1) as a freestanding Performance Validity Test (PVT) as compared to the full TOMM in a criminal forensic sample.Method: Participants included 119 evaluees in a Midwestern forensic hospital. Criterion groups were formed based on passing/failing scores on other freestanding PVTs. This resulted in three groups: +MND (Malingered Neurocognitive Dysfunction), who failed two or more freestanding PVTs; possible MND (pMND), who failed one freestanding PVT; and –MND, who failed no other freestanding PVTs. All three groups were compared initially, but only +MND and –MND groups were retained for final analyses. TOMM1 performance was compared to standard TOMM performance using Receiver Operating Characteristic (ROC) analyses.Results: TOMM1 was highly predictive of the standard TOMM decision rules (AUC = .92). Overall accuracy rate for TOMM1 predicting failure on 2 PVTs was quite robust as well (AUC = .80), and TOMM1 ≤ 39 provided acceptable diagnostic statistics (Sensitivity = .68, Specificity = .89). These results were essentially no different from the standard TOMM accuracy statistics. In addition, by adjusting for those strongly suspected of being inaccurately placed into the ?MND group (e.g. false negatives), TOMM1 diagnostics slightly improved (AUC = .84) at a TOMM1 ≤ 40 (sensitivity = .71, specificity = .94).Conclusions: Results support use of TOMM1 in a criminal forensic setting where accuracy, shorter evaluation times, and more efficient use of resources are often critical in informing legal decision-making.     

Classification Accuracy of Multiple Visual Spatial Measures in the Detection of Suspect Effort

A wide variety of cognitive measures, particularly memory measures, have been studied for their ability to detect suspect effort, or biased responding on neuropsychological assessment instruments. However, visual spatial measures have received less attention. The purpose of this study was to evaluate the classification accuracy of several commonly used visual spatial measures, including the Judgment of Line Orientation Test, the Benton Facial Recognition Test, the Hooper Visual Organization Test, and the Rey Complex Figure Test-Copy and Recognition trials. Participants included 491 consecutive referrals who participated in a comprehensive neuropsychological assessment and met study criteria. Participants were divided into two groups identified as either unbiased responding (UR, N?=?415) or biased responding (BR, N?=?30) based on their performance on two measures of effort. The remaining participants (N?=?46) had discrepant performance on the symptom validity measures and were excluded from further analysis. The groups differed significantly on all measures. Additionally, receiver operating characteristic (ROC) analysis indicated all of the measures had acceptable classification accuracy, but a measure combining scores from all of the measures had excellent classification accuracy. Results indicated that various cut-off scores on the measures could be used depending on the context of the evaluation. Suggested cut-off scores for the measures had sensitivity levels of approximately 32–46%, when specificity was at least 87%. When combined, the measures suggested cut-off scores had sensitivity increase to 57% while maintaining the same level of specificity (87%). The results were discussed in the context of research advocating the use of multiple measures of effort.     

PTSD and Cognitive Functioning: Importance of Including Performance Validity Testing

Many studies have observed an association between post-traumatic stress disorder (PTSD) and cognitive deficits across several domains including memory, attention, and executive functioning. The inclusion of response bias measures in these studies, however, remains largely unaddressed. The purpose of this study was to identify possible cognitive impairments correlated with PTSD in returning OEF/OIF/OND veterans after excluding individuals failing a well-validated performance validity test. Participants included 126 men and 8 women with a history of mild traumatic brain injury (TBI) referred for a comprehensive neuropsychological evaluation as part of a consortium of five Veterans Affairs hospitals. The PTSD CheckList (PCL) and Word Memory Test (WMT) were used to establish symptoms of PTSD and invalid performance, respectively. Groups were categorized as follows: Control (PCL < 50, pass WMT), PTSD-pass (PCL ≥ 50, pass WMT), and PTSD-fail (PCL ≥ 50, fail WMT). As hypothesized, failure on the WMT was associated with significantly poorer performance on almost all cognitive tests administered; however, no significant differences were detected between individuals with and without PTSD symptoms after separating out veterans failing the WMT. These findings highlight the importance of assessing respondent validity in future research examining cognitive functioning in psychiatric illness and warrant further consideration of prior studies reporting PTSD-associated cognitive deficits.     

Comparison of Credible Patients of Very Low Intelligence and Non-Credible Patients on Neurocognitive Performance Validity Indicators

The purpose of this archival study was to identify performance validity tests (PVTs) and standard IQ and neurocognitive test scores, which singly or in combination, differentiate credible patients of low IQ (FSIQ ≤ 75; n = 55) from non-credible patients. We compared the credible participants against a sample of 74 non-credible patients who appeared to have been attempting to feign low intelligence specifically (FSIQ ≤ 75), as well as a larger non-credible sample (n = 383) unselected for IQ. The entire non-credible group scored significantly higher than the credible participants on measures of verbal crystallized intelligence/semantic memory and manipulation of overlearned information, while the credible group performed significantly better on many processing speed and memory tests. Additionally, credible women showed faster finger-tapping speeds than non-credible women. The credible group also scored significantly higher than the non-credible subgroup with low IQ scores on measures of attention, visual perceptual/spatial tasks, processing speed, verbal learning/list learning, and visual memory, and credible women continued to outperform non-credible women on finger tapping. When cut-offs were selected to maintain approximately 90% specificity in the credible group, sensitivity rates were highest for verbal and visual memory measures (i.e., TOMM trials 1 and 2; Warrington Words correct and time; Rey Word Recognition Test total; RAVLT Effort Equation, Trial 5, total across learning trials, short delay, recognition, and RAVLT/RO discriminant function; and Digit Symbol recognition), followed by select attentional PVT scores (i.e., b Test omissions and time to recite four digits forward). When failure rates were tabulated across seven most sensitive scores, a cut-off of ≥ 2 failures was associated with 85.4% specificity and 85.7% sensitivity, while a cut-off of ≥ 3 failures resulted in 95.1% specificity and 66.0% sensitivity. Results are discussed in light of extant literature and directions for future research.     

The Effects of Coaching on the Verbal and Nonverbal Medical Symptom Validity Tests

Evaluation of resistance to coaching is an important step in the validation of symptom validity tests (SVTs) for clinical use in neuropsychological evaluations. In the present study coaching effects were evaluated for two recently developed SVTs, the Medical Symptom Validity Test (MSVT) and Nonverbal Medical Symptom Validity Test (NVMSVT) as compared with a well-validated existing SVT, the Test of Memory Malingering (TOMM). This study used a simulation design that included 103 healthy younger study volunteers who were randomly assigned into one of four conditions: Symptom Coaching, Test Coaching, Combined Coaching, or Best Effort Control. Specificity for all SVTs was excellent (96–100%). Test Coaching, either alone or combined with Symptom Coaching, was more effective than Symptom Coaching alone in producing raw scores suggestive of “better” effort for all SVTs. However, there were only modest declines in the obtained sensitivity, which remained above 80% for all SVTs. These results provide empirical support for the classification accuracy of the MSVT and NVMSVT, even when challenged with combined coaching interventions. However, further validation using known-groups designs and clinical samples is needed.     

Effects of Severity of Dementia on a Symptom Validity Measure

While prior research has shown symptom validity tests (SVTs) to have clinical utility with cognitively impaired individuals, these studies typically excluded those diagnosed with mild cognitive impairment (MCI) and dementia. The purpose of this study was to determine the utility of an SVT, the Test of Memory Malingering (TOMM), in those with MCI and moderate to severe dementia. Participants included 30 cognitively intact individuals (Control Group), 28 diagnosed with MCI (MCI Group), and 31 diagnosed with moderate to severe dementia (Moderate-Severe Group). The range of Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) Total Standard Scores were then calculated for each group and all participants were administered the TOMM. Analysis of covariance (ANCOVA) revealed no significant differences on TOMM Trail 2 scores between the Control Group and the MCI Group. While all group means were above established cut-off scores, approximately 20% of participants in the Moderate-Severe Group failed the TOMM according to established criteria. Results suggest that the TOMM may be an appropriate test of effort in older adults diagnosed with MCI, but is not recommended for assessing potential malingering in those with at least moderate to severe dementia.     

A Model to Approaching and Providing Feedback to Patients Regarding Invalid Test Performance in Clinical Neuropsychological Evaluations

The use of symptom validity assessment has become commonplace in clinical neuropsychological evaluations. However, clinicians often struggle with how to provide patients with feedback regarding invalid responding or effort, because of the sensitive nature of the information that must be conveyed. A conceptual framework for providing such feedback is outlined in clinical neuropsychological evaluations, and recommendations for how to handle complaints are offered. Our feedback model is not meant to apply to individuals referred by attorneys or other non-clinical third parties (e.g., independent medical examination companies).     

Rates of Below-Chance Performance in Forced-Choice Symptom Validity Tests

The rates of significantly below-chance results on three neuropsychological symptom validity tests (SVTs) including the Portland Digit Recognition Test (PDRT), Test of Memory Malingering (TOMM), and Word Memory Test (WMT) were compared in a private practice forensic sample of 1032 examinees with alleged mild traumatic brain injury, moderate to severe traumatic brain injury, alleged toxic exposure, and reported chronic pain. The PDRT and WMT were equivalent to one another in the rates of below-chance results, with both yielding more frequent below-chance results than the TOMM. Seemingly more difficult sections of the PDRT and WMT had higher yields than seemingly easier sections. Multiple SVTs were more likely to yield below-chance results than a single test, supporting the use of multiple SVTs in forensic neuropsychological evaluations.     

Detecting Malingered Pain-Related Disability: Classification Accuracy of the Portland Digit Recognition Test

This study used criterion groups validation to determine the classification accuracy of the Portland Digit Recognition Test (PDRT) at a range of cutting scores in chronic pain patients undergoing psychological evaluation ( n = 318), college student simulators ( n = 29), and patients with brain damage ( n = 120). PDRT scores decreased and failure rates increased as a function of greater independent evidence of intentional underperformance. There were no differences between patients classified as malingering and college student simulators. The PDRT detected from 33% to nearly 60% of malingering chronic pain patients, depending on the cutoff used. False positive error rates ranged from 3% to 6%. Scores higher than the original cutoffs may be interpreted as indicating negative response bias in patients with pain, increasing the usefulness and facilitating the clinical application of the PDRT in the detection of malingering in pain.     

The Structured Inventory of Malingered Symptomatology (SIMS): A Systematic Review and Meta-Analysis

We meta-analytically reviewed studies that used the Structured Inventory of Malingered Symptomatology (SIMS) to detect feigned psychopathology. We present weighted mean diagnostic accuracy and predictive power indices in various populations, based on 31 studies, including 61 subsamples and 4009 SIMS protocols. In addition, we provide normative data of patients, claimants, defendants, nonclinical adults, and various experimental feigners, based on 41 studies, including 125 subsamples and 4810 SIMS protocols. We conclude that the SIMS (1) is able to differentiate well between instructed feigners and honest responders; (2) generates heightened scores in groups that are known to have a raised prevalence of feigning (e.g., offenders who claim crime-related amnesia); (3) may overestimate feigning in patients who suffer from schizophrenia, intellectual disability, or psychogenic non-epileptic seizures; and (4) is fairly robust against coaching. The diagnostic power of the traditional cut scores of the SIMS (i.e., > 14 and > 16) is not so much limited by their sensitivity—which is satisfactory—but rather by their substandard specificity. This, however, can be worked around by combining the SIMS with other symptom validity measures and by raising the cut score, although the latter solution sacrifices sensitivity for specificity.     

Relationship between Tomm performance and PAI validity scales in a mixed clinical sample

This study addressed the relationship between Personality Assessment Inventory (PAI) validity indicators and cognitive effort measures on the Test of Memory Malingering (TOMM). Significant correlations were found between TOMM and some PAI validity scales. Factor analysis results found separate cognitive and personality components, but the Negative Impression Management (NIM) scale, a measure of response bias, had factor loadings on both the cognitive and the personality components. Follow-up hierarchical multiple regression and t-test analysis generally confirmed this result, and found that NIM and the Infrequency (INF) scale have significant relationships with the TOMM. The results indicate that individuals with elevations on the PAI's INF and NIM scales often display decreased cognitive effort on the TOMM. The current results support the hypothesis that personality assessment validity indicators have a modest but significant relationship with poor cognitive effort.     

Evaluation of the Appropriateness of Multiple Symptom Validity Indices in Psychotic and Non-Psychotic Psychiatric Populations

Although it is recognized that significant cognitive deficits are inherent in many psychiatric disorders, there is minimal research on whether the deficits can cause a failing score on symptom validity tests (SVTs). The performances of 104 and 178 patients with psychotic disorders and non-psychotic psychiatric disorders, respectively, on seven SVTs were examined. Analyses indicate that most of these SVTs have specificity rates of 90% or better for both clinical groups. Further, only 7% of patients in the psychotic group and 5% of patients in the non-psychotic psychiatric group produced false-positive classifications based on malingering criteria similar to those suggested by Slick et al. (i.e., failure of two or more SVTs or failure of one SVT at statistically significantly worse than chance rates). Consequently this research indicates that psychiatric disorders typically do not adversely affect SVT performance.