The Performance of Individuals with Mental Retardation on Cognitive Tests Assessing Effort and Motivation

The purpose of this study was to determine which tests of effort and motivation would be appropriate for use with patients with mental retardation when feigning of cognitive deficits is suspected. The seven measures evaluated included the WMS-III Rarely Missed Index Test, forced-choice recognition portion of the California Verbal Learning Test-II, Reliable Digit Span test, Rey 15-Item Test, Rey Dot Counting Test, the Rey 15-Item Test with Recognition Trial, and the Vocabulary (V)-Digit Span (DS) difference score. Results indicated that the forced-choice portion of the CVLT-II, the V-DS difference score, and the Rarely Missed Index Test from the WMS-III might be appropriate for use with this population with passing rates of 89%, 98%, and 91% respectively.     

Dementia and effort test performance

Research on the performance of patients with dementia on tests of effort is particularly limited. We examined archival data from 214 non-litigating patients with dementia on 18 effort indices derived from 12 tests (WAIS-III/WAIS-R Digit Span and Vocabulary, Dot Counting Test, Warrington Recognition Memory Test-Words, WMS-III Logical Memory, Rey Word Recognition Memory Test, Finger Tapping, b-Test, Rey 15-Item, Test of Memory Malingering, Rey Auditory Verbal Learning Test, and Rey Complex Figure Test). Results indicated that recommended cut-offs for Digit Span indicators (Vocabulary Minus Digit Span and four-digit forward span time score) provided > or =90% specificity across participants, while the majority of other effort tests displayed specificities in the 30-70% range. Analyses of test specificity as a function of Mini Mental Status Examination (MMSE) score and specific dementia diagnosis are provided, as well as adjustments to cut-offs to maintain specificity where feasible.     

Examination Of Various Wms-Iii Logical Memory Scores In The Assessment Of Response Bias

The assessment of response validity during neuropsychological evaluation is an integral part of the testing process. Research has increasingly focused on the use of “embedded” effort measures (derived from standard neuropsychological tasks) because they do not require additional administration time and are less likely to be identified as effort indicators by test takers because of their primary focus as measures of cognitive function. The current study examined the clinical utility of various WMS-III Logical Memory scores in detecting response bias, as well as the Rarely Missed Index, an embedded effort indicator derived from the WMS-III Logical Memory Delayed Recognition subtest. The Rarely Missed Index cut-off only identified 24.1% of 63 non-credible participants (at ≥90% specificity in 125 credible patients), and cut-offs for other Logical Memory variables were in fact found to be more sensitive to non-credible performance. A new indicator, consisting of the weighted combination of the two most sensitive Logical Memory subtest scores (Logical Memory II raw score and Logical Memory Delayed Recognition raw score), was associated with 53% to 60% sensitivity, and thus may be an effective adjunct when utilized in conjunction with other validated effort indicators and collateral information in identifying non-credible performance.     

Brief Screening Indexes for Malingering: A Confirmation of Vocabulary Minus Digit Span from the WAIS-III and the Rarely Missed Index from the WMS-III

This study investigated the specificity of empirically derived screening measures for the detection of symptom exaggeration in persons with a diagnosis of alcohol abuse(n?=?30), polysubstance abuse(n?=?43), or head trauma(n?=?27). The first measure evaluated was Vocabulary(V)minus Digit Span(DS)(Mittenberg, Theroux-Fichera, Zielinski, & Heilbronner, 1995); the second measure was the Rarely Missed Index(RMI)for the WMS-III Logical Memory subtest(Killgore & DellaPietra, 2000). V-DS misclassified 0%of individuals in the alcohol abuse group, 2%of those in the polysubstance abuse group, and 0%of head injury cases. RMI misclassification rates were 3%, 5%, and 7%for the alcohol abuse, polysubstance abuse, and head injury groups, respectively. Overall accuracy rates were 99%for V-DS and 95%for RMI.     

Brief screening indexes for malingering: A confirmation of Vocabulary minus Digit Span from the WAIS-III and the Rarely Missed Index from the WMS-III

This study investigated the specificity of empirically derived screening measures for the detection of symptom exaggeration in persons with a diagnosis of alcohol abuse (n = 30), polysubstance abuse (n = 43), or head trauma (n = 27). The first measure evaluated was Vocabulary (V) minus Digit Span (DS) (Mittenberg, Theroux-Fichera, Zielinski, & Heilbronner, 1995); the second measure was the Rarely Missed Index (RMI) for the WMS-III Logical Memory subtest (Killgore & Della-Pietra, 2000). V-DS misclassified 0% of individuals in the alcohol abuse group, 2% of those in the polysubstance abuse group, and 0% of head injury cases. RMI misclassification rates were 3%, 5%, and 7% for the alcohol abuse, polysubstance abuse, and head injury groups, respectively. Overall accuracy rates were 99% for V-DS and 95% for RMI.     

Relationships Between Eight Measures of Suspect Effort

Previous studies have recommended that multiple measures be employed concurrently to provide converging evidence regarding the presence of suspect effort during neuropsychological assessment. However, if the tests are highly correlated they do not represent independent sources of information. To date, no study has examined correspondence between effort tests. The present study assessed the relationships between eight measures which can be used to assess effort (Rey 15-item, Rey Dot Counting Test, Rey Word Recognition Test, RAVLT recognition trial, Rey-Osterrieth Complex Figure Test effort equation, Digit Span, Warrington Recognition Memory Test-Words, and “b” Test) in a sample of 105 patients in litigation or attempting to obtain/maintain disability compensation and who displayed noncredible symptoms based on psychometric performance and behavioral criteria. Modest to moderate correlations were observed between test summary scores with only two measures sharing more than 50% score variance (Digit Span and Dot Counting). Moderate correlations were also observed between individual test scores reflecting indices of response time, free recall, recognition, and false positive errors, providing possible evidence that patients may use specific strategies when producing noncredible performances. Overall the results suggest that the use of these various tests generally provides nonredundant data regarding patient credibility in neuropsychological evaluations.     

Relationships between eight measures of suspect effort

Previous studies have recommended that multiple measures be employed concurrently to provide converging evidence regarding the presence of suspect effort during neuropsychological assessment. However, if the tests are highly correlated they do not represent independent sources of information. To date, no study has examined correspondence between effort tests. The present study assessed the relationships between eight measures which can be used to assess effort (Rey 15-item, Rey Dot Counting Test, Rey Word Recognition Test, RAVLT recognition trial, Rey-Osterrieth Complex Figure Test effort equation, Digit Span, Warrington Recognition Memory Test-Words, and "b" Test) in a sample of 105 patients in litigation or attempting to obtain/maintain disability compensation and who displayed noncredible symptoms based on psychometric performance and behavioral criteria. Modest to moderate correlations were observed between test summary scores with only two measures sharing more than 50% score variance (Digit Span and Dot Counting). Moderate correlations were also observed between individual test scores reflecting indices of response time, free recall, recognition, and false positive errors, providing possible evidence that patients may use specific strategies when producing noncredible performances. Overall the results suggest that the use of these various tests generally provides nonredundant data regarding patient credibility in neuropsychological evaluations.     

The use of the Rey 15-Item Test and recognition trial to evaluate noncredible effort after pediatric mild traumatic brain injury

The Rey 15-Item Test (FIT) is a performance validity test commonly used in adult neuropsychological assessment. FIT classification statistics across studies have been variable, so a recognition trial was created to enhance the measure (Boone, K. B., Salazar, X., Lu, P., Warner-Chacon, K., & Razani, J. (2002). The Rey 15-Item recognition trial: A technique to enhance sensitivity of the Rey 15-Item Memorization Test. Journal of Clinical and Experimental Neuropsychology, 24(5), 561–573.). The current study assessed the utility of the FIT and recognition trial in a pediatric mild traumatic brain injury sample (N = 319, M = 14.57 years). All participants were administered the FIT and recognition trial as part of an abbreviated clinical neuropsychological evaluation. Failure on the Medical Symptom Validity Test was used as the criterion for noncredible effort. Fifteen percent of the sample met the criterion. The traditional adult cutoff score of <9 on the FIT recall trial yielded excellent specificity (98%), but very poor sensitivity (12%). When the recognition trial was utilized, a total score of <26 resulted in the best combined cutoff score (sensitivity = 55%, specificity = 91%). Results indicate that the FIT with recognition trial may be useful in the assessment of noncredible effort with children and adolescents, at least among relatively high-functioning populations.     

Inability of the Rarely Missed Index to identify simulated malingering under more realistic assessment conditions

The Wechsler Memory Scale-III (WMS-III) Rarely Missed Index (RMI) was developed by Killgore and DellaPietra (1999b, 2000b, 2001) to detect malingering. Its development was based on item-response biases in the Logical Memory Delayed Recognition (LMDR) items of the WMS-III (Killgore & DellaPietra, 1999a, 2000a). We completed replication studies using undergraduates, analog malingerers, and patients. In Experiment 1, 100 undergraduates naive to the WMS-III Logical Memory stories were administered the LMDR, and item-response biases were explored. In Experiment 2, 49 undergraduates feigned brain injury on cognitive and symptom validity tests. Data were also extracted from 83 traumatic brain injury (TBI), dementia, psychiatric/pain, stroke, and other patient files. Neither the previously reported item-response biases nor the diagnostic utility of the RMI in identifying simulated malingerers were replicated.     

Using the WMS-III to Detect Malingering: Empirical Validation of the Rarely Missed Index (RMI)

To detect malingering during memory assessment, we evaluated item response biases to the Logical Memory Delayed Recognition (LMDR) subtest of the WMS-III. In a sample of 50 healthy volunteers who were completely naïve to the content of the Logical Memory stories, 6 LMDR items were correctly endorsed above chance probabilities. These 6 rarely missed items significantly discriminated 51 patients with neurological impairment from 36 volunteers who attempted to feign head injury and poor cognitive performance. A weighted combination of the 6 items was summed to form a single Rarely Missed Index (RMI). The RMI accurately classified over 98% of participants and demonstrated high sensitivity (97%) and specificity (100%) in discriminating between analog malingerers and patients. Because the RMI is calculated directly from the LMDR items, it has the advantage of requiring no additional administration time or materials, and thus may serve as a quick screen for dissimulation that can be obtained without additional testing.     

Using the WMS-III to detect malingering: empirical validation of the rarely missed index (RMI)

To detect malingering during memory assessment, we evaluated item response biases to the Logical Memory Delayed Recognition (LMDR) subtest of the WMS-III. In a sample of 50 healthy volunteers who were completely na?ve to the content of the Logical Memory stories, 6 LMDR items were correctly endorsed above chance probabilities. These 6 rarely missed items significantly discriminated 51 patients with neurological impairment from 36 volunteers who attempted to feign head injury and poor cognitive performance. A weighted combination of the 6 items was summed to form a single Rarely Missed Index (RMI). The RMI accurately classified over 98% of participants and demonstrated high sensitivity (97%) and specificity (100%) in discriminating between analog malingerers and patients. Because the RMI is calculated directly from the LMDR items, it has the advantage of requiring no additional administration time or materials, and thus may serve as a quick screen for dissimulation that can be obtained without additional testing.     

The Rey 15-item memory test and Spanish-speaking older adults

The Rey 15-Item Memory Test (Rey-15) is a standard instrument frequently employed to assess suspect effort/motivation in English-speaking populations. The objective of the current study was to examine the influence of socio-demographic variables on this measure and provide normative data for use with Spanish speakers. The performance of 130 primarily Spanish-speaking, cognitively intact, older adults (ages 50-69) on six Rey-15 scoring systems and six embedded measures of suboptimal performance was examined. Approximately 8% of the sample scored below the recommended cut-off of 9 on the Rey-15. The lowest recall score of 6 was also the minimum score obtained on the recognition trial. Additionally, scores on the alternative Rey-15 scoring methods and the embedded measures of suboptimal performance were lower in comparison to the normative data presently utilized with English speakers, yet comparable across the examined measures. Basic mental status and education level were significant predictors of Rey-15 performance; however, results indicate that these variables may share a close relationship with socio-demographic characteristics such as acculturation level and years of U.S. residency. Preliminary normative data on the Rey-15 for primarily Spanish-speaking older adults, stratified by education, is provided in conjunction with a recommendation for the use of the recognition trial when interpreting results.     

Use of specific malingering measures in a Spanish sample

There are an increasing number of tests available for detecting malingering. However, these tests have not been validated for using in Spanish speakers. The purpose of this study is to explore the value of three specific malingering tests in the Spanish population. This study used a known-groups design, together with a group of analog students. The results show that both the Victoria Symptom Validity Test and the b Test can be used to detect malingering in Spanish population. However, some restrictions must be applied when the Rey 15-Item Test is administered and interpreted.     

The Rey 15-Item Memory Test and Spanish-Speaking Older Adults

The Rey 15-Item Memory Test (Rey-15) is a standard instrument frequently employed to assess suspect effort/motivation in English-speaking populations. The objective of the current study was to examine the influence of socio-demographic variables on this measure and provide normative data for use with Spanish speakers. The performance of 130 primarily Spanish-speaking, cognitively intact, older adults (ages 50–69) on six Rey-15 scoring systems and six embedded measures of suboptimal performance was examined. Approximately 8% of the sample scored below the recommended cut-off of 9 on the Rey-15. The lowest recall score of 6 was also the minimum score obtained on the recognition trial. Additionally, scores on the alternative Rey-15 scoring methods and the embedded measures of suboptimal performance were lower in comparison to the normative data presently utilized with English speakers, yet comparable across the examined measures. Basic mental status and education level were significant predictors of Rey-15 performance; however, results indicate that these variables may share a close relationship with socio-demographic characteristics such as acculturation level and years of U.S. residency. Preliminary normative data on the Rey-15 for primarily Spanish-speaking older adults, stratified by education, is provided in conjunction with a recommendation for the use of the recognition trial when interpreting results.     

Comparison of the effectiveness of two versions of the Rey memory test in discriminating between actual and simulated memory impairment, with and without the addition of a standard memory test

The superiority of the modified 16-item Rey memory test over the original 15-item test has yet to be established in relation to the detection of feigned memory deficits. This study compares the effectiveness of these two versions of the test in combination with a standard memory test. Sixty-four participants, equally divided into "controls" and "simulators", and 32 "memory-impaired" volunteers completed the WAIS-III Digit Span test and either the 15-item or 16-item Rey test. The median scores of the simulators were worse than those of both the controls and the memory-impaired subjects on two types of score on the 16-item test compared to just one on the 15-item test. However, combining the total score from the 16-item test with the digits forward span accurately detected 63% of simulators (with only 6% of nonsimulators wrongly classified as simulators). This performance was better than that obtained from any of the test scores taken singly or from any other combination of a digit span score with a Rey score. It is concluded that the addition of a standard memory test to the Rey memory test improves the detection of individuals feigning memory impairment.     

Performance validity in undergraduate research participants: a comparison of failure rates across tests and cutoffs

Objective: This study compared failure rates on performance validity tests (PVTs) across liberal and conservative cutoffs in a sample of undergraduate students participating in academic research.Method: Participants (n = 120) were administered four free-standing PVTs (Test of Memory Malingering, Word Memory Test, Rey 15-Item Test, Hiscock Forced-Choice Procedure) and three embedded PVTs (Digit Span, letter and category fluency). Participants also reported their perceived level of effort during testing.Results: At liberal cutoffs, 36.7% of the sample failed ≥1 PVTs, 6.7% failed ≥2, and .8% failed 3. At conservative cutoffs, 18.3% of the sample failed ≥1 PVTs, 2.5% failed ≥2, and .8% failed 3. Participants were 3 to 5 times more likely to fail embedded (15.8–30.8%) compared to free-standing PVTs (3.3–10.0%). There was no significant difference in failure rates between native and non-native English speaking participants at either liberal or conservative cutoffs. Additionally, there was no relation between self-reported effort and PVT failure rates.Conclusions: Although PVT failure rates varied as a function of PVTs and cutoffs, between a third and a fifth of the sample failed ≥1 PVTs, consistent with high initial estimates of invalid performance in this population. Embedded PVTs had notably higher failure rates than free-standing PVTs. Assuming optimal effort in research using students as participants without a formal assessment of performance validity introduces a potentially significant confound in the study design.     

Significant discrepancies between immediate and delayed WMS-III indices are rare in temporal lobe epilepsy patients

Initial factor analysis of Wechsler Memory Scale-Third Edition (WMS-III) standardization data resulted in a five-factor model that included separate immediate (IMM) and delayed (DEL) memory factors for auditory (AUD) and visual (VIS) memory. However, recent factor analyses that revealed three factors--AUD, VIS, and working memory--were discovered to be more accurate. Continued use of separate WMS-III IMM and DEL indices has been recommended because future studies with clinical groups might support this distinction. Data from this investigation of 88 temporal lobe epilepsy (TLE) patients suggested separate IMM and DEL WMS-III indices are not necessary in this population. Results were as follows: 1) There was no significant difference between the IMM and DEL indices for AUD or VIS memory. 2) The percentage of individuals with a significant difference between the IMM and DEL indices (IMM minus DEL >11 or <-12) was similar for our TLE group and the standardization sample. 3) The mean percent retention standard score (SS) was at the average level for the four memory subtests, and for each subtest only about 10% of the TLE patients had impaired scores. 4) Hippocampal volumes correlated significantly with both IMM and DEL indices. These data suggest that combining the appropriate WMS-III IMM and DEL memory subtest scores to form AUD and VIS memory indices is appropriate for TLE patients.     

Discordance between WMS-III Word Lists subtest and RAVLT scores in older patients: four case examples

To examine the adequacy of the WMS-III in the identification of memory deficits in older individuals, we administered the Word Lists subtest, along with a comparison measure, the Rey Auditory Verbal Learning Test (RAVLT), to four patients referred for assessment of dementia. Although the number of words recalled by the patients on the delayed trials of the Word Lists subtest and RAVLT were nearly identical, norms for the Word Lists subtest indicated that memory performance was within normal limits (i.e., low average or higher), while the RAVLT norms accurately characterized performance as impaired. We suspect that normative data for the WMS-III were compromised by inadvertent inclusion of older subjects with dementia or mild cognitive impairment. The findings from our cases call into question the use of the WMS-III Word Lists for older patients.     

WMS-III performance in patients with temporal lobe epilepsy: group differences and individual classification.

The utility of the WMS-III in detecting lateralized impairment was examined in a large sample of patients with temporal lobe epilepsy. Methods of analysis included evaluation of group means on the various indexes and subtest scores, the use of ROC curves, and an examination of Auditory-Visual Index discrepancy scores. In addition, performance on immediate and delayed indexes in the auditory and the visual modality was compared within each group. Of the WMS-III scores, the Auditory-Visual Delayed Index difference score appeared most sensitive to side of temporal dysfunction, although patient classification rates were not within an acceptable range to have clinical utility. The ability to predict laterality based on statistically significant index score differences was particularly weak for those with left temporal dysfunction. The use of unusually large discrepancies led to improved prediction, however, the rarity of such scores in this population limits their usefulness. Although the utility of the WMS-III in detecting laterality may be limited in preoperative cases, the WMS-III may still hold considerable promise as a measure of memory in documenting baseline performance and in detecting those that may be at risk following surgery.     

The Rey 15-item recognition trial: a technique to enhance sensitivity of the Rey 15-item memorization test

Numerous publications on the Rey 15-item Memorization Test have cited limitations primarily in test sensitivity, as well as to some extent in specificity. In the current study, 49 patients with suspect effort, 36 neuropsychology clinic patients not in litigation or attempting to secure disability, 33 learning disabled college students, and 60 normal controls were administered the Rey Test in standard format followed by a recognition trial. A free recall score <9 was found to have excellent specificity (97-100%), although sensitivity was modest (47%). However, use of a combined recall and recognition score (i.e., free recall + [recognition - false positives] <20) substantially increased sensitivity (71%) while maintaining high specificity (> or=92%).