Low Specificity

If psychologists rely on the SIMS Manual's recommended cut score of 14, the SIMS has substandard specificity.^[2] For instance, SIMS specificity calculated by Richard Rogers et al. for a total SIMS cutoff score of over 14 was 0.28:

With this inpatient sample, the recommended SIMS total cut score of >14 functioned poorly, because most genuine responders also exceeded this cut score (i.e., specificity = .28). Even at a 25% base rate, more than two thirds (positive predictive power, PPP = .70) of those identified will be genuine responders. At least for these inpatients, a much higher SIMS total cut score (> 44) is required to achieve a very high specificity.^[3]

Low specificity hinders the test's ability to accurately distinguish legitimate patients from malingerers. Stating that an individual is malingering can cause iatrogenic harm to patients if they are actually not exaggerating or feigning. Such iatrogenic harm may consist in delaying or denying medical attention, therapies, or insurance benefits. In the U.S. military, malingering is a court-martial offense under the Uniform Code of Military Justice.

Investigations of the Validity of SIMS Item Content

Page 15 of the SIMS manual^[4] informs the readers in an unsubstantiated manner that the SIMS items are descriptive of  “atypical, improbable, inconsistent, or illogical symptoms” that would be “highly atypical in patients with genuine psychiatric or cognitive disorders …”.  In fact, content analyses via ratings by teams of doctoral level clinicians with more than 35 years of experience each in clinical psychology or psychiatry indicated that the SIMS scales contain no items with the reasonable capacity to differentiate legitimate patients from malingerers^[5,6,7,8].  

The SIMS has been widely used by psychologists contracted by car insurance companies to evaluate insurance claims of persons injured in motor vehicle accidents (MVAs).  More than 50% of SIMS items conceptually overlap with those of the Rivermead Post-Concussion Symptoms scale^[9] or of those from the Post-MVA Neurological Symptoms scale,^[10] i.e., with post-concussive and whiplash symptoms legitimately reported by post-MVA patients(and also by some injured war veterans or by some persons injured in industrial accidents: ^[11]  the more of their medical or psychological symptoms these patients honestly report, the more likely are they falsely classified as malingerers.^[11]

Archival data suggested that more than 70% of legitimate claims such as those for payments for therapies are thus falsely rejected or unduly delayed (see Gutierrez et al., page 16)^[12] with iatrogenic consequences for the patients.

Meta-analytic investigation of criterion validity of the SIMS and its scales

A recent meta-analytic study^[13] showed that the lowest SIMS scores are obtained in a group of normal volunteers, somewhat higher SIMS scores are obtained from persons with mild symptoms from car accidents, and the highest SIMS scores are those from patients injured more severely in high impact car accidents and also by malingerers.  In the analysis of variance (ANOVA), there was no statistically significant difference between the more severely injured patients (those with post-concussion and whiplash syndrome and pain related insomnia) and malingerers: these two groups may report similar number of symptoms on the SIMS.  Briefly, the SIMS is a pseudo psychological test.   

The detection of malingering is usually a difficult task.  Psychologists using the SIMS may appear more expert in detecting false insurance compensation claims because they classify more patients as malingerers or as suspected malingerers and are thus, being perceived as more professionally competent, more likely rehired by insurance companies.   

The SIMS is usually scored by calculating the total score and also scores on 5 scales with 15 items each:  Neurological Impairment (NI), Amnestic Disorder (AM), Psychosis (P), Affective Disorders (AF), and Low Intelligence (LI).^[4] Of these 5 scales, the Psychosis (P), Affective Disorders (AF), Neurologic Impairment (NI), and Amnestic Disorder (AM) scales list obviously legitimate medical symptoms^[5,6,7] that could be endorsed by both patients and malingerers at similar rates.  It has been also shown that the Low Intelligence scale (LI) of the SIMS consists mainly of arithmetic and logical reasoning tasks or tasks assessing general knowledge on which patients tired by chronic illness, or those with the post-concussion syndrome, or persons whose attentional focus is disrupted by chronic pain may perform worse than uninjured persons^[8,14].  As a consequence, none of the SIMS scales shows an adequate criterion validity when the data from patients injured severely in high impact car accidents was compared via ANOVA to data from less injured persons, from malingerers, and from uninjured non-malingering normal persons.^[13]

Validation procedure used in the development of the SIMS

The test “validation” of the SIMS^[1,4] by Smith and Burger^[1] proceeded by comparing healthy undergraduates instructed to respond honestly to responses of healthy undergraduates instructed to feign medical or psychological symptoms.  As a logical result, the SIMS indeed differentiates persons reporting certain medical symptoms from those who do not, but it fails differentiating malingerers from legitimate patients.  Such pseudo-validation does not meet test standards stipulated by the American Psychological Association (APA).^[15]  Since the purpose of the SIMS is to differentiate malingerers from legitimate patients, the APA standards require a comparison of these two groups to demonstrate that the SIMS indeed differentiates malingerers from real patients, i.e., that it has an adequate specificity.   In fact, SIMS specificity calculated by Richard Rogers et al.^[3] for total SIMS score cutoff  > 14 points was only .28.  These authors^[3] mentioned,  with respect to SIMS cutoff of > 14 points, that “research (e.g., Clegg et al., 2009^[16]) has found that non-feigning clients often exceed this cut score.”  Rogers and his research team suggested that the cutoff for SIMS total score might need to be set as high as > 44 points to improve specificity, when dealing with certain diagnostic groups.^[3]  Statistics provided by Rogers et al. suggested that more than two thirds of honestly responding psychiatric patients would be misclassified as malingerers. ^[3] Unfortunately, the SIMS has been translated into many other languages, thus exposing thousands of legitimate patients to iatrogenic malpractice.^{[17,18,19,20,21, 22]}

Rare Symptoms (RS) and Symptom Combination (SC) scales of the SIMS

The RS and SC scales were published by Richard Rogers’s team^[3] in 2014 to determine if more accurate SIMS scales could be developed by extracting SIMS items that might differentiate psychiatric patients responding honestly from those instructed to exaggerate their symptoms. Rogers used two strategies. The first was to locate SIMS items listing medical symptoms reported rarely by the honest group but frequently by the exaggerating group: "The rare symptoms (RS) scale was created by identifying SIMS items endorsed by less than 10% of genuine responders but more than 25% of feigners." The SIMS RS scale developed by Rogers contains 15 SIMS items.^[3]

The second strategy used by Rogers identified unlikely symptom combinations endorsed by likely feigners but rarely endorsed by genuine patients. The following procedure was used to develop the SIMS symptom combination (SC) scale: "The correlations of all SIMS pairs were first calculated. Pairs of items were selected on two criteria: (a) they were uncorrelated or negatively correlated for genuine responders; and (b) they were positively correlated for feigners and accounted for more than 10% of the variance (φ coefficient > .35)."^[3]  The SC scale contains 13 pairs of SIMS items.

Content analysis of RS scale^[23] suggested irremediable flaws.  A third of the RS items are logical or algebraic reasoning tasks on which patients with severe post-concussive symptoms and fatigue from insomnia (such as caused by persistent pain) could perform less well.^[23]  Patients with cerebral microvascular injuries and axonal shearing from their accident are more likely to score higher on the RS and be misclassified as “malingerers” than less injured persons.  Another third of RS scale items lists delusional symptoms or those of thought disorder: psychotic patients are more likely to be branded as “malingerers” and deprived of pharmacotherapy. ^[23]

The SC scale is based on a precarious assumption that correlations among its symptoms remain the same across varied groups of genuine medical patients, regardless of the type and intensity of their own symptoms. Patients more severely disabled by their symptoms might be less consistent in their responses and thus more often misclassified as “malingerers” by the SC. ^[23]  

Furthermore, Rogers's psychiatric sample on which the RS and SC scales were developed was diagnostically mixed, too heterogeneous, mainly diagnosed with PTSD (>77%) and/or mood disorders (>32%):^[3]  this makes generalizations of RS and SC cutoffs to other diagnostic groups of psychiatric patients uncertain. ^[23]

Rogerian methodological approach used in development of such scales could indeed be promising, but only if validated by contrasting responses of malingerers reporting symptoms of a particular medical or psychological condition to those of legitimate patients with that identical particular condition, on large samples, and separately for each medical condition, and preferably with scales derived from non-SIMS items.^[23]