Are your personality tests accurate?
Lead consultant at Test Partnership, Ben Schwencke, answers the timely question: Are your personality tests accurate? How are they impacted by social desirability bias and impression management?
Transcript
Social desirability bias and impression management with regards to personality questionnaires is a really tedious issue realistically for occupational psychologists and test providers. People often just off the bat will say, oh we don't want to use them because we're concerned people would just fake it.
Let's address it because it is indeed something that comes up very frequently. Overall, trying to map out a specific profile when you don't actually know what each question is measuring. It's very hard to do and the research suggests that people don't do it very often and they don't do it very well.
And social desirability bias fundamentally is responding excessively positively or excessively negatively to personality questionnaire items which are framed, shall we say, excessively positively or negatively. For example, a question like, 'I'm a really good employee'. Realistically, people are not going to say no.
They're not going to say STRONGLY DISAGREE to that. Questions like, oh, 'I'm a really friendly, happy person to be around'. Whether or not that's true, there is an inclination to try to rate highly on those questions. Something of a problem, because obviously it inflates scores. It also reduces the effectiveness of those questions, because fundamentally everyone's answering the same, so it doesn't distinguish between people high or low, high or low on those specific characteristics.
So that is more, in my opinion, of a problem. Then something like intentional impression management, which is very difficult to do and happens very rarely. One thing I don't like that a lot of assessments have in the personality space: 'consistency score'. It's a gotcha. They put in questions Something like, 'I've never told a lie before'.
'I would never lie to anyone ever'. And then naturally, the liars say, STRONGLY AGREE. The research does not suggest that those scores mean anything. That was never an evidence based approach in the first place. My advice is to always recommend against using those. And there are stakes involved, because if you deprive someone of a job because this consistency scale suggested that they're a liar, It's a very big claim to make.
A hard one to justify, a hard one to support if anything, you know, sort of went legal in that instance. Um, I really recommend against those. The evidence is not supportive of their use. However, more modern advances in psychometrics, particularly with regards to item response theory, have some interesting solutions here.
It reveals a whole lot about individual questions in a questionnaire. For example, you can see social desirability bias. just psychometrically, empirically, using item response theory methods. What will happen is you'll see a sort of score caps out a sort of bottom end, this lower asymptote will appear when you model it.
That can tell you this question is in fact socially desirable, and people are refusing to use one end of the rating scale. Additionally, with item response theory methods, questions themselves are assigned something like a 'difficulty', sort of a difficulty level like you would have with an ability test, where some questions are harder to endorse than others.
For example, a question measuring extroversion, for example, like, 'I like to go to parties'. Fairly easy to endorse, even for, you know, a fairly introverted person. A harder question would be something like, 'I love to go to parties'. 'I spend all of my time at parties'. That's a harder to endorse question, particularly for an introvert.
And as a result, you can model this information. And a question like, 'I'm a really good employee', is going to be really easy. Psychometrically, because everyone is saying STRONGLY AGREE, which means it's weighted. So when you calculate a score using our response theory, it weights it based on the difficulty of the questions given.
And so you don't get a huge advantage falling for a social desirability bias effect. Because it's modelled into the questionnaire itself. So it's much safer, it's much more reliable, it's less pervasive, it's much less vulnerable to social desirability bias. And I'm of the opinion now that it's definitely not something that should stop you using personality questionnaires anymore.
The issue, particularly of impression management, social desirability bias is fundamentally under control.
