Top 3 Recruitment Test Mistakes and How to Fix Them

When recruitment testing, it’s crucial to avoid common pitfalls that can lead to suboptimal recruitment outcomes. One significant mistake is failing to properly screen candidates based on their psychometric test results, which can provide deep insights into their suitability for the role. Another error is a lack of selectiveness during the hiring process, often resulting in a poor fit between the employee and the organisation. Additionally, an overemphasis on tangible but narrow skills can overlook candidates' broader potential and adaptability, which are increasingly valued in today’s dynamic work environments. By addressing these issues, companies can enhance their hiring strategies and secure the most capable and fitting candidates.

In this article, we will outline the top 3 mistakes when it comes to recruitment testing.

Recruitment tests including personality questionnaires, should ideally be used as screening tools early in the recruitment process. Consequently, the results of the personality questionnaire should help inform who attends the interview in the first place. This means that the personality questionnaire should have already served its purpose and improved the quality of the applicant pool by the time the candidate makes it to interview. It’s also worth noting that recruitment tests only add value when used to screen candidates out, as this improves the average quality of the applicant pool. If you fail to screen candidates out, and then merely hand the results to a hiring manager, you miss the opportunity to improve the applicant pool, minimising the return on investment.

Instead of screening candidates out many employing organisations will use personality questionnaires to support interviews. Although it sounds intuitive, interviews are not effective measures of personality traits, and so exploring personality traits in an interview relies on flawed logic. Consequently, skilled interviewees may simply talk their way out of concerning scores, nullifying the benefits of the assessments. For example, imagine a candidate receives a low score on “resilience”, and the hiring manager starts asking probing questions about resilience. Because the candidate is particularly interpersonally skilled and experienced with interviews, the interviewee changes their mind, and is hired anyway. Shortly after, the candidate leaves the job, being entirely unable to deal with the stresses of the role, highlighting the futility of this approach.

Perhaps the costliest mistake associated with this approach is the weakening of the interview itself. Research shows that structured interviews i.e. interviews where each candidate gets the same questions, are substantially more predictive of performance than unstructured or conversational interviews. Although many believe that personality questionnaires help structure interviews, they actually do the opposite. Naturally, interviewers will flag different areas for different candidates, resulting is very different lines of questioning. This makes structured interviewing impossible, substantially reducing the predictive validity of the interview itself.

Lastly, because interview performance and personality results tend to be uncorrelated, discrepancies between the two are likely to result in distrust of the chosen assessment. Naturally, if a candidate receives a low score on resilience, but convincingly claim to be highly resilient in the interview, one of those information sources must be incorrect. Realistically, hiring managers are more inclined to believe their own judgements than an external assessment, regardless of what the evidence would suggest. This results in reduced buy-in from managers, and could result in assessments being dropped all together.

The next major mistake that I see people make is not being selective enough with their candidates. This is particularly true when using cognitive ability assessments, such as verbal, numerical, and inductive reasoning tests. Often, organisations set particularly low pass marks for these assessments, claiming they want to avoid “screening out good candidates.” The problem with this approach is that cognitive ability tests (and other valid recruitment tests) are powerful indicators of performance in the workplace, and the greater the selectivity, the greater the return on investment. Therefore, the more selective you are, the better the quality of candidate in a linear fashion. Ironically, by being less selective than you are able, you minimise the probability of finding good candidates; you don’t maximise it.

Lastly, low pass marks have an added disadvantage of increasing the likelihood of success using AI-based cheating. Currently, AI models can achieve low-mediocre results when used to answer recruitment tests, particularly ability tests. Low pass marks, therefore, may be low enough to allow AI-based cheaters to actually pass, weakening the validity of the selection process. High pass marks, however, would guarantee failure using AI-based cheating, protecting the integrity of the assessment process. Moreover, as AI becomes more adept at facilitating this kind of cheating, the importance of being highly selective is likely to increase over time, allowing organisations to be even more selective in the future.

This is a frustratingly common mistake that HR professionals, and even many occupational psychologists make when deciding which characteristics to screen for. Typically, organisations conduct a job analysis when drafting a job description, identifying the specific characteristics that underpin performance in-role. Often, this takes the form of a brainstorming session, whereby the HR team generates a list of criteria which they believe are key. Although this logic is sound, this approach tends to bias towards specific, tangible, but overly narrow skills, at the expense of more general ones.

For example, when conducting a job analysis for a data scientist role, the first skills that come to mind are things like VLOOKUPs in Excel, machine learning in Python, Bayesian statistics, etc. Although these are undoubtedly relevant, they are likely to account for relatively narrow aspects of the role but come to mind quickly. More amorphous competencies, such as the ability to learn, solve problems, make decisions, and work with learned information, probably account for 50%+ of the job itself, but are far more likely to be overlooked during a job analysis. This leads to a significant overweighting of tangible, but narrow technical skills, and an under-focus on broader competencies.

Additionally, many organisations are simply unaware of how best to measure these broader competencies, and thus stick with tangible skills out of necessity. If the organisation is unfamiliar with cognitive ability tests, situational judgement tests, personality questionnaires, and other more advanced psychometric assessments, then they will lack any effective mechanism for measuring broader competencies. Instead, they will opt for technical and hard skills tests, narrowing the scope of selection significantly.

However, the biggest problem with screening candidates exclusively based on tangible skills is that those skills are often easily learned. The best example of this problem is with early-stage careers, particularly graduate and apprentice schemes. In these roles, the organisation will be spending years providing training and development for harder skills, making their current level of skills entirely irrelevant. Ultimately, when screening based on hard skills, you must always consider this time horizon and only consider hiring for specific skills if they need to hit the ground running and have no time to learn on the job.

Each of these three mistakes, although seemingly unrelated, ultimately stem from the same problem: a lack of trust in recruitment tests. Many employers simply don’t trust that their assessments can identify top talent or measure the characteristics which really matter in the workplace, and thus try to limit their scope. They will hesitate to screen candidates out, or they will set unreasonably low pass marks, or they will stick with overly specific abilities because they just can’t see how else to use them. On one hand, this concern is understandable, and organisations certainly should be judicious when using recruitment tests. Naturally, not all providers are equal, and many out there offer assessments that are unreliable, invalid, and unfair to candidates, actively harming the organisations that use them.

On the other hand, however, the empirical evidence behind recruitment tests as a whole is extremely supportive of their use. For example, research shows that cognitive ability tests are the most powerful predictors of performance in complex work that we know of. Additionally, personality questionnaires not only predict performance but also engagement, satisfaction, retention, and commitment in organisations, making them incredibly versatile tools. Although some providers create poor-quality assessments, there is no logical basis to discount recruitment tests in general, as their supporting academic evidence is overwhelming.

Ultimately, if organisations are serious about maximising the quality of hire, streamlining the selection process, and ensuring the fairness of the process, they should strongly consider making the most of their recruitment tests. Given the robustness of the evidence behind them, if you still don’t feel comfortable using assessments in recruitment, then it may be worth seeking an alternative provider, one who can put your mind at ease. To learn more about how recruitment tests could revolutionise your recruitment processes, book a call with one of our experts or register for a free trial now.