Ability Tests Have Never Been More Powerful (2024)

For decades, it has been known that ability tests are the strongest predictors of job performance and training performance known. The have been found to be stronger than interviews, other psychometric tests, and any identifiable variable known to occupational psychologists.

However, these often cited figures rely on relatively old evidence, often incorporating research from the mid 20th century.

The most cited paper in this area - the famous Schmidt and Hunter’s 1998 meta analysis - showed the correlation between general cognitive ability and job performance to vary depending on the complexity of the role:

A clear continuum can be seen, with ability tests better predicting performance in more complex work.

This makes intuitive sense.

Obviously, to perform in highly complex technical, professional or managerial roles, a high level of cognitive ability is required to learn the necessary skills, make solid judgements, and use multiple sources of information to solve meaningful problems.

In unskilled, low complexity work however, much of the process is automated and requires little cognitive input from the employee. As a result, high levels of cognitive ability are not required to perform well in these roles.

Statistically speaking, a correlation coefficient tells us a great deal about how relevant cognitive ability is regarding employee performance. Using the correlation coefficient, we can identify the percentage of variance that can be accounted for in job performance by cognitive ability. For example, in the case of professional/managerial work:

• .58 x .58 = 0.34
• 0.34 x 100 = 34
• 34% of variance

Based on the above numbers, 34% of variance in professional/managerial job performance is associated with cognitive ability, with the remaining 66% of variance likely accounted for by personality traits, skills, motivations, preferences, and the thousands of other factors that effect performance at work.

It is clear to see why occupational psychologists recommend using cognitive ability tests in employee selection, especially for complex roles. However, has this trend continued into the 21st century, and if so, how predictive is it now?

A key finding in the new UK meta analysis by Bertua, Anderson, and Salgado, is that in professional/managerial work, the correlation with job performance is significantly larger than in the older meta analysis.

In fact, they found the correlation to be 0.69, meaning that almost half the variance in professional/managerial job performance can be accounted for by cognitive ability.

Not only does this mean that cognitive ability is the strongest predictor of professional/managerial job performance known, but it almost accounts for an absolute majority of the variance in performance. This means that cognitive ability is individually as important as every single other factor that could influence job performance combined.

Organisations not using ability testing for complex roles are put at a tremendous competitive disadvantage over organisations that do. Never before has employee performance been so easily and conveniently predicted, and organisations not taking advantage of this research are likely to have serious difficulties recruiting high performers.

There are principally two potential explanations for the different findings between the two large meta analyses, the first mundane, and the second with huge implications:

• Improved measurement and research methods: It is simply possible that meta-analytic techniques, research methods and statistical procedures have improved, resulting in larger correlation coefficients. If this is the case, then cognitive ability has always been this highly predictive of future performance, however research is now better able to quantify the magnitude of that association than in previous decades.
• The world is getting more complex: As the original meta analysis showed, performance in complex work is better predicted by cognitive ability. Compared to the 1970-1990s, the world has become radically more complex due to globalisation, the internet, big data, increases in computing power etc. As a result, what was considered complex work in the 1980s may be considered relatively simple work now, and what is considered complex work now, is radically more complex than at any other time in history. Theoretically, based on our understanding of cognitive ability, ability tests should be more predictive of complex job performance than ever before, and this would explain the difference between the two findings.

In practice, I suspect that both explanations play a factor in the disparate results shown from the two meta analyses. However, I believe the latter explanation presents the strongest case, and follows logically from our understanding of complexity, cognitive ability and job performance.

If cognitive ability testing is becoming more predictive of future performance, then we can expect ability tests to become more integral to employee selection processes.

Already, occupational psychologists are recommending that ability tests be seen as the primary selection tool, with interviews being a supplement, and not the other way around. Increasingly, ability tests will be recognised as the powerful predictive tools that they are, and HR practitioners will feel more confident making high level decisions based on their results.

Similarly, with the increased demand, test publishers will invest more money into R&D, developing more powerful, accurate and engaging ability tests. Already, item response theory, computer adaptive testing and gamified assessments represent attempts to improve ability testing through technology and science. This trend is expected to continue, helping organisations achieve greater ROI from their employee selection and assessment processes.

Bertua, C., Anderson, N., & Salgado, J. F. (2005). The predictive validity of cognitive ability tests: A UK meta‐analysis. Journal of Occupational and Organizational Psychology, 78(3), 387-409.

Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological bulletin, 124(2), 262.