AI candidate matching tools promise to solve your volume problem. And if you're drowning in applications right now, that promise is genuinely tempting. One platform, a bit of machine learning, and suddenly the pile gets manageable.
But it's important not to outsource this to AI no matter how tempting it may be. The real problem is that CVs are a poor predictor of who'll actually do the job well, and feeding thousands of them into an algorithm doesn't change that. It just means you're making poor decisions faster, and at scale.
Before you hand your shortlisting process over to an AI tool, it's worth understanding what the science says, what's already happening on the candidate side, and what the legal record is starting to look like.
CVs were already a weak predictor before AI got anywhere near them
Schmidt, Oh, and Shaffer's landmark meta-analysis, which reviewed close to a century of research on what actually predicts job performance, found that years of experience correlates with performance at around 0.18, and educational attainment at 0.10. Both are classified, in their framework, as "unlikely to be useful." These aren't contentious, one-off findings, they've been replicated and built on for decades.
This is most acute for early-career candidates. A 22-year-old applying for their first professional role has, at best, a degree, a part-time job, and maybe some extracurriculars on their CV. Every candidate in your pipeline looks roughly the same on paper, and the ones who look slightly better are often just the ones who've had better advice about how to write one, or who've had access to more polished work experience. It very much advantages higher socio-economic background candidates.
Schmidt, Oh, and Shaffer's own conclusion for candidates without prior relevant experience is that general mental ability is the strongest predictor of future performance. Not their degree classification. Not their work history. Their capacity to learn, reason, and solve problems.
Putting a sophisticated AI on top of that CV data doesn't fix any of this. It just automates a poor decision more efficiently.
Candidates are already using AI to write the CVs your AI will screen
While you've been thinking about AI screening tools, candidates have been busy on the other side of the equation.
Roughly 586,000 job seekers used AI to write or improve their CVs in 2025 alone, and around 773,000 used it specifically to optimise their applications for automated screening systems. Those figures come from Kickresume's 2025 research, and they're almost certainly conservative given how mainstream these tools have become.
So what you've actually got is a closed loop: AI-generated CVs being evaluated by AI screeners, with no meaningful human input at any point in the middle. The system has, quietly, stopped measuring candidates. It's measuring how well someone (or their AI assistant) can reverse-engineer what the screening algorithm is looking for.
And when you step back and look at it, it's a bit absurd, isn't it? A candidate uses ChatGPT to write a CV optimised for AI screeners. Your AI screening tool reads that CV and decides they're a strong match. Nobody involved in that transaction has any idea whether the person can actually do the job. The "shortlisting" process has become a game of prompt engineering, and the winners aren't necessarily your best candidates.
Where's the humanity!!!
AI candidate matching embeds bias, it doesn't remove it
The most common selling point for AI screening tools is that they remove human bias from the process. And honestly, it's a compelling pitch. Human CV reviewers do introduce bias. Sorry, not sorry - that's part of being human, no matter how much we try.
But the evidence on AI bias is increasingly uncomfortable, and it should give anyone pause before assuming automation is the fairer option.
- A study from the University of Washington found that AI CV screening tools favoured candidates with White-associated names 85% of the time, when other factors were held constant.
- Stanford researchers in 2025 found that AI CV tools rated older male candidates more favourably than female or younger candidates, even when the underlying qualifications were identical.
- And perhaps the most well-known example of this is Amazon, who built an internal AI recruiting tool that ended up systematically downgrading CVs from women. The reason wasn't a glitch or an oversight, it was that the system had been trained on Amazon's own historical hiring data, which reflected years of decisions made in a male-dominated industry. The AI learned what "a good candidate" looked like from that data, and it looked like a man. Amazon eventually scrapped the tool entirely.
That's the thing with AI: it's only ever as good as the data it learns from. If that data reflects historical human bias, the AI will faithfully reproduce it, often at speed and at scale, and often with nobody noticing until the damage is done.
What makes this particularly problematic is something called automation bias, the tendency people have to treat algorithmic outputs as more objective and trustworthy than human judgements. When an AI tells you a candidate isn't a good fit, you're less likely to question it than if a human colleague said the same thing. That means discriminatory outputs are more likely to stick, unchallenged, than they would be if a person had made the same call.
The "AI is fairer" claim isn't just wrong in many cases. It actively makes things worse, because it encourages organisations to lower their guard at exactly the point where scrutiny matters most.
The legal exposure from AI CV screening is live and growing
This isn't theoretical risk. It's in the courts right now.
Mobley v. Workday is a class action case alleging that AI screening tools showed a "pattern and practice" of discrimination by race, age, and disability. A federal judge has since expanded the case and ruled that AI tools can be treated as an "agent" of the employer, meaning the company using the tool carries liability for what it decides. Not the vendor. The employer.
That ruling matters enormously. It means "we used a third-party AI tool" is not, on its own, a defence.
And beyond the legal risk, there's a reputational dimension that's easy to underestimate. Research suggests around 65% of candidates are uncomfortable with AI being used in hiring, and around 90% want employers to be upfront about it. "Our AI reviewed your application and didn't think you were a strong match" is a difficult thing to say publicly, especially when you can't fully explain what the AI was looking for or why it made that call. Candidates talk. Reviews get written on Glassdoor.
Can we maybe not try and outsource everything to AI and just be a bit more human again? (maybe, just a thought).
What actually predicts performance, and where AI can genuinely help
It'd be unfair to paint AI as universally bad for hiring, because that's not quite right either.
The same Schmidt, Oh, and Shaffer meta-analysis that finds CVs wanting is fairly clear about what does work. General cognitive ability correlates with job performance at around 0.65, which is about as strong as it gets in applied psychology. Structured interviews, where every candidate is asked the same questions and assessed against the same criteria, are significantly more predictive than unstructured ones. Validated psychometric assessments give you data on the things that actually matter: how someone reasons, learns, and approaches problems.
AI has genuine value in recruitment. It's just not in processing CVs. Scheduling interviews, managing candidate communications, surfacing people from existing talent pools, reducing administrative load on your team - these are all legitimate and useful applications. Go for it. The problem is specifically using AI to make or heavily influence shortlisting decisions based on CV data, because the underlying data isn't reliable enough to warrant that.
For early-career hiring in particular, this is where validated assessment tools come into their own. You get the volume management you were hoping for from AI screening, but you're working from data that actually predicts something. At Test Partnership, the shift we see when teams move away from CV screening towards cognitive and behavioural assessments is pretty consistent: instead of trying to find meaningful differences between applications that all look the same, they get real information about candidate capability from the start of the process. MindmetriQ, our game-based cognitive assessment, is specifically designed for this, measuring reasoning and learning ability in a way that's both candidate-friendly and resistant to the kind of AI-cheating that's affecting traditional assessments.
Conclusion and next steps
The case for AI talent matching rests on an assumption that's hard to sustain once you look at the underlying research. CVs are not worth processing at scale in the first place. The information is too thin, too gameable, and too weakly correlated with actual performance to be the foundation of a shortlisting process, let alone an automated one.
Layer on top of that the fact that candidates are already optimising their CVs for AI screeners, and you've got a system that's mostly measuring who played the game best, not who'd do the job best.
The next steps are to consider using assessments as a pre-screen to precede CV review in your process, particularly for high-volume or early-career pipelines. If you're already using AI screening tools, it's worth reviewing them for bias and checking what your legal exposure looks like in light of cases like Mobley v. Workday. And if you're looking for a scientifically grounded alternative to CV-based shortlisting, that's exactly what Test Partnership and MindmetriQ are built for. You can get in touch with our team here to talk through what that might look like for your hiring, or browse our aptitude tests for more information.
Joshua Hancock
Digital Marketing Manager at Test Partnership. Over 7 years experience as a writer, content strategist, SEO and digital marketer.