Adaptive Testing

Computer Adaptive Testing (CAT) is the future of assessment due to the recognised benefits of speed, accuracy, and candidate experience.

What is adaptive testing?

With adaptive testing, the test's difficulty adjusts to the performance of the candidate, getting harder or easier following a correct or incorrect answer respectively.

The principle behind CAT is simple: Ask candidates the right questions, not the same questions. As candidate’s progress through the test, CAT algorithms estimate their ability in real time. They then use this information to adjust the difficulty to the test, ensuring that optimally difficult questions are administered.

This provides each candidate with a unique testing experience, designed to reach the maximum level of precision in the minimum amount of time. By adopting this approach, CATs show a greater level of accuracy, become much harder to cheat on, and provide a better candidate experience.

Adaptive tests vs traditional tests

A one-size-fits-all approach simply doesn’t work with psychometrics. Give easy questions to high performers, they get them all right. Give hard questions to low performers, they get them all wrong. In both cases, it was a waste of time giving them a test at all.

More worryingly, if every candidate sees the same questions, what happens if an unscrupulous candidate leaks those questions? For fixed form tests, this means the test is now ruined, as anyone can simply download the questions and figure out the answers. Adaptive tests, however, are protected from this kind of cheating, making them far more suitable for online testing.

Designing, calibrating and publishing adaptive tests requires considerable expertise and resources. Publishers must write hundreds of questions per item bank, only retaining the questions which meet psychometric quality control procedures.

Advantages of Adaptive Testing

Shorter tests

Compared to fixed form tests, adaptive tests provide a comparable level of accuracy in roughly half the number of questions. This means that CATs can be completed in half the time, without sacrificing precision.

Better candidate experience

Candidates are only shown questions which are suited to their level of ability, so they don’t get put off by overly difficult questions or bored by overly easy questions. This means a greatly improved candidate experience.

Increased test security

Adaptive tests employ question banks that contain hundreds of unique questions, making it very unlikely that any two candidates will see the same questions. This means CATs are harder to cheat, ensuring the integrity of the results.

Frequent Updating

Because adaptive tests employ question banks, new questions can be added on an ongoing basis, keeping the content fresh, relevant, and engaging. This means that CATs are constantly and consistently improved over time.

Customisable content

Adaptive tests allows items to be added to the item bank (or dropped out) very easily without negatively affecting the performance of the test. This means test experiences can be customised to suit different presentation requirements.

Accurate results

Adaptive test algorithms administer a unique and optimal set of questions to every candidate based on their estimated ability. This increases the accuracy and precision of the assessments, quickly achieving a high level of accuracy.

Modern adaptive testing

Computer Adaptive Testing represents the pinnacle of psychometric testing. By combining technology and science, CATs offer a wide range of benefits over traditional testing methods.

Gone are the days of paper / pencil testing, where assessors needed to manually score test papers to gauge results.

More importantly though, no longer do employers need to worry about candidates trying to download the answers online.

Fair and accurate

Although giving each candidate the same questions may seem fair, in reality this reduces the fairness and accuracy of the assessment.

When low performing candidates are given hard questions, it stresses them out and forces them to guess. When high performing candidates are given easy questions, they get them all correct, and we learn nothing about their level of ability.

By adapting the difficulty of the test to the ability of the candidate, everyone is given the same testing experience, maximising fairness and accuracy.

Principles of adaptive testing

If a candidate answers a question incorrectly, the next question automatically selected from the item bank will be slightly easier and receive a lower weighting.
By homing in on the candidate level of ability, CATs administer the most informative questions available, reaching a higher level of accuracy in less time.
By reducing administration time, you prevent candidates getting frustrated, fatigued or just plain bored with the testing process, improving candidate experience.

If a candidate answers a question correctly, the next question automatically selected from the item bank will be slightly harder and receive a larger weighting.
Candidates are never bored by overly easy questions, or stressed out by excessively difficult questions, instead everyone receives an optimal testing experience.
Proving an optimal testing experience ensures that candidates perceive the assessment as fair and creditable, independent of how well they perform.

CAT methodology

Adaptive tests are the future of personalised, targeted, and fair assessments.

Each time a candidate answers a question (item), our Item Response Theory algorithm selects the best item to show the candidate next. A more difficult item is shown in response to a correct answer, and a less difficult item is shown in response to an incorrect answer.