Construct valdity relates to whether a particular psychometric assessment...
Lead consultant at Test Partnership, Ben Schwencke, explains reliability and how it differs from validity.
In the context of psychometric testing, reliability and validity are related, but are ultimately separate constructs. Put simply, reliability relates to the precision, accuracy, and replicability of psychometric test scores.
Validity however, answers the question “does this assessment actually measure the construct it claims to?”. As a result, reliability is required for validity, but not necessarily the other way around. For example, if a student completes a psychometric assessment 10 times, and gets the exact same score each time, the assessment can be said to show “reliability”.
However, more investigation is required to determine if the scores themselves are meaningful, and measure the psychological construct it purports to measure.
From a classical test theory perspective, there are two primary forms of reliability: Test-retest reliability, and internal consistency. Test-retest reliability involves giving the assessment to a group of participants two or more times, and evaluating the differences between each attempt. If the scores differ significantly between attempts, the test lacks reliability. If the tests scores are broadly similar (not necessarily identical), then the test can be said to be reliable. Internal consistency relates to the relationships between the individual items in the test and the overall score. With internally consistent tests, high scores on each specific item should correlate with a higher score on the assessment overall. Conversely, low scores on each specific item should correlate negatively with the score overall. This suggests that each question individually is measuring the same psychological construct, suggesting that the assessment is reliable.
Validity relates to whether or not a psychometric assessment measures its intended psychological construct. Although validity requires reliability, as unreliable tests cannot measure anything at all, reliability does not guarantee validity. There are various forms of validity, which include:
To show that a test is “valid”, multiple forms of validation are required, especially for newer assessments and less established psychological constructs. As part of the R&D process for psychometric assessments, psychometricians conduct many studies investigating the reliability and validity of the assessment, presenting the findings in a technical manual or academic journal article.