Structural validity describes how well the grouping of scores within an assessment aligns with the theoretical grouping of what the item content measures.

Substantive validity is established through a description of the theoretical rationales that explain consistency in one’s response to test items. A unifying perspective for the validity of screener scores is that there are six broad forms of validity that may be housed under an umbrella term of construct validity.Ĭontent validity are the characterizations of the assessment content’s relevance, the overall representativeness of the content (e.g., test items or stimuli), and the quality of the test items or stimuli.

This qualifier, an evidence of validity, is used because there are many ways to operationalize and provide evidence for how well scores (e.g., from an assessment such as a screener) reflect what an assessment is supposed to measure. In our Ice Cream Personality Test example, if a researcher hypothesized that ice cream preference is associated with personality description, a moderate to strong correlation between ice cream and personality preference would provide an evidence of validity. Where reliability would be evaluated through the consistency of scores, validity is concerned with how well a set of scores reflects the intended construct or domain being assessed. A higher correlation for each question across the two-week testing would provide evidence that participants’ ice cream preferences and personality selections were stable over two weeks. The reliability of scores could be demonstrated by administering the questions twice over a two-week period and then correlating the responses for each question over the assessment period. The second question asks the participant which of six personality descriptions bests fits their personality. The first question asks the participant to select their top ice cream preference from a list of six flavor options. A group of participants answer two questions. This assessment is often used to demonstrate the concepts of reliability and validity.

Marianne Miserandino’s Ice Cream Personality Test can help illustrate their overlap and differences.

The reliability and validity of scores from assessments are two concepts that are closely knit together and feed into each other. Validity is broadly defined as how well something measures what it’s supposed to measure.

What is reliability and validity