Reliability and Validity
The twin pillars of educational measurement — and the single largest block of marks in the 2022 paper.
Where candidates lose marks on this
-
Treating reliability and validity as interchangeable. They are logically distinct; reliability is a necessary condition for validity, not a synonym.
-
Getting the heterogeneity direction backwards. More heterogeneous group → higher reliability. Intuition misleads here.
-
Squaring or square-rooting the reliability coefficient when asked for true variance. Reliability of 0.64 = 64% true variance. Directly.
-
Confusing concurrent with predictive validity. The only distinguishing feature is when the criterion is measured — now (concurrent) or later (predictive).
-
Confusing difficulty index direction. A high difficulty index means an easy item (covered fully in Chapter 2, but it leaks into reliability questions).
-
Assuming marks are on a ratio scale. They are interval — and so is the raw count of correct answers: zero correct is not zero ability. Ratio needs a true zero of the attribute itself (time, height, weight).
-
Forgetting the Spearman-Brown correction after a split-half correlation. The uncorrected figure is an underestimate.
-
Believing a test valid for one purpose is valid generally. It is not.
-
Confusing objectivity with validity. An objective test can measure the wrong thing.
-
Thinking a longer test is always better. It raises reliability but at diminishing returns, and at some cost to usability and to candidate fatigue.
Where this is taught
Start your VP & Principal 2026 preparation → How the exam is marked