Paper-digital comparison study

Two studies were conducted to see if there was a difference in the way students scored between the paper and digital editions of CAT4.

  • The overall numbers of students doing the digital and paper versions in the standardisation sample were large. This allowed a study to be undertaken looking at the relative difference in scores between those students doing paper and digital editions during the CAT4 standardisation.
  • The second study, also in autumn 2011, looked at the results of an equivalence study conducted in three year groups. Around 1,300 students in this study did both the paper and digital versions of the CAT4 Nonverbal Battery for Levels A, B and E. To reduce practice effects, around half the students completed the paper edition first followed by the digital while the other half took the digital edition first followed by the paper.

The results of both studies have shown small differences in scores, with students completing the paper edition scoring slightly higher on average than on the digital edition. For example, the Nonverbal Reasoning Battery Level E paper raw score is, on average, half a mark higher than for the digital edition and around one mark higher for Level B.

The normative scores have therefore been adjusted to take into account any differences in the way students respond digitally or on paper.