CAT4 UK standardisation: levels X and Y

Pre-trials

CAT4 Levels X and Y were developed after the main CAT4 Levels A-G were published. Small-scale trials were conducted in Autumn 2009 to check some of the new spatial questions being developed for the CAT4. Three versions of the spatial tests were created and were trialled with around 850 students in Years 4, 6, 8 and 9. Results from this study informed the development of further spatial questions for trialling.

Main trials

The main trials of the CAT4 Levels X and Y questions were carried out in Autumn 2013. Approximately 1200 students in Years 2 and 3 took part in the trials.

Four test booklets were created - two test booklets for each year group. Around 300 pupils took each booklet, with the parallel booklets of each year group alternated within a class. All the questions used in CAT4 Levels X and Y were used in the trialling with some of the questions duplicated in booklets across the two different year groups.

The data from the trials were analysed to provide information on the difficulty level of each question, its ability to discriminate between high and low scorers and the extent to which it proved equally difficult for both sexes, once overall score was taken into account. This information was then used to select and order the sequences of questions for the final standardisation version. Two versions of the test were created: Form X for 7 year-olds (Year 2 in England and Wales or equivalent) and Form Y for 8 year-olds (Year 3 in England and Wales or equivalent).

Standardisation

The standardisation of CAT4 Levels X and Y took place between May and June 2014 in England, Wales, Scotland and Northern Ireland. A national database of schools was created and schools were grouped into nine categories by country and within England. This was further grouped into ‘Independent’ plus five categories of maintained sector schools based on the proportion of students taking free school meals.

Schools were selected by stratified random sampling procedures within these groupings. As this was a national sample, many schools taking part in the standardisation had never used CAT4 before. Around 1900 students completed Form X and around 1100 students completed Form Y. The standardisation results were weighted to account for sample response bias.

The mean CAT4 Levels X and Y standard age scores (SAS) for males and females for Levels X and Y are in the tables below.

Level X

Level Y

Overall, female mean CAT4 scores are around 2 SAS points higher than for males for both Levels X and Y. In particular, the mean Verbal and Nonverbal scores are around 4 SAS points for females.

Note that the mean CAT4 score is not a Standard Age Score but an average of the nonverbal, verbal, quantitative and spatial SAS. The standard deviation for the mean CAT4 score is around 12, lower than the 15 that is expected for an SAS. This does not indicate the sample was unrepresentative in its spread of ability: rather, that the scores for the four components are correlated, so the spread narrows as scores are averaged.