International benchmarking with CAT4

Introduction

The Cognitive Abilities Test® (CAT4) and Progress Test Series® for English, maths and science are standardised against the UK school population. Many international schools have asked for an international school standardisation of these assessments. However, we feel that:

  • A single representative international standardisation is difficult due to the diversity of international schools – for example curriculum, geography, number of students with English as Additional Language (EAL) differ between schools.
  • And, more importantly, our UK standardisations, based on a diverse student population*, have proven to be relevant and reliable for many international schools. For the second year in a row we have carried out benchmarking exercises to compare the scores achieved by thousands of international students globally against the standardisation of CAT4® (~224,000 students). The benchmark study for the Progress Test Series® will be released later this year.

* For example, during our last standardisation exercise, for the Baseline assessment, the EAL ratio was 18%

Standard Age Score

The Standard Age Score (SAS) is based on the student’s raw score, which is then adjusted for age and scaled to a mean of 100. When comparing SAS scores, a difference of less than 3 points is not usually seen as statistically significant

Comparison of UK standardisations and international scores in 2017

The distribution of international scores for CAT4®Level D is typical for international students and is very similar to the UK standardisation profile overall. Results show that typically international school students tend to score more highly on quantitative, nonverbal and spatial batteries, but lower on verbal.

Figure 1 shows the slight skew to the left on the verbal distribution; Figure 2 shows the slight skew to the right on the nonverbal for CAT4 D.

Distribution of verbal scores against the UK standardisation

Figure 1 - Distribution of verbal scores against the UK standardisation

Distribution of nonverbal scores against the UK standardisation

Figure 2 - Distribution of nonverbal scores against the UK standardisation

In other words, the typical international class might be expected to be slightly more able overall but have weaker verbal skills. Given the higher level of students with EAL in the typical international school, this is not surprising, but it is an important point. Low verbal reasoning scores can be indicative of students who may have difficulty in accessing the curriculum when it is presented in English.

Table 1

CAT Level UK International (Mean CAT4 Verbal Mean Quantitative Mean Non Verbal Mean Spatial Mean Students (N)
A 100 101 97 103 102 102 34.083
B 100 100 96 101 101 100 40,322
C 100 102 98 102 104 102 16,801
D 100 101 99 101 103 102 45.284
E 100 103 98 106 104 105 15.959
F 100 102 97 103 104 103 44,904
G 100 104 97 106 105 106 26,535
Overall 100 102 97 103 103 103 223,888

 

Table 1 shows mean scores for a number of CAT4® levels. The mean scores for some of the levels indicate that the international student benchmarking sample at these ages is more able than that of the UK standardisation. At level G in particular, (highlighted in green) this difference is significant. Verbal scores are lower across all levels, although this is not a significant difference except at Level B. Mean scores across batteries were relatively similar between this year and last year’s mean scores for CAT4®, varying only by a maximum of 1 point depending on the level

How can schools use the data?

Plan resourcing at a cohort level

Does one year group appear to be significantly below average or less able than your other year groups? Can you provide additional teaching support for this cohort?

Consider the implications for EAL learners

Students with EAL may need literacy support throughout their school career to enable them to fully realise their potential in formal exams and to prepare them for higher education. Look out for students with verbal scores significantly lower than scores achieved in the other batteries.

Take advantage of the combination reports

Quickly identify students who are under-performing by using the automatic combination reports for CAT4® and the Progress Test Series®.

See more here: https://www.gl-education.com/reports#combination

Set realistic but challenging targets

The CAT4 reports provide a useful range of likely outcomes for each student. Some schools use these as a basis for a discussion with students helping them take ownership of their targets.

Your point of reference

The CAT4 standardisation is a reliable benchmark for international students. Schools may additionally wish to compare their own data to the findings of this benchmarking exercise to see if they reflect some of the nuances seen between the different sections of CAT4 and from year to year with the Progress Test Series®.