Standardisation

Rapid underwent a full national re-standardisation in January – July 2019. The standardisation was conducted in 48 Primary schools and 16 Secondary schools (England n = 50; Northern Ireland n = 11; Scotland n = 1; Wales = 1; Republic of Ireland n = 1). Of those schools where an Ofsted assessment has been published, 24% were rated as Outstanding, 68% were rated as Good and 8% were rated as Requiring Improvement (which compares well to national figures for the 2018/19 academic year: 20% Outstanding; 66% Good; 11% Requires improvement). The number of students on the roll for the Primary sample schools ranged from 30 to 683, with an average of 268. Whilst the number of students on the roll for the Secondary sample schools ranged from 280 to 1777, with an average of 846.

School characteristics (where these were available on Gov.uk or the equivalent websites for Scotland, Northern Ireland, Wales and the Republic of Ireland) for the sample schools were compared to the national average (for English state-funded Primary and Secondary schools) – see Table 5. It can be seen that the schools overall included a slightly higher proportion of girls than the national average and a slightly lower proportion of pupils with an ECHP than the national average. Within the secondary schools, the proportion of pupils requiring SEN support was slightly above the national average, whilst there is a lower proportion of pupils whose first language is not English and pupils eligible for FSM than the national averages.

Table 5. Characteristics of schools within the standardisation sample

Within the selected schools, students were included in the standardisation on an entire class basis, to avoid any selection bias. The number of students completing each subtest, within each age group of the standardisation sample, are shown in Table 6.

Table 6. Students per age group for each subtest

Demographic information concerning the students within the standardisation sample are given in Table 7 (note that information was not provided for all students). Population parameters are also provided, but these are based only on English state-funded Primary and Secondary schools, whereas the sample also includes students from Northern Ireland, Wales, Scotland and the Republic of Ireland, so the comparisons are limited.

For the Primary sample, it can be seen that the sample included a slightly higher proportion of female students than the national average for English state-funded Primary schools. With regards to ethnicity, the sample has a higher proportion of Asian students than is found in the population and lower proportions of White, Black, Mixed and Other students, although ethnicity information was not provided for 11.0% of the sample. The number of students within the sample who are eligible for Free School Meals is slightly higher than in the population. However, it should be noted that the national average for Northern Ireland (where 9 of the schools were based) is 29.4%, which may account for the higher proportion of students eligible for FSM within the sample. With regard to language, the percentage of students within the sample speaking English as an Additional Language is close to the population average. The proportion of students within the sample with a diagnosed SEN is slightly higher than within the population, whereas those with an Education, Health and Care plan reflects the national average. Again, it should be noted that the national average for SEN in Northern Ireland is 21.0%, which may account for the slightly higher proportion of students with SEN within the sample.

For the Secondary sample, it can be seen that the sample included a lower proportion of female students than the national average for English state-funded Secondary schools, although this information was not provided for 11.6% of the sample. With regards to ethnicity, the sample has a higher proportion of White and Other ethnicity students than are found in the population and lower proportions of Asian and Black students, although ethnicity information was not provided for 6.0% of the sample. The number of students within the sample who are eligible for Free School Meals is higher than in the population. However, it should be noted that the national average for Northern Ireland (where 2 of the schools were based) is 31.4%, which may account for the higher proportion of students eligible for FSM within the sample. The percentage of students within the sample speaking English as an Additional Language is below the population average, although this information was not provided for 23.6% of the sample. The proportion of students within the sample with a diagnosed SEN is slightly higher than within the population, whereas those with an Education, Health and Care plan is considerably higher than the national average. Again, it should be noted that the national average for SEN statements (equivalent to EHCP) in Northern Ireland is higher (4%), which may account for the higher proportion of students with ECHPs within the sample.

Table 7. Demographic details of sample

* Based on DfE school census data for English state-funded Primary schools, January 2019

** Based on DfE school census data for English state-funded Secondary schools, January 2019

Of the 4–7-year-olds within the standardisation sample, 91% undertook the tests using desktop computers, whilst 9% used tablets. Analysis showed that on all three subtests, there was evidence of a platform effect, with students using desktops outperforming those using tablets. However, it is possible that these differences are due to school effects, with the schools that used tablets showing higher proportions of students being eligible for free school meals, and more students with SEND support/ECHPs than the schools using desktops. Of the 8–10-year-olds within the standardisation sample, 84% undertook the tests using desktop computers, whilst 16% used tablets. Analysis shows no evidence of a platform effect. Of the 11–15-year-olds within the standardisation sample, 95% undertook the tests using desktop computers, whilst 5% used tablets. Analysis showed that on all three subtests, there was evidence of a platform effect, with students using tablets outperforming those using desktops. However, it is likely that these differences are due to school effects, with the two schools that used tablets both being independent schools, whilst the group of 14 schools testing on desktops included just one independent school.

The sample data has been weighted according to age, gender and SEND proportion against population parameters. Using a non-parametric age-standardisation model, the raw scores for each age group were transformed into Standardised Age Scores (SAS) with a mean of 100 and a standard deviation of 15. This builds on previous work conducted by Schagen (1990). SAS scores for the subtests range from 65 to 135, although on subtests where there is a ceiling effect, the SAS is capped at the upper end. However, the caps do not distort the scoring at the lower end of abilities, which are those most commonly interested in.

Table 8 shows the correlations between all Rapid subtests. The correlations range from .350 to .607, with the majority being within the moderate range (.4 to .6). All correlations are significant at the p<.001 level.

Table 8. Intercorrelations between subtests

*all correlations are significant at p<.001; (N is shown in brackets)

In order to check for any gender bias, comparisons were made between males and females (where gender had been identified) on each subtest (see Table 9). Small effects were found on Crayons and Mobile phone, with both subtests slightly favouring girls. There were no other gender effects.

Table 9. Gender differences

*Cohen’s d is a measure of effect size of the difference between two means

Checks were also made for ethnic group bias. Due to the small numbers in some ethnic minority groups, comparisons were made between White students and Other ethnic groups combined (where ethnicity had been identified) on each subtest (see Table 10). Small effects were found on Races and Rhymes, slightly favouring White students; and on Mobile phone (11–15), Non-words and Segments, favouring other ethnicities. There were no other ethnicity effects.

Table 10. Ethnic group differences

*Cohen’s d is a measure of effect size of the difference between two means