Standardisation
Rapid underwent a full national re-standardisation in January – July 2019. The standardisation was conducted in 48 Primary schools and 16 Secondary schools (England n = 50; Northern Ireland n = 11; Scotland n = 1; Wales = 1; Republic of Ireland n = 1). Of those schools where an Ofsted assessment has been published, 24% were rated as Outstanding, 68% were rated as Good and 8% were rated as Requiring Improvement (which compares well to national figures for the 2018/19 academic year: 20% Outstanding; 66% Good; 11% Requires improvement). The number of students on the roll for the Primary sample schools ranged from 30 to 683, with an average of 268. Whilst the number of students on the roll for the Secondary sample schools ranged from 280 to 1777, with an average of 846.
School characteristics (where these were available on Gov.uk or the equivalent websites for Scotland, Northern Ireland, Wales and the Republic of Ireland) for the sample schools were compared to the national average (for English state-funded Primary and Secondary schools) – see Table 5. It can be seen that the schools overall included a slightly higher proportion of girls than the national average and a slightly lower proportion of pupils with an ECHP than the national average. Within the secondary schools, the proportion of pupils requiring SEN support was slightly above the national average, whilst there is a lower proportion of pupils whose first language is not English and pupils eligible for FSM than the national averages.
Table 5. Characteristics of schools within the standardisation sample
School characteristic | Primary sample Mean | Primary National average | Secondary sample Mean | Secondary National average |
Girls on roll | 51.5% | 48.7% | 54.4% | 49.2% |
Pupils with an SEN Education, Health and Care Plan | 2.1% | 3.1% | 2.7% | 4.4% |
SEN Support | 12.6% | 12.2% | 12.0% | 10.4% |
Pupils whose first language is not English | 18.6% | 21.3% | 10.5% | 16.5% |
Pupils eligible for free school meals at any time during the past 6 years | 24.4% | 24.3% | 22.2% | 28.6% |
Within the selected schools, students were included in the standardisation on an entire class basis, to avoid any selection bias. The number of students completing each subtest, within each age group of the standardisation sample, are shown in Table 6.
Table 6. Students per age group for each subtest
Subtest | Age 4 | Age 5 | Age 6 | Age 7 | Age 8 | Age 9 | Age 10 | Age 11 | Age 12 | Age 13 | Age 14 | Age 15 | Total |
Crayons | 92 | 187 | 302 | 509 | 1090 | ||||||||
Races | 142 | 267 | 443 | 611 | 1463 | ||||||||
Rhymes | 126 | 262 | 432 | 599 | 1419 | ||||||||
Mobile phone | 900 | 780 | 695 | 803 | 775 | 577 | 561 | 316 | 5407 | ||||
Funny words / Non-words |
891 | 775 | 681 | 851 | 838 | 604 | 583 | 327 | 5550 | ||||
Word chopping / Segments |
776 | 719 | 684 | 777 | 752 | 560 | 509 | 277 | 5054 |
Demographic information concerning the students within the standardisation sample are given in Table 7 (note that information was not provided for all students). Population parameters are also provided, but these are based only on English state-funded Primary and Secondary schools, whereas the sample also includes students from Northern Ireland, Wales, Scotland and the Republic of Ireland, so the comparisons are limited.
For the Primary sample, it can be seen that the sample included a slightly higher proportion of female students than the national average for English state-funded Primary schools. With regards to ethnicity, the sample has a higher proportion of Asian students than is found in the population and lower proportions of White, Black, Mixed and Other students, although ethnicity information was not provided for 11.0% of the sample. The number of students within the sample who are eligible for Free School Meals is slightly higher than in the population. However, it should be noted that the national average for Northern Ireland (where 9 of the schools were based) is 29.4%, which may account for the higher proportion of students eligible for FSM within the sample. With regard to language, the percentage of students within the sample speaking English as an Additional Language is close to the population average. The proportion of students within the sample with a diagnosed SEN is slightly higher than within the population, whereas those with an Education, Health and Care plan reflects the national average. Again, it should be noted that the national average for SEN in Northern Ireland is 21.0%, which may account for the slightly higher proportion of students with SEN within the sample.
For the Secondary sample, it can be seen that the sample included a lower proportion of female students than the national average for English state-funded Secondary schools, although this information was not provided for 11.6% of the sample. With regards to ethnicity, the sample has a higher proportion of White and Other ethnicity students than are found in the population and lower proportions of Asian and Black students, although ethnicity information was not provided for 6.0% of the sample. The number of students within the sample who are eligible for Free School Meals is higher than in the population. However, it should be noted that the national average for Northern Ireland (where 2 of the schools were based) is 31.4%, which may account for the higher proportion of students eligible for FSM within the sample. The percentage of students within the sample speaking English as an Additional Language is below the population average, although this information was not provided for 23.6% of the sample. The proportion of students within the sample with a diagnosed SEN is slightly higher than within the population, whereas those with an Education, Health and Care plan is considerably higher than the national average. Again, it should be noted that the national average for SEN statements (equivalent to EHCP) in Northern Ireland is higher (4%), which may account for the higher proportion of students with ECHPs within the sample.
Table 7. Demographic details of sample
Variable | Classification | Primary sample | Primary population parameters* | Secondary sample | Secondary population parameters** |
Gender
|
Male Female Not available |
46.6% 50.8% 2.6% |
51.3% 48.7%
|
44.7% 43.6% 11.6% |
50.8% 49.2%
|
Ethnicity
|
White Asian Black Mixed Other Not available |
59.1% 20.4% 2.9% 5.5% 1.1% 11.0% |
73.6% 11.7% 5.5% 6.3% 2.0% 1.0% |
79.5% 5.1% 1.5% 4.7% 3.1% 6.0% |
73.2% 11.7% 6.0% 5.5% 1.9% 1.7% |
Free School Meals (FSM) |
Eligible for FSM Not eligible for FSM Not available |
17.2% 75.3% 7.5% |
15.8%
|
20.3% 62.8% 16.9% |
14.1%
|
English as an Additional Language (EAL) |
EAL Not EAL Not available |
22.5% 56.3% 21.1% |
21.2%
|
3.3% 73.2% 23.6% |
16.9%
|
Special Educational Need / Disability (SEND) |
Diagnosed SEND Suspected SEND No SEND Not available |
15.1% 3.1% 79.6% 2.2% |
14.2%
|
16.6% 10.2% 49.9% 23.3% |
12.4%
|
Education, Health and Care Plan (EHCP) |
Has EHCP No EHCP Not available |
1.3% 91.7% 7.0% |
1.6%
|
9.3% 75.1% 15.6% |
1.7%
|
* Based on DfE school census data for English state-funded Primary schools, January 2019
** Based on DfE school census data for English state-funded Secondary schools, January 2019
Of the 4–7-year-olds within the standardisation sample, 91% undertook the tests using desktop computers, whilst 9% used tablets. Analysis showed that on all three subtests, there was evidence of a platform effect, with students using desktops outperforming those using tablets. However, it is possible that these differences are due to school effects, with the schools that used tablets showing higher proportions of students being eligible for free school meals, and more students with SEND support/ECHPs than the schools using desktops. Of the 8–10-year-olds within the standardisation sample, 84% undertook the tests using desktop computers, whilst 16% used tablets. Analysis shows no evidence of a platform effect. Of the 11–15-year-olds within the standardisation sample, 95% undertook the tests using desktop computers, whilst 5% used tablets. Analysis showed that on all three subtests, there was evidence of a platform effect, with students using tablets outperforming those using desktops. However, it is likely that these differences are due to school effects, with the two schools that used tablets both being independent schools, whilst the group of 14 schools testing on desktops included just one independent school.
The sample data has been weighted according to age, gender and SEND proportion against population parameters. Using a non-parametric age-standardisation model, the raw scores for each age group were transformed into Standardised Age Scores (SAS) with a mean of 100 and a standard deviation of 15. This builds on previous work conducted by Schagen (1990). SAS scores for the subtests range from 65 to 135, although on subtests where there is a ceiling effect, the SAS is capped at the upper end. However, the caps do not distort the scoring at the lower end of abilities, which are those most commonly interested in.
Table 8 shows the correlations between all Rapid subtests. The correlations range from .350 to .607, with the majority being within the moderate range (.4 to .6). All correlations are significant at the p<.001 level.
Table 8. Intercorrelations between subtests
Cr | Ra | Rh | MP8 | FW | WC | MP11 | NW | Se | |
Crayons 4-7 (Cr) | 1 | ||||||||
Races 4-7 (Ra) | 440*(983) | 1 | |||||||
Rhymes 4-7 (Rh) | 350*(965) | 363*(1315) | 1 | ||||||
Mobile phone 8-10 (MP8) | 1 | ||||||||
Funny Words 8-10 (FW) | .461*(2156) | 1 | |||||||
Word Chopping 8-10 (WC) | 540*(2000) | 542*(1987) | 1 | ||||||
Mobile Phone 11-15 (MP11) | 1 | ||||||||
Non-Words 11-15 (NW) | 434*(2976) | 1 | |||||||
Segments 11-15 (Se) | .501*(2680) | 607*(2852) | 1 |
*all correlations are significant at p<.001; (N is shown in brackets)
In order to check for any gender bias, comparisons were made between males and females (where gender had been identified) on each subtest (see Table 9). Small effects were found on Crayons and Mobile phone, with both subtests slightly favouring girls. There were no other gender effects.
Table 9. Gender differences
Subtest | Gender | N | Mean | SD | SE of Mean | Cohen’s d* |
Crayons (4-7) |
Female Male |
597 454 |
101.66 98.63 |
14.594 15.204 |
0.595 0.714 |
0.20
|
Races (4-7) |
Female Male |
764 648 |
101.55 99.80 |
14.798 14.871 |
0.535 0.584 |
0.12
|
Rhymes (4-7) |
Female Male |
744 625 |
101.23 98.75 |
13.013 13.097 |
0.477 0.524 |
0.19
|
Mobile phone (8-10) |
Female Male |
1228 1122 |
102.47 98.89 |
14.742 15.196 |
0.421 0.454 |
0.24
|
Funny words (8-10) |
Female Male |
1193 1128 |
100.36 99.31 |
14.846 15.450 |
0.430 0.460 |
0.07
|
Word chopping (8-10) |
Female Male |
1101 1055 |
100.95 98.16 |
14.764 15.596 |
0.445 0.480 |
0.18
|
Mobile phone (11-15) |
Female Male |
1346 1355 |
102.91 99.77 |
14.895 14.549 |
0.406 0.395 |
0.21
|
Non-words (11-15) |
Female Male |
1465 1484 |
101.46 99.84 |
15.015 14.885 |
0.392 0.386 |
0.10
|
Segments (11-15) |
Female Male |
1290 1280 |
102.19 99.33 |
14.445 15.384 |
0.402 0.430 |
0.19
|
*Cohen’s d is a measure of effect size of the difference between two means
Checks were also made for ethnic group bias. Due to the small numbers in some ethnic minority groups, comparisons were made between White students and Other ethnic groups combined (where ethnicity had been identified) on each subtest (see Table 10). Small effects were found on Races and Rhymes, slightly favouring White students; and on Mobile phone (11–15), Non-words and Segments, favouring other ethnicities. There were no other ethnicity effects.
Table 10. Ethnic group differences
Subtest | Ethnicity | N | Mean | SD | SE of Mean | Cohen’s d* |
Crayons (4-7) |
White Other |
607 341 |
100.43 100.05 |
14.778 15.600 |
0.600 0.845 |
0.03
|
Races (4-7) |
White Other |
802 473 |
102.37 99.21 |
14.840 14.618 |
0.524 0.672 |
0.21
|
Rhymes (4-7) |
White Other |
775 457 |
101.25 98.07 |
12.801 13.412 |
0.460 0.627 |
0.24
|
Mobile phone (8-10) |
White Other |
1538 626 |
101.13 100.56 |
14.632 15.839 |
0.373 0.633 |
0.04
|
Funny words (Non-words) (8-10) |
White Other |
1494 632 |
100.13 99.65 |
15.215 15.152 |
0.394 0.603 |
0.03
|
Word chopping (Segments) (8-10) |
White Other |
1381 589 |
100.46 98.13 |
15.010 15.520 |
0.404 0.639 |
0.15
|
Mobile phone (11-15) |
White Other |
2489 407 |
100.90 104.28 |
14.789 14.865 |
0.296 0.737 |
0.23
|
Non-words (11-15) |
White Other |
2660 484 |
100.16 103.86 |
14.907 14.342 |
0.289 0.652 |
0.25
|
Segments (11-15) |
White Other |
2352 411 |
100.31 103.40 |
14.828 14.923 |
0.306 0.736 |
0.21
|
*Cohen’s d is a measure of effect size of the difference between two means