Standardisation

Rapid underwent a full national re-standardisation in January – July 2019. The standardisation was conducted in 48 Primary schools and 16 Secondary schools (England n = 50; Northern Ireland n = 11; Scotland n = 1; Wales = 1; Republic of Ireland n = 1). Of those schools where an Ofsted assessment has been published, 24% were rated as Outstanding, 68% were rated as Good and 8% were rated as Requiring Improvement (which compares well to national figures for the 2018/19 academic year: 20% Outstanding; 66% Good; 11% Requires improvement). The number of students on the roll for the Primary sample schools ranged from 30 to 683, with an average of 268. Whilst the number of students on the roll for the Secondary sample schools ranged from 280 to 1777, with an average of 846.

School characteristics (where these were available on Gov.uk or the equivalent websites for Scotland, Northern Ireland, Wales and the Republic of Ireland) for the sample schools were compared to the national average (for English state-funded Primary and Secondary schools) – see Table 5. It can be seen that the schools overall included a slightly higher proportion of girls than the national average and a slightly lower proportion of pupils with an ECHP than the national average. Within the secondary schools, the proportion of pupils requiring SEN support was slightly above the national average, whilst there is a lower proportion of pupils whose first language is not English and pupils eligible for FSM than the national averages.

Table 5. Characteristics of schools within the standardisation sample

School characteristic Primary sample Mean Primary National average Secondary sample Mean Secondary National average
Girls on roll 51.5% 48.7% 54.4% 49.2%
Pupils with an SEN Education, Health and Care Plan 2.1% 3.1% 2.7% 4.4%
SEN Support 12.6% 12.2% 12.0% 10.4%
Pupils whose first language is not English 18.6% 21.3% 10.5% 16.5%
Pupils eligible for free school meals at any time during the past 6 years 24.4% 24.3% 22.2% 28.6%

Within the selected schools, students were included in the standardisation on an entire class basis, to avoid any selection bias. The number of students completing each subtest, within each age group of the standardisation sample, are shown in Table 6.

Table 6. Students per age group for each subtest

Subtest Age 4 Age 5 Age 6 Age 7 Age 8 Age 9 Age 10 Age 11 Age 12 Age 13 Age 14 Age 15 Total
Crayons 92 187 302 509                 1090
Races 142 267 443 611                 1463
Rhymes 126 262 432 599                 1419
Mobile phone         900 780 695 803 775 577 561 316 5407
Funny words
/ Non-words
        891 775 681 851 838 604 583 327 5550
Word
chopping /
Segments
        776 719 684 777 752 560 509 277 5054

Demographic information concerning the students within the standardisation sample are given in Table 7 (note that information was not provided for all students). Population parameters are also provided, but these are based only on English state-funded Primary and Secondary schools, whereas the sample also includes students from Northern Ireland, Wales, Scotland and the Republic of Ireland, so the comparisons are limited.

For the Primary sample, it can be seen that the sample included a slightly higher proportion of female students than the national average for English state-funded Primary schools. With regards to ethnicity, the sample has a higher proportion of Asian students than is found in the population and lower proportions of White, Black, Mixed and Other students, although ethnicity information was not provided for 11.0% of the sample. The number of students within the sample who are eligible for Free School Meals is slightly higher than in the population. However, it should be noted that the national average for Northern Ireland (where 9 of the schools were based) is 29.4%, which may account for the higher proportion of students eligible for FSM within the sample. With regard to language, the percentage of students within the sample speaking English as an Additional Language is close to the population average. The proportion of students within the sample with a diagnosed SEN is slightly higher than within the population, whereas those with an Education, Health and Care plan reflects the national average. Again, it should be noted that the national average for SEN in Northern Ireland is 21.0%, which may account for the slightly higher proportion of students with SEN within the sample.

For the Secondary sample, it can be seen that the sample included a lower proportion of female students than the national average for English state-funded Secondary schools, although this information was not provided for 11.6% of the sample. With regards to ethnicity, the sample has a higher proportion of White and Other ethnicity students than are found in the population and lower proportions of Asian and Black students, although ethnicity information was not provided for 6.0% of the sample. The number of students within the sample who are eligible for Free School Meals is higher than in the population. However, it should be noted that the national average for Northern Ireland (where 2 of the schools were based) is 31.4%, which may account for the higher proportion of students eligible for FSM within the sample. The percentage of students within the sample speaking English as an Additional Language is below the population average, although this information was not provided for 23.6% of the sample. The proportion of students within the sample with a diagnosed SEN is slightly higher than within the population, whereas those with an Education, Health and Care plan is considerably higher than the national average. Again, it should be noted that the national average for SEN statements (equivalent to EHCP) in Northern Ireland is higher (4%), which may account for the higher proportion of students with ECHPs within the sample.

Table 7. Demographic details of sample

Variable Classification Primary sample Primary population parameters* Secondary sample Secondary population parameters**

Gender

 

Male

Female

Not available

46.6%

50.8%

2.6%

51.3%

48.7%

 

44.7%

43.6%

11.6%

50.8%

49.2%

 

Ethnicity

 

 

White

Asian

Black

Mixed

Other

Not available

59.1%

20.4%

2.9%

5.5%

1.1%

11.0%

73.6%

11.7%

5.5%

6.3%

2.0%

1.0%

79.5%

5.1%

1.5%

4.7%

3.1%

6.0%

73.2%

11.7%

6.0%

5.5%

1.9%

1.7%

Free School Meals (FSM)

Eligible for FSM

Not eligible for FSM

Not available

17.2%

75.3%

7.5%

15.8%

 

 

20.3%

62.8%

16.9%

14.1%

 

 

English as an Additional Language (EAL)

EAL

Not EAL

Not available

22.5%

56.3%

21.1%

21.2%

 

 

3.3%

73.2%

23.6%

16.9%

 

 

Special Educational Need / Disability (SEND)

Diagnosed SEND

Suspected SEND

No SEND

Not available

15.1%

3.1%

79.6%

2.2%

14.2%

 

 

 

16.6%

10.2%

49.9%

23.3%

12.4%

 

 

 

Education, Health and Care Plan (EHCP)

Has EHCP

No EHCP

Not available

1.3%

91.7%

7.0%

1.6%

 

 

9.3%

75.1%

15.6%

1.7%

 

 

* Based on DfE school census data for English state-funded Primary schools, January 2019

** Based on DfE school census data for English state-funded Secondary schools, January 2019

Of the 4–7-year-olds within the standardisation sample, 91% undertook the tests using desktop computers, whilst 9% used tablets. Analysis showed that on all three subtests, there was evidence of a platform effect, with students using desktops outperforming those using tablets. However, it is possible that these differences are due to school effects, with the schools that used tablets showing higher proportions of students being eligible for free school meals, and more students with SEND support/ECHPs than the schools using desktops. Of the 8–10-year-olds within the standardisation sample, 84% undertook the tests using desktop computers, whilst 16% used tablets. Analysis shows no evidence of a platform effect. Of the 11–15-year-olds within the standardisation sample, 95% undertook the tests using desktop computers, whilst 5% used tablets. Analysis showed that on all three subtests, there was evidence of a platform effect, with students using tablets outperforming those using desktops. However, it is likely that these differences are due to school effects, with the two schools that used tablets both being independent schools, whilst the group of 14 schools testing on desktops included just one independent school.

The sample data has been weighted according to age, gender and SEND proportion against population parameters. Using a non-parametric age-standardisation model, the raw scores for each age group were transformed into Standardised Age Scores (SAS) with a mean of 100 and a standard deviation of 15. This builds on previous work conducted by Schagen (1990). SAS scores for the subtests range from 65 to 135, although on subtests where there is a ceiling effect, the SAS is capped at the upper end. However, the caps do not distort the scoring at the lower end of abilities, which are those most commonly interested in.

Table 8 shows the correlations between all Rapid subtests. The correlations range from .350 to .607, with the majority being within the moderate range (.4 to .6). All correlations are significant at the p<.001 level.

Table 8. Intercorrelations between subtests

  Cr Ra Rh MP8 FW WC MP11 NW Se
Crayons 4-7 (Cr) 1                
Races 4-7 (Ra) 440*(983) 1              
Rhymes 4-7 (Rh) 350*(965) 363*(1315) 1            
Mobile phone 8-10 (MP8)       1          
Funny Words 8-10 (FW)       .461*(2156) 1        
Word Chopping 8-10 (WC)       540*(2000) 542*(1987) 1      
Mobile Phone 11-15 (MP11)             1    
Non-Words 11-15 (NW)             434*(2976) 1  
Segments 11-15 (Se)             .501*(2680) 607*(2852) 1

*all correlations are significant at p<.001; (N is shown in brackets)

In order to check for any gender bias, comparisons were made between males and females (where gender had been identified) on each subtest (see Table 9). Small effects were found on Crayons and Mobile phone, with both subtests slightly favouring girls. There were no other gender effects.

Table 9. Gender differences

Subtest Gender N Mean SD SE of Mean Cohen’s d*
Crayons (4-7)

Female

Male

597

454

101.66

98.63

14.594

15.204

0.595

0.714

0.20

 

Races (4-7)

Female

Male

764

648

101.55

99.80

14.798

14.871

0.535

0.584

0.12

 

Rhymes (4-7)

Female

Male

744

625

101.23

98.75

13.013

13.097

0.477

0.524

0.19

 

Mobile phone (8-10)

Female

Male

1228

1122

102.47

98.89

14.742

15.196

0.421

0.454

0.24

 

Funny words (8-10)

Female

Male

1193

1128

100.36

99.31

14.846

15.450

0.430

0.460

0.07

 

Word chopping (8-10)

Female

Male

1101

1055

100.95

98.16

14.764

15.596

0.445

0.480

0.18

 

Mobile phone (11-15)

Female

Male

1346

1355

102.91

99.77

14.895

14.549

0.406

0.395

0.21

 

Non-words (11-15)

Female 

Male

1465

1484

101.46

99.84

15.015

14.885

0.392

0.386

0.10

 

Segments (11-15)

Female

Male

1290

1280

102.19

99.33

14.445

15.384

0.402

0.430

0.19

 

*Cohen’s d is a measure of effect size of the difference between two means

Checks were also made for ethnic group bias. Due to the small numbers in some ethnic minority groups, comparisons were made between White students and Other ethnic groups combined (where ethnicity had been identified) on each subtest (see Table 10). Small effects were found on Races and Rhymes, slightly favouring White students; and on Mobile phone (11–15), Non-words and Segments, favouring other ethnicities. There were no other ethnicity effects.

Table 10. Ethnic group differences

Subtest Ethnicity N Mean SD SE of Mean Cohen’s d*
Crayons (4-7)

White

Other

607

341

100.43

100.05

14.778

15.600

0.600

0.845

0.03

 

Races (4-7)

White

Other

802

473

102.37

99.21

14.840

14.618

0.524

0.672

0.21

 

Rhymes (4-7)

White

Other

775

457

101.25

98.07

12.801

13.412

0.460

0.627

0.24

 

Mobile phone (8-10)

White 

Other

1538

626

101.13

100.56

14.632

15.839

0.373

0.633

0.04

 

Funny words (Non-words) (8-10)

White

Other

1494

632

100.13

99.65

15.215

15.152

0.394

0.603

0.03

 

Word chopping (Segments) (8-10)

White 

Other

1381

589

100.46

98.13

15.010

15.520

0.404

0.639

0.15

 

Mobile phone (11-15)

White

Other

2489

407

100.90

104.28

14.789

14.865

0.296

0.737

0.23

 

Non-words (11-15)

White

Other

2660

484

100.16

103.86

14.907

14.342

0.289

0.652

0.25

 

Segments (11-15)

White

Other

2352

411

100.31

103.40

14.828

14.923

0.306

0.736

0.21

 

*Cohen’s d is a measure of effect size of the difference between two means