Validity of Lucid Recall

What is validity?

Validation of a psychological or educational test is not the same thing as the psychometric standardisation of a test, nor should it be confused with the reliability of a test: ‘reliability’ generally refers to the extent to which a test can be expected to give the same results when administered on different occasions or by a different administrator, or the extent to which the components of a test give consistent results (see Section 1.5.1). ‘Validity’ is a measure of the extent to which the test measures what it is supposed to measure (e.g. reading or spelling ability). Validity is usually established by comparing the test with some independent criterion or with a recognised test of the same ability. Inevitably, this raises the thorny issue of what is the ‘gold standard’ – i.e. which is the ‘best’ measure of any given ability against which all other should be compared? Professional opinions differ as to the merits of various tests and consequently there are no generally agreed ‘gold standards’ for assessing reading, spelling and writing. Hence the conventional method of establishing test validity is to show that a new test produces results that agree reasonably closely with well-established test(s) of the same ability.

External validity of Lucid Recall was explored in three ways. Firstly, analyses were used to examine the relationships between performance on Lucid Recall and children’s attainment in school. Secondly, the Lucid Recall profiles of children with special educational needs were examined. Finally, convergent validity was explored by examining the relationships between scores on Lucid Recall and scores on the Working Memory Rating Scale (Alloway, Gathercole & Kirkwood, 2008). 

Children's school attainment

Several previous studies using measures of the multiple-component model of working memory have demonstrated close relationships between working memory and children’s attainment in school (e.g. Gathercole et al., 2003; Gathercole & Pickering, 2000; Gathercole et al., 2004; Jarvis & Gathercole, 2003; St Clair-Thompson & Gathercole, 2006). These have typically used formal assessments of children’s progress on the National Curriculum. However, policy changes have recently resulted in schools carrying out less testing. Instead, teachers are expected to record each child’s progress each academic term, measured by tasks and tests that are administered informally. Teacher ratings of the National Curriculum levels of 337 children (166 males and 171 females) from the standardization sample were therefore obtained from the schools at the time of testing. Table 5 shows the correlations between the three Lucid Recall subtests and children’s National Curriculum levels at aged 7, 8, 9, 10 and 11. Consistent with previous research using other working memory tasks, there were statistically significant correlations between scores on Lucid Recall and children’s scholastic attainment (see Table 4).

Children with Special Educational Needs

Research has suggested that performance on working memory measures can be used to accurately identify children who are likely to require special educational provision. For example, Gathercole and Pickering (2001) compared the working memory profiles of children with special educational needs to the profiles of children without special educational needs. Children with special educational needs performed significantly more poorly on measures of the central executive, and one measure of the visuo-spatial sketchpad.

Two schools who participated in the standardisation were therefore asked to supply the names of children recognised by the school or local education authority as having special educational needs. There were 37 such children. Their performance on Lucid Recall, compared to 46 age-matched children from the same schools, is shown in Figure 1. Consistent with previous studies children with special educational needs performed significantly poorer than children without special educational needs on the pattern recall task, F (1, 92) = 12.23, p = 0.001, and the counting recall task, F (1,83) = 11.27 p = 0.001. The difference between the two groups was not statistically significant for word recall, F (1,91) = 1.10, p = 0.30.

 

Table 4. Correlations between Lucid Recall scores and National Curriculum Levels.

Note: ** correlation is significant at the p<0.01 level, * correlation is significant at the p>0.05 level (2 tailed test). Figure 1. Profiles of children with and without special educational needs. 0 5 10 15 20 25 30

Figure 1. Profiles of children with and without special educational needs.

Scores on the Working Memory Rating Scale

Researchers have suggested that there are a number of behaviours typically associated with a poor working memory. These include, for example, children losing their place in complex tasks with multiple steps, or requiring regular repetition of instructions in the classroom. Such behaviours can be examined using the Working Memory Rating Scale (Alloway, Gathercole, & Kirkwood, 2008). Using this scale teachers are asked to rate how typical each behaviour is of a child using a four point scale. Cognitive assessments of children’s working memory have been found to be significantly related to teacher ratings on the Working Memory Rating Scale (e.g. Alloway, Gathercole, Kirkwood, & Elliott, 2009; St ClairThompson, 2011). Therefore to further establish the validity of Lucid Recall the relationships between scores on each subtest and scores on the Working Memory Rating Scale were explored. Ratings were obtained for 51 children (27 males and 24 females) aged 7 years of age. The correlations are shown in Table 5.

 

Table 5. Correlations between scores on Lucid Recall and ratings on the Working Memory Rating Scale.

Note: ** correlation is significant at the .01 level.

Performance on each memory subtest was significantly negatively related to teacher ratings on the Working Memory Rating Scale. Negative correlations are expected because on the Lucid Recall subtests a higher score indicates a better working memory, whereas on the Working Memory Rating Scale a higher score indicates more problematic behaviours.

In summary, Lucid Recall has good validity. Correlations revealed that the scores on Lucid Recall are significantly related to children’s academic performance across the childhood years. Children with special educational needs performed significantly more poorly than agematched controls on the pattern recall and counting recall subtests. Finally, scores on Lucid Recall were significantly related to teacher ratings of children’s behaviour on the Working Memory Rating Scale (Alloway et al., 2008). It can therefore be concluded that Lucid Recall is a valid assessment of children’s working memory.