Reading

Low scores for Exact word recognition or reading comprehension or reading comprehension speed can help to identify students who have reading difficulties that may require support in examinations. When assessing larger groups of students, a time-saving strategy would be to administer Exact to all students. As difficulties with reading are sometimes not as apparent in the classroom as difficulties with writing or speed of working, this has the benefit of highlighting students whose reading difficulties have not been identified previously.

 

Exact reading comprehension accuracy and word recognition can be used to screen for reading difficulties which may require support in examinations. However, assessment evidence in Part 2 of Form 8 is not required for a student to have the support of a reader in examinations.

The Exact reading comprehension speed is one of a very few tests which assess the time it takes a learner to read and actually understand a passage, as opposed to simply reading it the first time. The Joint Council for Qualifications ‘Access Arrangements and Readable Adjustments’ document (AARA) explains the difference between the two [2022-23, Section 7.5.10].  Slow reading speed is where a learner takes longer than expected to decode the words the first time they read – either aloud or silently. Slow reading comprehension speed is where a learner needs to re-read text several times in order to absorb its meaning. The latter is clearly a disadvantage in a timed assessment, and the Exact is a very useful tool that assessors can use to confirm the need for extra time where a candidate scores less than 85 on the reading comprehension speed subtest.

 

Exact has quite a wide range of applications in addition to assessment for exam access arrangements, e.g.

1. Exact is appropriate for assessing students with specific learning difficulties in, secondary, further or higher education, or for teachers wishing to obtain a standardised objective assessment of literacy for groups of students from ages 11-24, or for individual students within that age range who have specific problems (such as slow handwriting, spelling or reading comprehension).
2. Although individual tests from Exact may be helpful in suggesting dyslexia, or may form part of a dyslexia assessment, this group of tests are not sufficient in themselves to make a diagnosis of dyslexia and are not designed for that purpose. Administrators who require a test that will screen for dyslexia should consider using LASS 11-15 (for the age range 11:0 – 15:11) or LADS / LADS Plus (for ages 16 and upwards).
3. Exact has two forms of equivalent difficulty – Form A and Form B. This allows for repeated assessment if desired, without undue concern about practice effects and without violating psychometric principles. The two forms can be alternated over time in order to record progress, e.g. in response to intervention given to students with literacy difficulties.

The program checks whether the student has devoted a reasonable amount of time to the

reading comprehension passages. If a student has completed the reading comprehension test in less than eight minutes the results should be regarded as ‘doubtful’, i.e. it is unlikely that proper consideration has been given to the answers, and hence the scores will be unreliable and should not (on their own) be used as meaningful evidence for exam access arrangements.

If a student completes the reading comprehension test in less than five minutes, the results should be regarded as ‘impossible’, i.e. the student has answered the comprehension passages so quickly that it is impossible for them to have given proper consideration to the answers, and hence the scores are not safe to be used as evidence for any purpose (see Section 3.1.2 of the Exact Manual for guidance on this).

Reading comprehension is a complex skill that depends on many cognitive sub-skills, including word recognition, vocabulary knowledge, inferential thinking and working memory, to name but a few. During the process of reading the nature of the task and the complexity of the material to be read interact with limitations imposed by the varying degrees of competence with which the component sub-skills can be executed by the reader. The outcome affects both the accuracy andspeed of how we can read with understanding.

Each person does not necessarily read at the same speed all the time: speed depends on a number of factors, including the difficulty of text content for that individual, complexity of the grammar, familiarity of the vocabulary, physical/environmental conditions such a lighting, font size and distractions, as well as personal factors such as purpose, motivation, interest and tiredness. One may liken it to driving a car: the speed at which we drive is dependent onfactors such as road and weather conditions, amount of traffic, speed restrictions, purpose of the journey and whether we are late. These factors can vary continuously throughout a journey, allowing us to speed up or slow down as required, just as we do when reading.

Given these complex factors it can be appreciated that assessing reading comprehension ability is far from straightforward. In order to explain how Exact measures reading comprehension speed, it is first worthwhile considering the disadvantages of other methods (see Table 1).

 

Table 1. Disadvantages of various method of measuring reading speed.

Method	Problems / Disadvantages
Measure the time taken to read a list of wordsaloudand convert into words per minute (wpm).	Not a true measure of readingspeed. What is actually being measured is a combination of decoding speedand pronunciation speed.What if the person does not recognise a word or cannot pronounce a word correctly?
Measure the time taken to read a passage of connected textaloudand convert to wpm.	Same as above.Not the normal way in which people read text.
Measure the time taken to read a passage of connected textsilentlyand convert to wpm.	Cannot be sure the person has read every word and not skipped over some.Cannot be certain how much of the text has been understood.
Measure the time taken to read a passage of connected textsilentlyand convert to wpm, and then ask them to answer comprehension questions.	Difficulty of factoring comprehension accuracy into the calculation in such a way that the measure of reading speed does not become actually a measure of reading comprehension accuracy.What happens if the person gets a lot of the comprehension questions wrong?

 

It can be appreciated that there are many factors which make the task of measurement of reading comprehension speed tricky. When reading aloud (which in itself is problematic because it is not the usual way in which people read) pronunciation speedbecomes a major influence; not only can some people speak faster than others, but longer words require more time to pronounce, so the person’s reading speed will appear slower when tackling longer words. When a person is reading silently, however, we cannot be sure they are reading every word and not skipping some, and unless we check we do not know whether the person has understood what they have read. With any reasonably complex texts and over a certain basic speed, there is a broad trade-off between reading speed and comprehension: the faster you read the less you are likely to understand.

To illustrate the consequences of this fact for the process of assessment, say a person reads a passage of 240 words silently in two minutes and then answers 10 questions about it, and they get six questions correct. How shall we most accurately calculate their reading speed? We could simply divide 240 by 2 to get 120 wpm, but since they only got six questions right how we cannot be sure whether they actually read the whole passage properly. Unfortunately, if we try to deal with that by taking 60%of 120, i.e. 84 wpm as their ‘true’ rate of reading on the basis that they only understood 60% of the passage, we are left with a measure that is much a function of reading comprehension accuracy as it is of reading speed. Conversely, our ‘direct’ measure of reading comprehension accuracy (i.e. 60%) is also as much a function of reading speed because we do not know whether the person would have obtained a better score if they had taken more time to read the passage. Although measures of reading comprehension accuracy and reading comprehension speed can never be completelyindependent of one another, it would be downright misleading to have two measures (accuracy and speed) that are purported to be different and distinct but which actually turn out to be measuring more-or-less the same thing.

Most conventional tests of reading comprehension have a fixed time limit. Timed tests of reading comprehension have been shown to have particular value over untimed tests when assessing student and adults, because untimed tests fail to distinguish adequately between better readers and poorer readers, whereas the former do not have this limitation (Lesaux, Pearson & Siegel, 2006). However, these tests inevitably yield a score that is a product of both reading comprehension accuracy and reading comprehension speed, and problems of interpretation of results can arise. For example, if person A scores, 60% having taken all the time available on a test of reading comprehension, and person B also scores 60% having taken only 80% of the time available on the same test, it suggests that B is a faster reader than A, but that will not be apparent from their score, which assumes that everyone uses the sameamount of time.

 

How Exact overcomes these various limitations

Exact overcomes the limitations of other methods by using a complex algorithm that factors into the calculations not only time taken but also the number of questions attempted and the proportion of those questions correct, which are proxy measures of the amount of the text that has been processed. The algorithm ensures a reasonable degree of separation between the two key measures of reading comprehension accuracy and reading comprehension speed. The success of this may be judged from the results of the validation of Exact, in which, among other things, Exact was compared with the Edinburgh Reading Test (a timed test that is widely used for exam access assessments). The findings werethat Exact Reading Comprehension Accuracy score and Exact Reading Comprehension Speed score both correlated very highly with Edinburgh Reading Test score (showing that Exact is as good as the Edinburgh Reading Test for assessing reading comprehension) butalso that the correlations with Edinburgh Reading Test were both higher than the correlation between the two scores derived from Exact Reading Comprehension. This shows that Exact achieves a satisfactory separation between the two key measures of reading comprehension accuracy and reading comprehension speed

Reference:
Lesaux, N.K., Pearson, M.F. & Siegel, L.S. (2006) The effects of timed and untimed testing conditions on the reading comprehension performance of adults with reading disabilities. Reading and Writing, 19, 21-48.

 

The results from all the tests in Lucid Exact are based on standardised norms derived from a representative national sample, and are for the whole of each test. In other words, failing to answer all the questions in the reading comprehension test undoubtedly affects the outcome, but that does not necessarily invalidate the results.

By checking the time the student has devoted to the task and the number of questions attempted the administrator can determine whether or not the student made a serious attempt or not, and consequently how the results should be treated.

• If the time taken on the test was between eight and ten minutes then the result is will generally to be an accurate reflection of the student’s reading comprehension skills.
• If less than eight minutes but not less than five minutes the result could be an acceptable reflection of the student’s reading comprehension skills but the administrator should to bear in mind that it could be unreliable and so further investigation will be necessary.
• If less than five minutes the result is almost certainly unreliable, in which case the administrator would be well advised to repeat this test having provided appropriate guidance to the student regarding how the test should properly be attempted (see Section 3.1.2 of the Lucid Exact Administrator’s Manual for further advice on this matter).

 

The short answer to this question is yes, probably. Reading comprehension accuracy and reading speed are not entirely independent measures and never can be in any testthat involves reading. However, the picture is more complicated than it first appears. The slower the student’s reading speed, the less of the text and questions that can be read in the time allowed, so slow reading speed can obviously affect the reading accuracy in a standardised test. But, equally,a very fastreading speed can also negatively affect the accuracy score. Because mental effort has to bedevotedto understandingwhat isread and this takes a certain amount of time, it is generally the case that, within certain limits, reading faster tends to result in poorer comprehension and reading slower tends to result in better comprehension. There are exceptions, of course. If a student reads exceptionally slowly–e.g. they spent the whole 10 minutes just on the first one or two passagesof the reading comprehension test –the results will, quite rightly, give a slow speed of reading and a relatively poor comprehension score because almost all students aged 11 and older normally do much better than this. At the other end of the scale, avery bright,skilledreadermaycomplete all the passageswithin the 10 minute periodand get most of the questions correct, obtainingboth a fast reading speed and a high accuracy score. In general, however, most studentslie somewhere in between these two extremes.

There could be several different reasons why a student scores poorly both on reading speed and on reading comprehension accuracy. Working exceptionally slowly (as in the example above) could be one reason. Another reason could be that they skipped through the passages, finishing the test early without having attempted a reasonable number of questions or just guessed. Another reason could be that despite devoting reasonable time and effort to the task the student’s limited understanding meant that after the first couple of passages they were unable to make much sense of the rest. Examination of the student’s performance pattern (this is shown on page 2 of the Exact report) passage-by-passage, including time taken, should enable the administrator to determine which is the most likely cause.

The usual pattern of performance by a student who has made a fair and conscientious effort in the test is revealed in a greater number of attempts and higher scores in the earlier (easier) passages, gradually tailing off to fewer attempts and lower scores in the later (harder) passages.The time taken is usually around one or two minutes in the first two passages, increasing to three minutes in the third and fourth passages. How much time is taken by the fifth and final passage will generally depend on what time is left, but is often only around a minute, except for older or brighterstudents, who are able to master the earlier passages more speedily and so have more time available to devote to the final passage.

If an administrator has misgivings about any results from the reading comprehension test the best course of action is to repeat thetest using the other form, after first having provided appropriate guidance to the student regarding how the test should properly be attempted (see Section 3.8 of the Exact Administrator’s Manual for further advice on this matter).