The major purpose of this probe study is to analyse the video recordings of students oral reading assessments with a focus on their comprehension performance.

A random sample of nine Year Four students and ten Year Eight students has been taken for analysis from the Year 2000 Oral Reading Assessments.

Each student reads two passages in the NEMP testing. The starting point is determined by the test administrator using the Band indicator. In this probe exercise each of the passages read is viewed carefully and replayed as many times as is necessary to make a very accurate recording of the students reading behaviour. A complete running record was made of each reading using standard conventions. The asking of the comprehension questions was observed and marked. As well as scoring the word accuracy, self correction rates and comprehension a descriptive comment was made about the students reading behaviour and observable features not part of the running record. Observations were also recorded of the teacher administrators handling of the task.

The following questions have guided this exercise:

• Does comprehension performance reach the same levels as students word accuracy (decoding) assessments?
• Are self correction behaviours a feature of reading strategies?
• Is the oral reading assessment a suitable task for the purpose of measuring comprehension?
• Is there any significant difference in achievement between the range of question types (literal to higher order)?
• Were the teacher administrators adequate in:
  a) drawing attention to the comprehension aspect of the ask
  b) engaging with the students when giving instructions or asking questions
  c) rewording, repeating or prompting?
• Does an analysis of the passage independent questions show that it is reasonable to expect students to be able to give an adequate response as part of higher order comprehension without relying on specialised factual knowledge outside of the texts?

Over the years various conventions have been used to determine what is a satisfactory rate of comprehension accuracy. Informal Prose Reading Inventories have along been a popular assessment tool and often 60% (three out of five questions) accuracy was regarded as an indication of adequate understanding. In the Literacy Experts Report (1999) it states that there is some research support for requiring children to answer, on average 75% of appropriate questions on a comprehension task.

In this study children achieving at two cut off points will be shown. The lower end of adequate understanding will be indicated by 60% achievement and the lower end of very good understanding by 80% achievement.

For decoding (word accuracy) the standard convention as used in Reading Recovery is:

Where reference is made to word accuracy levels in this study the above convention is used.

There are a number of aspects to comprehension. Generally these are placed in three areas:

next page