7.1
THE STUDY
The purpose of this study was to investigate changes in children’s achievement and attitude over time using data from the first and second cycles of NEMP assessment in mathematics. The investigation involved classifying the task components in each cycle according to their relationship to the achievement objectives in the mathematics curriculum and according to the nature of the skills required for their completion.
These classifications provided useful information concerning the coverage in the NEMP assessments of the skills, knowledge and understandings within the various curriculum strands.
The nature of the changes in achievement was investigated using the task components which were common to both the 1997 and the 2001 assessments. Those components for which the difference in performance between the two years was greatest were identified and patterns of change suggested.
The changes in attitude were identified using the common questions in the mathematics surveys of the two assessments.
While the data indicated that there was a good spread of achievement objectives tested at levels 2 and 3 of the curriculum, the coverage at level 4 was weaker. This was emphasised when the year 4 and year 8 results were separated.
The curriculum document indicates that we might expect the majority of year 4 students to be working at about level 2 and the year 8 students at about level 4. The year 4 results were consistent with this. 61% of the year 4 tasks were judged to involve level 2 achievement objectives and the mean level of achievement objectives over all tasks was 2.4. However, at year 8, only 17% of the tasks involved level 4 achievement objectives and the mean level of achievement objectives over all tasks was 2.8. This must bring into question how well the NEMP assessments are monitoring the achievement of year 8 students.
The situation clearly arises as a consequence of the decision to include so many tasks which are common to both years. In the 2001 assessment, 71% of tasks were either identical for the two years or included some common components. The purpose of these tasks is to provide a measure of growth from year 4 to year 8. However, it does seem that an overemphasis on this, difficult to interpret, aspect of monitoring has compromised the objective of monitoring the achievement of year 8 students, particularly the more able ones.
The results indicated that there was a wide coverage of skills in the assessment tasks except, perhaps, for justifying and proving.
The distribution of skills over the tasks was very similar at year 4 and year 8. This might be seen as reinforcing the view in the previous section that the year 8 students, particularly the more able ones are being under-assessed. One could argue that the emphasis should be moving from the lower level skills of knowing and routine procedures at year 4 towards the higher level complex procedures, solving problems and justifying and proving at year 8.
The gains outnumbered losses in each of the content areas, at each of the achievement objective levels, and in each of the performance expectation groups.
Gains easily outnumbered losses in Number and in Algebra and Statistics, but in the combined results for Measurement and Geometry there were 20 losses and only one gain.
Losses outnumbered gains at each of the achievement objective levels and there were no performance expectations for which the gains outnumbered the losses.
At both year levels there was a small, but consistent, decline in enthusiasm and a small overall increase in self-confidence.
One result, which reflects well on teachers, was that more students in both years felt that they knew how good their teachers thought the students were at mathematics.
The principal emphasis in the new mathematics curriculum is a problem solving approach to the teaching of mathematics and one of the motivations of this study was to see if there was any evidence in the NEMP reports of progress in implementing this aspect of the curriculum. There is not a great deal of evidence in this study to answer this question, but what there is indicates that at year 4 the 1997 students performed better on problem solving tasks than the 2001 students. The opposite is true at year 8.
There was certainly an increase in achievement on numeracy tasks over the period, both at year 4 and year 8.
There is insufficient evidence to be sure that at year 8 the better performance in numeracy tasks has come at the expense of problem solving skills, but perhaps warning bells should be ringing.
7.7.1 Year 4
It is possible to argue, of course, that numeracy is of such prime importance that it is worth sacrificing something to achieve it. For less able students this may well be true. However, there is a danger that the more able students are being disadvantaged by a change of emphasis in the classroom. The NEMP assessments are perhaps ringing alarm bells that this may be the case.
When the achievement of year 8 students is being considered it is important to recognise that only 20% of the tasks on which they were assessed were at curriculum level 4 or higher. This is the level around which the curriculum document suggests most of the students should be working. The mean curriculum level of the year 8 tasks was 2.8, which centres the tasks at about the year 5 - year 6 level. This reflects the emphasis in the assessments on tasks which are attempted by both year 4 and year 8 students. There is, of course, nothing necessarily wrong with the tasks being aimed at this level, and it is consistent with the NEMP policy of staying relatively independent of the details of current curriculum statements. However, one consequence of the policy is that the assessments at year 8 do not do a good job of monitoring the achievement of the more able students. It is also possible that the assessments give a false impression to teachers and others of what might be expected of year 8 students.
There was little change between 1997 and 2001 at year 8 in student attitudes to mathematics. The subject did drop from third to fourth in the favourite subject ranking, but it remains popular with students.
Garden, R. (ed). (1997). Mathematics and science performance in middle
primary school. Wellington: Ministry of Education.
Knight, G. (2001). Analysis of achievement data from NEMP and international studies and implications for action. A report to the Ministry of Education.
Ministry of Education, (1992). Mathematics in the New Zealand curriculum. Wellington: Ministry of Education.
Ministry of Education. (2003). Numeracy Professional Development Projects. Book 1: The number framework. Wellington: Ministry of Education.
NEMP, (1998). Mathematics assessment results 1997. Dunedin: EARU University of Otago.
NEMP, (2002). Mathematics assessment results 2001. Dunedin: EARU University of Otago.
Thomas, G.,
Tagg, A. and Holton, D. (2003) Mathematics curriculum framework and
map: Levels 2 - 6. Technical Report 36, Project asTTle, University
of Auckland. (Draft Copy).
