| ACKNOWLEDGEMENTS |
 |
The Project
directors acknowledge the vital support and contributions of many people
to this report, including:
• the very dedicated staff of the Educational Assessment Research Unit
• Heleen Visser, Hadyn Green, Nicola Durling and other staff members of the Ministry of Education
• members of the Project’s National Advisory Committee
• principals and children of the schools where tasks were trialled
• principals, staff and Board of Trustee members of the 248 schools included in the 2007 sample
• the 2877 children who participated in the assessments and their parents
• the 96 teachers who administered the assessments to the children
• the 44 senior tertiary students who assisted with the marking process
• the 170 teachers who assisted with the marking of tasks early in 2008.
| SUMMARY |
|
| Overview : |
Year 4 and year 8 students show a basic ability to read and interpret graphs, tables and maps, with year 8 students showing marked improvement over year 4 students. Tasks that involve interpreting information, combining information, or using information to make inferences or deductions are particularly difficult for year 4 students. Many year 8 students are skilled at producing or completing graphs and tables, with year 4 students having substantially more difficulty in this area. When a task requires the construction of a graph or table without a model to work from, many students at both years have trouble. Overall performance levels were very similar in 2007 and 2003, and differences between demographic subgroups had not changed substantially. On average, across the range of 2007 tasks at both year levels, differences in performance between boys and girls were very small; Pakeha students
|
|
| performed better than Mäori students by moderate to large margins; Pakeha students performed better than Pasifika students by large margins; and students for whom the predominant language at home was English performed slightly to moderately better than other students. |
|
| The NEMP Approach to National Monitoring |
|
New Zealand’s National Education Monitoring Project (NEMP) commenced in 1993, with the task of assessing and reporting on the achievement of New Zealand primary school children in all areas of the school curriculum. Children are assessed at two class levels: year 4 (halfway through primary education) and year 8 (at the end of primary education). Different curriculum areas and skills are assessed each year, over a four-year cycle. The main goal of national monitoring is to provide detailed information about what children can do so that patterns of performance can be recognised, successes celebrated and desirable changes to educational practices and resources identified and implemented.
Each year, random samples of children are selected nationally, then assessed in their own schools by teachers specially seconded and trained for this work. Task instructions are given orally by teachers, through video presentations, on laptop computers, or in writing. Many of the assessment tasks involve the children in the use of equipment and supplies. Their responses are presented orally, by demonstration, in writing, in computer files, or through submission of other physical products. Many of the responses are recorded on videotape for subsequent analysis.
The use of many tasks with both year 4 and year 8 students allows comparisons of the performance of year 4 and 8 students in 2007. Because some tasks have now been used twice, in 2003 and again in 2007, trends in performance across the four-year period can also be analysed. We have also included in this report results from a re-administration of two tasks from 1995.
| Assessing Skills in the Use of Graphs, Tables and Maps |
|
In 2007, the first year of the fourth cycle of national monitoring, three areas were assessed: science, art, and the use of graphs, tables and maps. This report presents details and results of the assessments of students’ skills in the use of graphs, tables and maps. Understanding and using information presented in the form of graphs, tables and maps is an important part of everyday life in our community. This report highlights two aspects of the use of graphs, tables and maps: extracting and interpreting information, and constructing or completing graphs, tables and maps.
| Interpretation of Graphs, Tables and Maps |
|
| Chapter 3 focuses on extracting and interpreting information from a wide variety of graphs, tables and maps. Averaged across 130 task components used with both year 4 and year 8 students, 65% of year 8 students produced correct responses compared to 51% of year 4 students. This indicates that, on average, students have made good progress between year 4 and year 8 in the skills assessed by the tasks. Year 8 students were quite good at reading basic information from graphs, and most performed well when the graph called for calculations or finding information that involved a more sophisticated use of the graph (such as combining information or making deductions from graphical information). Most year 4 students had difficulty when asked to do anything more than a straightforward reading of the information from the graph. |
Thirteen trend tasks involving 64 task components were administered to year 4 students in both the 2003 and 2007 assessments. Averaged across all 64 components, 1% fewer students succeeded in 2007 (46%) than in 2003 (47%). Thus, performance stayed almost the same over the four-year time span. Two tasks, with a total of 10 components, were administered in 1995 and 2007, allowing for a 12-year look at change. Averaged across these 10 components, students in 2007 were successful 65% of the time, compared to 58% of the time in 1995. Thus, we see a gain over the 12-year period on these two tasks.
Fifteen trend tasks involving 72 task components were administered to year 8 students in both the 2003 and 2007 assessments. Averaged across all 72 components, students in 2007 were successful at a rate of 68%, compared to 71% in 2003. Thus, a small decline is seen in performance over the four-year period. Two tasks, with a total of 10 components, were administered in 1995 and 2007, allowing for a 12-year look at change. Averaged across these ten components, students in both 2007 and 2003 were successful 86% of the time, indicating no change over the 12-year period on these two tasks. |
|
| Constructing or Completing Graphs, Tables and Maps |
|
|
The focus of Chapter 4 is constructing or completing graphs, tables and maps. Performance in graphs, tables, and maps shows strong growth from year 4 to year 8 overall. Averaged across 64 task components used with both year 4 and year 8 students, 80% of year 8 students produced correct responses compared to 50% of year 4 students. Thus, on average 30% more year 8 students than year 4 students successfully responded to task components. Furthermore, many students at year 8 showed a mastery of the skills necessary to produce graphs.
The biggest weakness for both years was the lack of appropriate labels and titles to accompany the graphs. Students also had trouble in developing a graph or table if they were not given some suggestion as to what the display should look like. If they had to do all of the layout and design, they were far less successful than if they were completing work that was already started. The difficulty appears to be in the ability to conceptualise what the finished product might look like as opposed to working from a model.
|
Six trend tasks involving 40 components were administered at year 4 in 2003 and again in 2007. The average successful completion rate on these tasks was 46% in 2003 and 45% in 2007. Thus, performance in the two years is very similar. Seven trend tasks involving 52 components were administered at year 8 in both 2003 and 2007. The average success rate in 2003 was 72% and in 2007 the comparable figure is 70%. Again, performance across the four-year span is quite similar.
| Performance of Subgroups |
|
Chapter 5 reports the results of analyses that compared the performance of different demographic subgroups. School type (full primary school, intermediate school, or year 7 to 13 high school), geographic zone, community size and school size produced few statistically significant differences among subgroups, and no strong trends were seen in the data. Furthermore, gender differences were very small, with less than ten percent of tasks at either year level showing statistically significant differences between the performance of boys and girls (with some tasks favouring boys but others, girls). There were, however, stronger differences by socio-economic status (SES), ethnicity and home language.
With regard to SES, we used the Ministry of Education decile system to form three SES levels for schools: deciles 1-3 formed the lowest group, deciles 4-7 formed the middle group, and deciles 8-10 formed the highest group. There were statistically significant differences in the performance of students from low, medium and high decile schools on 80% of the tasks at year 4 level and 75% of the tasks at year 8 level. The corresponding figures in 2003 were 51% and 73%, in 1999 were 52% and 84%, and in 1995 were 67% and 60%.
For comparisons of Pakeha students with Mäori students, Pakeha with Pasifika students, and students for whom the predominant language at home was English with those for whom it was not, effect sizes were used. Effect size is the difference in mean (average) performance of the two groups, divided by the pooled standard deviation of the scores on the particular task.
In general, Pakeha students performed better than Mäori students. For year 4 students, the mean effect size across 40 tasks was 0.38 (Pakeha students averaged 0.38 standard deviations higher than Mäori students). This is a moderate to large difference. For year 8 students, the mean effect size across the 45 tasks was 0.42. This is also a moderate to large difference. The corresponding effect sizes in earlier assessments were similar or slightly smaller: 0.33 for year 4 students and 0.40 for year 8 students in 2003, and 0.33 and 0.33 in 1999.
|
The results for comparisons of Pakeha and Pasifika students show larger disparities. For year 4 students, the mean effect size across the 40 tasks was 0.59 (Pakeha students averaged 0.59 standard deviations higher than Pasifika students). This is a large difference. For year 8 students, the mean effect size across the 40 tasks was 0.56. This also is a large difference. The corresponding mean effect sizes in the 2003 and 1999 assessments were also large: 0.50 for year 4 students and 0.70 for year 8 students in 2003, and 0.59 and 0.66 in 1999. The disparity for year 4 students has not changed over eight years, but at year 8 level the disparity has decreased a little. |
Compared to students for whom the predominant language at home was English, students from homes where other languages predominated scored lower, on average, with mean effect sizes of 0.21 for year 4 and 0.15 for year 8. These are small to moderate disparities, substantially smaller than the moderate disparities in the 2003 assessments (0.35 for year 4 students and 0.27 for year 8 students) and in the 1999 assessments (0.37 and 0.31).
| Summary of Performing Trends |
|
An indication of overall trends in performance across the four-year period between 2003 and 2007 can be obtained by looking at the patterns of change across all of the trend tasks. Averaged across 104 components of the year 4 trend tasks, 1% fewer students succeeded in 2007 than in 2003. Averaged across 124 components of the year 8 trend tasks, 2% fewer students succeeded in 2007 than in 2003. The report on the 2003 graphs, tables and maps assessments reported trends between 1999 and 2003, with an average loss over that four-year period of 4% on year 4 trend task components and a decline of 1% on year 8 trend task components. The report on the 1999 graphs, tables and maps assessments reported trends between 1995 and 1999, with an average gain over that four-year period of 6% on year 4 trend task components and a decline of 1% on year 8 trend task components. Taken together, these three sets of trend results suggest little change in performance overall, at either year level, for the twelve year period from 1995 to 2007.
|
