These pages present a concise outline of the rationale and operating procedures for national monitoring, together with some information about the reactions of participants in the 2009 assessments. Detailed information about the sample of students and schools is available in Sample of Schools and Students in 2009 (Appendix).

Purpose of national monitoring
The New Zealand Curriculum Framework (1993, p26) states that the purpose of national monitoring is to provide information on how well overall national standards are being maintained, and where improvements might be needed.

The focus of the National Education Monitoring Project (NEMP) is on the educational achievements and attitudes of New Zealand primary and intermediate school children. NEMP provides a national “snapshot” of children’s knowledge, skills and motivation, and a way to identify which aspects are improving, staying constant or declining. This information allows successes to be celebrated and priorities for curriculum change and teacher development to be debated more effectively, with the goal of helping to improve the education which children receive.

Assessment and reporting procedures are designed to provide a rich picture of what children can do and thus to optimise value to the educational community. The result is a detailed national picture of student achievement. It is neither feasible nor appropriate, given the purpose and the approach used, to release information about individual students or schools.

Monitoring at two class levels   
National monitoring assesses and reports what children know and can do at two levels in primary and intermediate schools: year 4 (ages 8-9) and year 8 (ages 12-13).

National samples of students   
National monitoring information is gathered using carefully selected random samples of students, rather than all year 4 and year 8 students. This enables a relatively extensive exploration of students’ achievement, far more detailed than would be possible if all students were to be assessed. The national samples of 1320 year 4 children and 1320 year 8 children represent about 2.2% of the children at those levels in New Zealand schools, large enough samples to give a trustworthy national picture.

Three sets of tasks at each level   
So that a considerable amount of information can be gathered without placing too many demands on individual students, different students attempt different tasks. The 1320 students selected in the sample at each year level are divided into three groups of 440 students, comprising four students from each of 110 schools. Each group attempts one third of the tasks

Timing of assessments   
The assessments take place in the second half of the school year, between August and November. The year 8 assessments occur first, over a five-week period. The year 4 assessments follow, over a similar period. Each student participates in about four hours of assessment activities spread over one week.

Specially trained teacher administrators   
The assessments are conducted by experienced teachers, usually working in their own region of New Zealand. They are selected from a national pool of applicants, attend a week of specialist training in Wellington led by senior Project staff and then work in pairs to conduct assessments of 60 children over five weeks. Their employing school is fully funded by the Project to employ a relief teacher during their secondment.

Four year assessment cycle   
Each year, the assessments cover about one quarter of the areas within the national curriculum for primary schools. The New Zealand Curriculum Framework is the blueprint for the school curriculum. It places emphasis on seven essential learning areas, eight essential skills and a variety of attitudes and values. National monitoring aims to address all of these areas, rather than restrict itself to pre-selected priority areas.

The first four-year cycle of assessments began in 1995 and was completed in 1998. The second cycle ran from 1999 to 2002.
The third cycle began in 2003 and finished in 2006. The fourth cycle began in 2007. The areas covered each year and the reports produced are listed opposite the contents page of this report.

Approximately 45% of the tasks are kept constant from one cycle to the next. This re-use of tasks allows trends in achievement across a four-year interval to be observed and reported.

Important learning outcomes assessed   
The assessment tasks emphasise
aspects of the curriculum which are particularly important to life in our community, and which are likely to be of enduring importance to students. Care is taken to achieve balanced coverage of important skills, knowledge and understandings within the various curriculum strands, but without attempting to follow the finer details of current curriculum statements. Such details change from time to time, whereas national monitoring needs to take a long-term perspective if it is to achieve its goals.

Wide range of task difficulty   
National monitoring aims to show what students know and can do. Because children at any particular class level vary greatly in educational development, tasks spanning multiple levels of the curriculum need to be included if all children are to enjoy some success and all children are to experience some challenge. Many tasks include several aspects, progressing from aspects most children can handle well to aspects that are less straightforward.

Engaging task approaches   
Special care is taken to use tasks and approaches that interest students and stimulate them to do their best. Students’ individual efforts are not reported and have no obvious consequences for them. This means that worthwhile and engaging tasks are needed to ensure that students’ results represent their capabilities rather than their level of motivation. One helpful factor is that extensive use is made of equipment and supplies which allow students to be involved in hands-on activities. Presenting some of the tasks on computer also allows the use of richer stimulus material and standardises the presentation of those tasks.

Positive student reactions to tasks   
At the conclusion of each assessment session, students completed evaluation forms in which they identified tasks that they particularly enjoyed, tasks they felt relatively neutral about and tasks that did not appeal. Averaged across all tasks in the 2009 assessments, 73% of year 4 students indicated that they particularly enjoyed the tasks. The range across the 124 tasks was from 95% down to 47%. As usual, year 8 students were more demanding. On average, 55% of them indicated that they particularly enjoyed the tasks, with a range across 171 tasks from 89% down to 31%. One task was more disliked than liked, by year 8 students only: a task involving finding information from a poster about New Zealand’s parliament.

Appropriate support for students   
A key goal in Project planning is to minimise the extent to which student strengths or weaknesses in one area of the curriculum might unduly influence their assessed performance in other areas. For instance, skills in reading and writing often play a key role in success or failure in paper-and-pencil tests in areas such as science, social studies, or even mathematics. In national monitoring, a majority of tasks are presented orally by teachers or on computer, and most answers are given orally or by demonstration rather than in writing. Where reading or writing skills are required to perform tasks in areas other than reading and writing, teachers are happy to help students to understand these tasks or to communicate their responses. Teachers are working with no more than four students at a time, so are readily available to help individuals.

To free teachers further to concentrate on providing appropriate guidance and help to students, so that the students achieve as well as they can, teachers are not asked to record judgements on the work the students are doing. All marking and analysis is done later, when the students’ work has reached the Project office in Dunedin. Some of the work comes on paper, but much of it arrives recorded on videotape. In 2009, about half of the students’ work came in that form, on a total of about 3250 videotapes. The video recordings give a detailed picture of what students and teachers did and said, allowing rich analysis of both process and task achievement.

Task approaches used   
Various task approaches are applied each year, depending on which areas of the curriculum are covered.

In 2009, four task approaches were used. Each student was expected to spend about an hour working in each format. The four approaches were:

• One-to-one interview: Each student worked individually with a teacher, with the whole session recorded on videotape.
• Stations: Four students, working independently, moved around a series of stations where tasks had been set up. This session was not videotaped.
• Team: Four students worked collaboratively, supervised by a teacher, on some tasks. This was recorded on videotape.
• Group and Independent: Four students worked collaboratively, supervised by a teacher, on one or two tasks. The students then worked individually on some paper-and-pencil tasks.

Professional development benefits for teacher administrators   
The teacher administrators reported that they found their training and assessment work very stimulating and professionally enriching. Working so closely with interesting tasks administered to 60 children in at least five schools offered valuable insights. Some teachers have reported major changes in their teaching and assessment practices as a result of their experiences working with the Project. Given that 88 teachers served as teacher administrators in 2009, or about 0.3% of all primary teachers, the Project is making a major contribution to the professional development of teachers in assessment knowledge and skills. This contribution will steadily grow, since preference for appointment each year is given to teachers who have not previously served as teacher administrators. The total after 15 years is 1365 different teachers, 108 of whom have served more than once.

Marking arrangements   
The marking and analysis of the students’ work occurs in Dunedin. The marking process includes extensive discussion of initial examples and careful checks of the consistency of marking by different markers.

Tasks which can be marked objectively or with modest amounts of professional experience usually are marked by senior tertiary students, most of whom have completed two or three years of pre-service preparation for primary school teaching. Forty-four student markers worked on the 2009 tasks, employed five hours per day for about four weeks.

The tasks that require higher levels of professional judgement are marked by teachers, selected from throughout New Zealand. In 2009, 160 teachers were appointed as markers. Most teachers worked either mornings or afternoons for one week. Teacher professional development through participation in the marking process is another substantial benefit from national monitoring. In evaluations of their experiences on a four-point scale (“dissatisfied” to “highly satisfied”), 70% to 96% of the teachers who marked student work in January 2010 chose “highly satisfied” in response to questions about:

• the instructions and guidance given during marking sessions
• the degree to which marking was professionally satisfying and interesting
• its contribution to their professional development in the area of assessment
• the overall experience.

Analysis of Results.
The results are analysed and reported task by task. Most task reports include a total score, created by adding scores for appropriate task components. Details of how the total score has been constructed for particular assessment tasks can be obtained from the NEMP office (earu@otago.ac.nz).

Although the emphasis is on the overall national picture, some attention is also given to possible differences in performance patterns for different demographic groups and categories of school. The variables considered are:

• Student gender:		Male	Female
• Student ethnicity:		Mäori	Pasifika	Päkehä (including Asian)
• Home language: (predominant language spoken at home)		English	any other language
• Geographical zone:		Greater Auckland	other North Island	South Island
• Size of community:		main centre over 100,000	provincial city of 10,000 to 100,000;	rural area or town of less than 10,000
• Socio-economic index for the school:		lowest three deciles	middle four deciles	highest three deciles
• Size of school:	Year 4 schools	less than 25 year 4 students	20-60 year 4 students	more than 60 year 4 students
	Year 8 schools	less than 35 year 8 students	35-150 year 8 students	more than 150 year 8 students
• Type of school (for year 8 sample only):		Full primary school	Intermediate school	year 7–13 high school (some students were in other types of schools, but too few to allow separate analysis)

Categories containing fewer children, such as Asian students or female Mäori students, were not used because the resulting statistics would be based on the performance of fewer than 70 children, and would therefore be unreliable.

An exception to this guideline was made for Pasifika children and children whose home language was not English because of the agreed importance of gaining some information about their performance.

Funding Arrangements.
National monitoring is funded by the Ministry of Education, and organised by the Educational Assessment Research Unit at the University of Otago, under the direction of Professors Terry Crooks and Jeffrey Smith. The current contract runs until June 2011. The cost is about $2.7 million per year, less than one tenth of a percent of the budget allocation for primary education. Almost half of the funding is used to pay for the time and expenses of the teachers who assist with the assessments as task developers, teacher administrators or markers.

Further information   
A more extended description of national monitoring, including detailed information about task development procedures, is available in: