Year 4 and Year 8 Samples
In 2008, 2867 children from 248 schools were in the main samples to participate in national monitoring. About half were in year 4, the other half in year 8. At each level, 120 schools were selected randomly from national lists of state, integrated and private schools teaching at that level, with their probability of selection proportional to the number of students enrolled in the level. The process used ensured that each region was fairly represented. Schools with fewer than four students enrolled at the given level were excluded from these main samples, as were special schools and Mäori immersion schools (such as Kura Kaupapa Mäori).

In late April 2008, the Ministry of Education provided computer files containing lists of eligible schools with year 4 and year 8 students, organised by region and district, including year 4 and year 8 roll numbers drawn from school statistical returns based on enrolments at 1 March 2008.

From these lists, we randomly selected 120 schools with year 4 students and 120 schools with year 8 students. Schools with four students in year 4 or 8 had about a 1% chance of being selected, while some of the largest intermediate (year 7 and 8) schools had a more than 90% chance of inclusion.

Pairing Small Schools
At the year 8 level, six of the 120 chosen schools in the main sample had fewer than 12 year 8 students. For each of these schools, we identified the nearest small school meeting our criteria to be paired with the first school. Wherever possible, schools with eight to 11 students were paired with schools with four to seven students, and vice versa. However, the travelling distances between the schools were also taken into account.

Similar pairing procedures were followed at the year 4 level. Here, two pairs of very small schools were included in the sample of 122 schools.

Contacting Schools
In the second week of May, we attempted to telephone the principals or acting principals of all schools in the year 8 sample. In these calls, we briefly explained the purpose of national monitoring, the safeguards for schools and students, and the practical demands that participation would make on schools and students. We informed the principals about the materials which would be arriving in the school (a copy of a 20-minute NEMP DVD, plus copies for all staff and trustees of the general NEMP brochure and the information booklet for sample schools). We asked the principals to consult with their staff and Board of Trustees and confirm their participation by the middle of June.

A similar procedure was followed at the end of July with the principals of the schools selected in the year 4 samples. They were asked to respond to the invitation within about three weeks.

Response from Schools
Of the 126 schools originally invited to participate at year 8 level, 119 agreed. Two paired schools with four students decreased to one or two students, and were not replaced because their paired school now had close to 12 students. A third paired school with eight students lost some students and was replaced by another small school from the same district. Two large intermediate or middle schools had major building work under way and could not find suitable accommodation for the assessments. Both were replaced by nearby schools of similar size and decile rating. One integrated college had a key personnel change affecting year 8 arrangements and was replaced by a school of similar character, size and decile rating. Finally, the principal of one independent school indicated that the school had more important priorities. It was replaced by another independent school with the same decile rating.

Of the 122 schools originally invited to participate at year 4 level, 121 agreed. One small primary school’s Board of Trustees declined participation because a new principal was being appointed. This school was replaced by a school of similar size and decile rating from the same district.

Sampling of Students
Each school sent a list of the names of all year 4 or year 8 students on their roll. Using computer-generated random numbers, we randomly selected the required number of students (12 or four plus eight in a pair of small schools), at the same time clustering them into random groups of four students. The schools were then sent a list of their selected students and invited to inform us if special care would be needed in assessing any of those children (e.g. children with disabilities or limited skills in English).

For the year 8 sample, we received 123 comments about particular students. In 70 cases, we randomly selected replacement students because the children initially selected had left the school between the time the roll was provided and the start of the assessment programme in the school, or were expected to be away or involved in special activities throughout the assessment week. Two students were replaced because of incorrect classification. The remaining 51 comments concerned children with special needs. Each such child was discussed with the school and a decision agreed. Seven students were replaced because they were very recent immigrants or overseas students who had extremely limited English-language skills. Sixteen students were replaced because they had disabilities or other problems of such seriousness that it was agreed that the students would be placed at risk if they participated. Participation was agreed upon for the remaining 28 students, but a special note was prepared to give additional guidance to the teachers who would assess them.

For the year 4 sample, we received 155 comments about particular students. Fifty-four students originally selected were replaced because they had left the school or were expected to be away throughout the assessment week. Nineteen students were replaced because of their NESB (Not from English-Speaking Background) status and very limited English, six because they were in Mäori immersion classes, and two because of a wrong year level. Forty-six students were replaced because they had disabilities or other problems of such seriousness the students appeared to be at risk if they participated. Special notes for the assessing teachers were made about 28 children retained in the sample.

Communication with Parents
Following these discussions with the school, Project staff prepared letters to all of the parents, including a copy of the NEMP brochure, and asked the schools to address the letters and mail them. Parents were told they could obtain further information from Project staff (using an 0800 number) or their school principal, and advised that they had the right to ask that their child be excluded from the assessment.

At the year 8 level, we received a number of phone calls including several from students or parents wanting more information about what would be involved. Nine students were replaced because they did not want to participate or their parents did not want them to (usually because of concern about missing regular classwork).

At the year 4 level we also received several phone calls from parents. Some wanted details confirmed or explained (notably about reasons for selection). Two children were replaced at their parents’ request.

Practical Arrangements with Schools
On the basis of preferences expressed by the schools, we then allocated each school to one of the five assessment weeks available and gave them contact information for the two teachers who would come to the school for a week to conduct the assessments. We also provided information about the assessment schedule and the space and furniture requirements, offering to pay for hire of a nearby facility if the school was too crowded to accommodate the assessment programme. This proved necessary in several cases.

Results of the Sampling Process
As a result of the considerable care taken, and the attractiveness of the assessment arrangements to schools and children, the attrition from the initial sample was quite low. About 3% of selected schools in the main samples did not participate, and less than 4% of the originally sampled children had to be replaced for reasons other than their transfer to another school or planned absence for the assessment week. The main samples can be regarded as very representative of the populations from which they were chosen (all children in New Zealand schools at the two class levels apart from the 1– 2% who were in special schools, Mäori immersion programmes, or schools with fewer than four year 4 or year 8 children).

Of course, not all the children in the samples actually could be assessed. Eleven student places in the year 8 sample and two in the year 4 sample were not filled because insufficient students were available in eight small schools. Six year 8 students and nine year 4 students left school at short notice and could not be replaced. Three year 8 students withdrew or were withdrawn by their parents too late to be replaced. Twenty year 8 students and twenty-two year 4 students were absent from school throughout the assessment week. Some other students were absent from school for some of their assessment sessions, and a very small percentage of performances were lost because of malfunctions in the video recording process. Some of the students ran out of time to complete the schedules of tasks. Nevertheless, for most of the tasks over 90% of the sampled students were assessed. Given the complexity of the Project, this is a very acceptable level of participation.

Composition of the Sample
Because of the sampling approach used, regions were fairly represented in the sample, in approximate proportion to the number of school children in the regions.