Sampling procedures
In 1995, 2871 children from 256 schools were in the final samples to participate in national monitoring. About half were in year 4, the other half in year 8. At each level, 120 schools were selected randomly from national lists of state, integrated and private schools teaching at that level, with their probability of selection proportional to the number of students enrolled in the level. The process used ensured that each region was fairly represented. Schools with fewer than four students enrolled were excluded, as were special schools and Kura Kaupapa schools (by mutual agreement, the latter will be included from 1999 onwards).

Late in April 1995, the Ministry of Education provided computer files containing a lists of eligible schools with year 4 and year 8 students, organised by region and district, including year 4 and year 8 roll numbers drawn from school statistical returns based on enrolments at 1 March 1995.

From these lists, we randomly selected 120 schools with year 4 students and 120 schools with year 8 students. Schools with four students in year 4 or 8 had about a one percent chance of being selected, while some of the largest intermediate (year 7 and 8) schools had a more than 90 percent chance of inclusion. In the two cases where the same school was chosen at both year 4 and year 8 level, a replacement year 4 school of similar size was chosen from the same region and district, type and size of school.

Pairing small schools
At the year 8 level, five of the 120 chosen schools had less than 12 year 8 students. For each of these schools, we identifed the nearest small school which met our criteria to be paired with the first school. Wherever possible, schools with 8 to 11 students were paired with schools with 4 to 7 students, and vice versa. However, the travelling distances between the schools were also taken into account. Similar pairing procedures were followed at the year 4 level, creating 11 pairs of schools. Intriguingly, one of these pairs was on Great Barrier Island. Contacting schools During the first week of May, we attempted to telephone the principals or acting principals of all schools in the year 8 sample. We made contact with all schools during that period, where necessary leaving messages for the principal to return our call on the Project's 0800 number. Discussions with the last few principals were not completed until the first day or two of term 2.

In our telephone calls with the principals, we briefly explained the purpose of national monitoring, the safeguards for schools and students, and the practical demands participation would make on schools and students. We informed the principals about the materials which would be arriving in the school (a copy of a 15 minute NEMP videotape plus copies for all staff and trustees of the NEMP brochure and detailed booklet for sample schools). We asked the principals to consult with their staff and Board of Trustees and confirm their participation by the end of June.

A similar procedure was followed in early June with the principals of the schools selected in the year 4 sample, and they were asked to respond to the invitation by the middle of July.

Response from schools
Of the 256 schools invited to participate, 254 agreed. Both schools which declined were in the year 8 sample. One of these declined because of major disruptions over the previous 18 months for its year 8 students – an argument which seemed well founded. The other declined because the principal wanted to make a political protest about the implications of proposed changes in staffing allocations for his school and the lack of response from the Ministry to his protests. Each of these two schools was replaced in the sample: one by the nearest school of similar type and size, the other by a randomly chosen alternative school from the same district (there was no other school of the same type in that district).

[ top of the page ]

Sampling of students
With their confirmation of participation, each school sent a list of the names of all year 4 or year 8 students on their roll. Using computer generated random numbers, we randomly selected the required number of students (12, or 4 plus 8 in a pair of small schools), at the same time clustering them into random groups of four students. The schools were then sent a list of their selected students and invited to inform us if special care would be needed in assessing any of those children (e.g. children with disabilities or limited skills in English).

At the year 8 level, we received about 90 comments from schools about particular students. In about 45 cases, we randomly selected replacement students because the children initially selected had left the school between the time the roll was provided and the start of the assessment programme in the school, or were expected to be away throughout the assessment week. The remaining 45 comments concerned children with special needs. Each such child was discussed with the school and a decision agreed. Six students were replaced because they were very recent immigrants (within six months) who had extremely limited English language skills. One student was replaced because of severe physical health problems, and eight students were replaced because they had disabilities of such seriousness that it was agreed that the students would be placed at emotional risk if they participated. Participation was agreed upon for the remaining 30 students, but a special note was prepared to give additional guidance to the teachers who would assess them.

In the corresponding operation at year 4 level, we received about 125 comments from schools about particular students. In part, the larger number arose because there was a longer time gap between our receipt of the class rolls and the assessment weeks. This meant that about 75 children originally selected needed to be replaced because they had left the school. Eleven students were mentioned because of their ESOL status, and two because they were participants in total immersion Mäori language programmes. Of these, four very recent immigrants were replaced and assessment in Mäori was arranged for the two immersion students. Two students were replaced because they had been reclassified as year 3. Other special needs were mentioned for 26 children, and 7 of these children were replaced (3 because of very severe physical disabilities, and 4 because of concerns about their ability to cope with the assessment situation). Special notes for the assessing teachers were made about 28 children retained in the sample.

[ top of the page ]

Communication with parents Following these discussions with the school, Project staff prepared letters to all of the parents, including a copy of the NEMP brochure, and asked the schools to address the letters and mail them. Parents were told they could obtain further information from Project staff (using an 0800 number) or their school principal, and advised that they have the right to ask that their child be excluded from the assessment.

Our 0800 number was monitored in evenings, as well as during the day, for two weeks following each mailing of letters to parents.

At the year 8 level, we received about 18 phone calls and one e-mail message, including several from students wanting more information about what would be involved. The main issues raised by parents were our reasons for selection of their child, a wish for fuller details or reiteration of what would be involved, concern that limited skills would place their child at risk, or reluctance of the child to take part. Four children were replaced as a result of these contacts, one at parents' request (a child with special needs who had previously been discussed by school and Project staff), and three where the parents were happy for their child to participate but the child was not (one because friends had not been selected, the other two – from high SES schools – because they were concerned about performing badly or falling behind in their regular schoolwork).

At the year 4 level we received about 12 phone calls from parents. Some wanted details confirmed or explained (notably about reasons for selection). Five children were withdrawn at parents' request: one because the child was not allowed to watch video material, one because the parent did not want the child videotaped, two because of concern about added stress on children who were already under stress at school, and one for unspecified reasons).

[ top of the page ]

Practical arrangements with schools On the basis of preferences expressed by the schools, we then allocated each school to one of the five assessment weeks available and gave them contact information for the two teachers who would come to the school for a week to conduct the assessments. We also provided information about the assessment schedule and the space and furniture requirements, offering to pay for hire of a nearby facility if the school was too crowded to accommodate the assessment programme.

Results of the sampling process As a result of the considerable care taken, and the attractiveness of the assessment arrangements to schools and children, the attrition from the initial sample was very low. Less than one percent of selected schools did not participate, and less than two percent of the originally sampled children had to be replaced for reasons other than their transfer to another school. The sample can be regarded as very representative of the population from which it was chosen (all children in New Zealand schools at the two class levels except the one to two percent in special schools, Kura Kaupapa schools, or schools with less than four year 4 or year 8 children). Of course, not all the children in the sample were actually able to be assessed. Some were absent from school for some or all of their assessment sessions, and a small percentage of performances were lost because of malfunctions in the video recording process. For many tasks, over 95 percent of the sample were assessed. No task had less than 90 percent of the sample assessed. Given the complexity of the Project, this was a very acceptable success rate.

Composition of the sample
Because of the sampling approach used, regions were fairly represented in the sample, in approximate proportion to the number of school children in the regions.

[ top of the page ]

Region

Northland