Main
samples
In 2002, 2868 children from 249 schools were in the main samples
to participate in national monitoring. Half were in year 4, the
other half in year 8. At each level, 120 schools were selected
randomly from national lists of state, integrated and private schools
teaching at that level, with their probability of selection proportional
to the number of students enrolled in the level. The process used
ensured that each region was fairly represented. Schools with fewer
than four students enrolled at the given level were excluded from
these main samples, as were special schools and Mäori immersion
schools (such as Kura Kaupapa Mäori).
Early in May 2002, the Ministry of Education provided computer
files containing lists of eligible schools with year 4 and year
8 students, organised by region and district, including year 4
and year 8 roll numbers drawn from school statistical returns based
on enrolments at 1 March 2002.
From these lists, we randomly selected 120 schools with year 4
students and 120 schools with year 8 students. Schools with four
students in year 4 or 8 had about a one percent chance of being
selected, while some of the largest intermediate (year 7 and 8)
schools had a more than 90 percent chance of inclusion. In the
four cases where the same school was chosen at both year 4 and
year 8 level, a replacement year 4 school of similar size was chosen
from the same region and district, type and size of school.
Additional samples
From 1999 onwards, national monitoring has included additional
samples of students to allow the performance of special categories
of students to be reported.
To allow results for Pacific students to be compared with those
of Mäori students and ‘other’ students, 10 additional
schools were selected at year 4 level and 10 at year 8 level. These
were selected randomly from schools that had not been selected
in the main sample, had at least 15 percent Pacific students attending
the school, and had at least 12 students at the relevant year level.
To allow results for Mäori students learning in Mäori
immersion programmes to be compared with results for Mäori
children learning in English, 10 additional schools were selected
at year 8 level only. They were selected from Mäori immersion
schools (such as Kura Kaupapa Mäori) that had at
least four year 8 students, and from other schools that had at
least four year 8 students in classes classified as Level 1 immersion
(80 to 100 percent of instruction taking place in Mäori).
Only students that the schools reported to be in at least their
fifth year of immersion education were included in the sampling
process.
Pairing small schools
At the year 8 level, 5 of the 120 chosen schools in the main
sample had less than 12 year 8 students. For each of these schools,
we identified the nearest small school meeting our criteria to
be paired with the first school. Wherever possible, schools with
8 to 11 students were paired with schools with 4 to 7 students,
and vice versa. However, the travelling distances between the
schools were also taken into account. Four of the 10 schools
in the year 8 Mäori immersion sample also needed to be paired
with other schools of the same type.
Similar pairing procedures were followed at the year 4 level.
Four pairs were required in the main sample of 120 schools.
Contacting schools
Late in May, we attempted to telephone the principals or
acting principals of all schools in the year 8 samples (excluding
the 14 schools in the Mäori immersion sample). We made
contact with all schools within a week.
In our telephone calls with the principals, we briefly explained
the purpose of national monitoring, the safeguards for schools
and students, and the practical demands that participation would
make on schools and students. We informed the principals about
the materials which would be arriving at the school (a copy of
a 20-minute NEMP videotape plus copies for all staff and trustees
of the general NEMP brochure and the information booklet for
sample schools). We asked the principals to consult with their
staff and Board of Trustees and confirm their participation by
the end of June.
A similar procedure was followed at the end of July with the
principals of the schools selected in the year 4 samples, and
they were asked to respond to the invitation by the end of August.
The principals of the 14 schools in the Mäori immersion
sample were contacted in early August and asked to respond by
early September. They were sent brochures in both Mäori
and English.
Response from schools
Of the 283 schools originally invited to participate, 277 agreed.
Three schools in the main year 8 sample declined to participate:
an intermediate school because of severe space problems, a brand
new independent school because of the challenges associated with
getting established, and a full primary school because it was
its first year as a recapitated school and there had been a major
upheaval in their year 8 class. One school in the year 8 Pacific
sample was replaced because of major building work and a bereavement
that had disrupted the work of the school. In the Mäori
immersion sample, two schools chose not to participate and were
replaced by nearby schools.
Sampling of students
With their confirmation of participation, each school sent a
list of the names of all year 4 or year 8 students on their roll.
Using computer generated random numbers, we randomly selected
the required number of students (12, or 4 plus 8 in a pair of
small schools), at the same time clustering them into random
groups of four students. The schools were then sent a list of
their selected students and invited to inform us if special care
would be needed in assessing any of those children (e.g. children
with disabilities or limited skills in English).
At the year 8 level, we received 144 comments from schools about
particular students. In 41 cases, we randomly selected replacement
students because the children initially selected had left the
school between the time the roll was provided and the start of
the assessment programme in the school, or were expected to be
away throughout the assessment week, or had been included in
the roll by mistake. The remaining 103 comments concerned children
with special needs. Each such child was discussed with the school
and a decision agreed. Fifteen students were replaced because
they were very recent immigrants or overseas students who had
extremely limited English language skills. Nine students were
replaced because they had disabilities or other problems of such
seriousness that it was agreed that the students would be placed
at risk if they participated. Participation was agreed upon for
the remaining 79 students, but a special note was prepared to
give additional guidance to the teachers who would assess them.
In the corresponding operation at year 4 level, we received 142
comments from schools about particular students. Eighteen students
were changed because the lists originally supplied were incorrect
(students from other than year 4 included). Thirty-three students
originally selected were replaced because they had left the school
or were expected to be away throughout the assessment week. Twelve
students were replaced because of their NESB status and very
limited English. Nineteen students were replaced because they
had disabilities or other problems of such seriousness the students
appeared to be at risk if they participated. Special notes for
the assessing teachers were made about 60 children retained in
the sample.
Communication with parents
Following these discussions with the school, Project staff prepared
letters to all of the parents, including a copy of the NEMP brochure,
and asked the schools to address the letters and mail them. Parents
were told they could obtain further information from Project
staff (using an 0800 number) or their school principal, and advised
that they had the right to ask that their child be excluded from
the assessment.
At the year 8 level, we received a number of phone calls including
several from students wanting more information about what would
be involved. Five children were replaced as a result of these
contacts, one at the child’s request and four at the parents’ request.
At the year 4 level we also received several phone calls from
parents. Some wanted details confirmed or explained (notably
about reasons for selection). One child was replaced at his parents’ request,
because they were worried about his missing classes.
Practical arrangement with schools
On the basis of preferences expressed by the schools, we then
allocated each school to one of the five assessment weeks available
and gave them contact information for the two teachers who would
come to the school for a week to conduct the assessments. We
also provided information about the assessment schedule and the
space and furniture requirements, offering to pay for hire of
a nearby facility if the school was too crowded to accommodate
the assessment programme. This proved necessary in several cases.
Results of the sampling process
As a result of the considerable care taken, and the attractiveness
of the assessment arrangements to schools and children, the attrition
from the initial sample was quite low. Only about two percent
of selected schools did not participate, and less than two percent
of the originally sampled children had to be replaced for reasons
other than their transfer to another school or planned absence
for the assessment week. The sample can be regarded as very representative
of the population from which it was chosen (all children in New
Zealand schools at the two class levels except the one to two
percent in special schools or schools with less than four year
4 or year 8 children).
Of course, not all the children in the samples actually could
be assessed. Eleven year 8 students, 26 year 4 students, and
5 Mäori immersion students left school at short notice and
could not be replaced. Parents withdrew 2 year 8 students and
1 year 4 student too late to be replaced. One year 4 student
was found to be in a Mäori immersion programme, and should
not be assessed in English. A further 12 year 8 students, 16
year 4 students, and 7 Mäori immersion students were absent
from school throughout the assessment week. Some others were
absent from school for some of their assessment sessions, and
a small percentage of performances were lost because of malfunctions
in the video recording process. Some of the students ran out
of time to complete the schedules of tasks. Nevertheless, for
many tasks over 95 percent of the sampled students were assessed.
No task had less than 90 percent of the sampled students assessed.
Given the complexity of the Project, this is a very acceptable
level of participation.
Composition
of the sample
Because of the sampling approach used, regions were fairly represented
in the sample, in approximate proportion to the number of school
children in the regions.
|
96kb