Main
samples
In 1999, 2866 children from 253 schools were in the main
samples to participate in national monitoring. About half
were in year 4, the other half in year 8. At each level,
120 schools were selected randomly from national lists of
state, integrated and private schools teaching at that level,
with their probability of selection proportional to the number
of students enrolled in the level. The process used ensured
that each region was fairly represented. Schools with fewer
than four students enrolled at the given level were excluded from
these main samples, as were special schools and Mäori immersion
schools (such as Kura Kaupapa Mäori).
Early
in June 1999, the Ministry of Education provided computer files
containing lists of eligible schools with year 4 and year 8
students, organised by region and district, including year
4 and year 8 roll numbers drawn from school statistical returns
based on enrolments at 1 March 1999.
From
these lists, we randomly selected 120 schools with year 4 students
and 120 schools with year 8 students. Schools with four students
in year 4 or 8 had about a one percent chance of being selected,
while some of the largest intermediate (year 7 and 8) schools
had a more than 90 percent chance of inclusion. In the two cases
where the same school was chosen at both year 4 and year 8 level,
a replacement year 4 school of similar size was chosen from the
same region and district, type and size of school.
Additional
samples
From 1999 onwards, national monitoring includes additional
samples of students to allow the performance of special categories
of students to be reported.
To
allow results for Pacific Island students to be compared with
those of Mäori students and other students, 10 additional
schools were selected at year 4 level and 10 at year 8 level.
These were selected randomly from schools that had not been selected
in the main sample, had at least 20 percent Pacific Island students
attending the school, and had at least 12 students at the relevant
year level.
To
allow results for Mäori students learning in Mäori
immersion programmes to be compared with results for Mäori
children learning in English, 10 additional schools were selected
at year 8 level only. They were selected from Mäori immersion
schools (such as Kura Kaupapa Mäori) that had at
least 4 year 8 students, and from other schools that had at least
4 year 8 students in classes classified as Level 1 immersion
(80 to 100 percent of instruction taking place in Mäori).
Pairing
small schools
At the year 8 level, 5 of the 120 chosen schools in the
main sample had less than 12 year 8 students. For each of these
schools, we identified the nearest small school meeting our criteria
to be paired with the first school. Wherever possible, schools
with 8 to 11 students were paired with schools with 4 to 7 students,
and vice versa. However, the travelling distances between the
schools were also taken into account. Three of the 10 schools
in the year 8 Mäori immersion sample also needed to be paired
with other schools of the same type.
Similar
pairing procedures were followed at the year 4 level. Eight pairs
were required in the main sample of 120 schools.
Contacting
schools
During the second and third weeks of June, we made telephone
contact with the principals or acting principals of all schools
in the year 8 samples (excluding the 13 schools in the Mäori
immersion sample).
In
our telephone calls with the principals, we briefly explained
the purpose of national monitoring, the safeguards for schools
and students, and the practical demands that participation would
make on schools and students. We informed the principals about
the materials which would be arriving in the school (a copy of
a 20 minute NEMP videotape plus copies for all staff and trustees
of the general NEMP brochure and the information booklet for
sample schools). We asked the principals to consult with their
staff and Board of Trustees and confirm their participation by
the middle of July.
A
similar procedure was followed in July with the principals of
the schools selected in the year 4 samples, and they were asked
to respond to the invitation by the end of August. The principals
of the 13 schools in the Mäori immersion sample at year
8 level were included at that time, and were sent brochures in
both Mäori and English.
Response
from schools
Of
the 293 schools invited to participate, 286 agreed. Five schools
in the year 8 sample declined, and two in the year 4 sample.
A reason for the higher than usual proportion declining at year
8 level appeared to be the short time between the first contact
with these schools and the planned start of assessments. Because
of delays in the provision of school roll information, year 8
schools had only two months notice instead of the usual three
months. Four schools declined participation because of extensive
other commitments during the assessment period, two because of
major building work taking place, and one because its Board did
not approve participation.
Sampling
of students
With their confirmation of participation, each school sent
a list of the names of all year 4 or year 8 students on their
roll. Using computer generated random numbers, we randomly selected
the required number of students (12, or 4 plus 8 in a pair of
small schools), at the same time clustering them into random
groups of four students. The schools were then sent a list of
their selected students and invited to inform us if special care
would be needed in assessing any of those children (e.g. children
with disabilities or limited skills in English).
At
the year 8 level, we received 96 comments from schools about
particular students. In 28 cases, we randomly selected replacement
students because the children initially selected had left the
school between the time the roll was provided and the start of
the assessment programme in the school, or were expected to be
away throughout the assessment week. The remaining 68 comments
concerned children with special needs. Each such child was discussed
with the school and a decision agreed. Three students were replaced
because they were very recent immigrants who had extremely limited
English language skills. Twenty students were replaced because
they had disabilities or other problems of such seriousness that
it was agreed that the students would be placed at risk if they
participated. Participation was agreed upon for the remaining
45 students, but a special note was prepared to give additional
guidance to the teachers who would assess them.
In
the corresponding operation at year 4 level, we received 111
comments from schools about particular students. Forty-two students
originally selected needed to be replaced because they had left
the school or were expected to be away throughout the assessment
week. Eight students were replaced because of their NESB (non-english
speaking background) status and very limited English. Twenty
students were replaced because they had disabilities or other
problems of such seriousness the students appeared to be at risk
if they participated (12 because of severe disabilities and 8
because of limited ability to cope emotionally with the assessment
situation). Special notes for the assessing teachers were made
about 41 children retained in the sample.
Communication
with parents
Following these discussions with the school, Project staff
prepared letters to all of the parents, including a copy of the
NEMP brochure, and asked the schools to address the letters and
mail them. Parents were told they could obtain further information
from Project staff (using an 0800 number) or their school principal,
and advised that they had the right to ask that their child be
excluded from the assessment.
Our
0800 number was monitored in evenings, as well as during the
day, for at least a week following each mailing of letters to
parents.
At
the year 8 level, we received about 20 phone calls including
several from students wanting more information about what would
be involved. The main issues raised by parents were our reasons
for selection of their child, a wish for fuller details or reiteration
of what would be involved, concerns about the use of video equipment,
or reluctance of the child to take part. Six children were replaced
as a result of these contacts, four at the child's request, and
two at the parents' request (one parent was concerned about difficulties
associated with her child's NESB status and the other gave no
reason).
At
the year 4 level we also received about 20 phone calls from parents.
Some wanted details confirmed or explained (notably about reasons
for selection). Four children were replaced at parents' request
because the parents were concerned about additional stress for
their children. Two children were replaced at their own request:
one because of shyness and the other because of worry about falling
behind in class. The other eight children were replaced at the
request of their parents (two because the parents were Exclusive
Brethren and did not want their children viewing video material,
two because of concern about the emotional demands on their children,
one because of concern about missing class time, and three with
no reason given).
Practical
arrangement with schools
On the basis of preferences expressed by the schools, we
then allocated each school to one of the five assessment weeks
available and gave them contact information for the two teachers
who would come to the school for a week to conduct the assessments.
We also provided information about the assessment schedule and
the space and furniture requirements, offering to pay for hire
of a nearby facility if the school was too crowded to accommodate
the assessment programme.
Results
of the sampling process
As a result of the considerable care taken, and the attractiveness
of the assessment arrangements to schools and children, the attrition
from the initial sample was quite low. Less than three percent
of selected schools did not participate, and about two percent
of the originally sampled children had to be replaced for reasons
other than their transfer to another school. The sample can be
regarded as very representative of the population from which it
was chosen (all children in New Zealand schools at the two class
levels except the one to two percent in special schools or schools
with less than four year 4 or year 8 children).
Of
course, not all the children in the sample actually were able
to be assessed. Eight year 8 students and eight year 4 students
left school at short notice and could not be replaced. A further
41 year 8 students and 19 year 4 students were absent from school
throughout the assessment week. Some others were absent from
school for some of their assessment sessions, and a small percentage
of performances were lost because of malfunctions in the video
recording process. Some of the students, particularly in the
Mäori immersion sample, ran out of time to complete the
schedules of tasks. Nevertheless, for many tasks over 95 percent
of the student sample were assessed. No task had less than 90
percent of the student sample assessed. Given the complexity
of the Project, this is a very acceptable success.
Composition
of the sample
Because
of the sampling approach used, regions were fairly represented
in the sample, in approximate proportion to the number of school
children in the regions.
Region |
64kb