Main Samples, Assessed in English
In 2004, 2876 children from 254 schools were in the main samples
to participate in national monitoring. Half were in year 4, the other
half in year 8. At each level, 120 schools were selected randomly
from national lists of state, integrated and private schools teaching
at that level, with their probability of selection proportional to
the number of students enrolled in the level. The process used ensured
that each region was fairly represented. Schools with fewer than
four students enrolled at the given level were excluded from these
main samples, as were special schools and Mäori immersion schools
(such as Kura Kaupapa Mäori).
In May 2004, the Ministry of Education provided computer files containing
lists of eligible schools with year 4 and year 8 students, organised
by region and district, including year 4 and year 8 roll numbers
drawn from school statistical returns based on enrolments at 1 March
2004.
From these lists, we randomly selected 120 schools with year 4 students
and 120 schools with year 8 students. Schools with four students
in year 4 or 8 had about a one percent chance of being selected,
while some of the largest intermediate (year 7 and 8) schools had
a more than 90 percent chance of inclusion. In the two cases where
the same school was chosen at both year 4 and year 8 level, a replacement
year 4 school of similar size was chosen from the same region and
district, type and size of school.
Mäori Immersion Sample, Assessed
Predominantly in Te Reo
Details of the sample for the Mäori immersion assessments
will be reported separately.
Pairing Small Schools
At the year 8 level, four of the 120 chosen schools in the main sample
had fewer than 12 year 8 students. For each of these schools, we
identified the nearest small school meeting our criteria to be paired
with the first school. Wherever possible, schools with eight to 11
students were paired with schools with four to seven students, and
vice versa. However, the travelling distances between the schools
were also taken into account.
Similar pairing procedures were followed at the year 4 level. Eight
pairs and one trio of very small schools were included in the sample
of 120 schools.
Contacting Schools
In late May and early June, we attempted to telephone the principals
or acting principals of all schools in the year 8 sample. In these
calls, we briefly explained the purpose of national monitoring, the
safeguards for schools and students, and the practical demands that
participation would make on schools and students. We informed the
principals about the materials which would be arriving in the school
(a copy of a 20-minute NEMP videotape plus copies for all staff and
trustees of the general NEMP brochure and the information booklet
for sample schools). We asked the principals to consult with their
staff and Board of Trustees and confirm their participation by the
end of June.
A similar procedure was followed at the end of July with the principals
of the schools selected in the year 4 samples, and they were asked
to respond to the invitation by the end of August.
Response from Schools
Of the 254 schools originally invited to participate, 251 agreed.
Two schools in the year 8 sample declined to participate: one integrated
full-primary school because of ERO and special character reviews
scheduled during the assessment period, and one intermediate school
for indeterminate reasons. One school was replaced in the year 4
sample because of workload and stress issues associated with the
impending closure of the school.
The three schools not participating were replaced with schools from
the same district, matched as closely as possible on size and decile
rating.
Sampling of Students
Each school sent a list of the names of all year 4 or year 8 students
on their roll. Using computer-generated random numbers, we randomly
selected the required number of students (12 or four plus eight in
a pair of small schools), at the same time clustering them into random
groups of four students. The schools were then sent a list of their
selected students and invited to inform us if special care would
be needed in assessing any of those children (e.g. children with
disabilities or limited skills in English).
For the year 8 sample, we received 102 comments from 62 schools about
particular students. In 43 cases, we randomly selected replacement
students because the children initially selected had left the school
between
the time the roll was provided and the start of the assessment programme
in the school, or were expected to be away or involved in special
activities throughout the assessment week, or had been included in
the roll by mistake. The remaining 59 comments concerned children
with special needs. Each such child was discussed with the school
and a decision agreed. Seven students were replaced because they
were very recent immigrants or overseas students who had extremely
limited English-language skills. Seventeen students were replaced
because they had disabilities or other problems of such seriousness
that it was agreed that the students would be placed at risk if they
participated. Participation was agreed upon for the remaining 35
students, but a special note was prepared to give additional guidance
to the teachers who would assess them.
For the year 4 sample, we received 111 comments from 57 schools about
particular students. Thirty-two students originally selected were
replaced because the lists originally supplied were incorrect or
the student had left the school or was expected to be away throughout
the assessment week. Thirteen students were replaced because of their
NESB status and very limited English. Nineteen students were replaced
because they had disabilities or other problems of such seriousness
the students appeared to be at risk if they participated. Special
notes for the assessing teachers were made about 47 children retained
in the sample.
Communication with Parents
Following these discussions with the school, Project staff prepared
letters to all of the parents, including a copy of the NEMP brochure,
and asked the schools to address the letters and mail them. Parents
were told they could obtain further information from Project staff
(using an 0800 number) or their school principal, and advised that
they had the right to ask that their child be excluded from the assessment.
At the year 8 level, we received a number of phone calls including
several from students or parents wanting more information about what
would be involved. Two children were replaced because of severe medical
issues, one for religious reasons, one because the parents did not
want their child videorecorded, one because the child had missed
a lot of school, and one for unspecified reasons. In two further
cases the parent was happy for the child to participate but the child
asked not to.
At the year 4 level we also received several phone calls from
parents. Some wanted details confirmed or explained (notably
about reasons for selection). Two children were replaced at their
parents’ request.
Practical Arrangement with Schools
On the basis of preferences expressed by the schools, we then allocated
each school to one of the five assessment weeks available and gave
them contact information for the two teachers who would come to the
school for a week to conduct the assessments. We also provided information
about the assessment schedule and the space and furniture requirements,
offering to pay for hire of a nearby facility if the school was too
crowded to accommodate the assessment programme. This proved necessary
in several cases.
Results of the Sampling Process
As a result of the considerable care taken, and the attractiveness
of the assessment arrangements to schools and children, the attrition
from the initial sample was quite low. Only about one percent of
selected schools in the main samples did not participate, and slightly
more than two percent of the originally sampled children had to be
replaced for reasons other than their transfer to another school
or planned absence for the assessment week. The main samples can
be regarded as very representative of the populations from which
they were chosen (all children in New Zealand schools at the two
class levels apart from the one to two percent who were in special
schools, Mäori immersion programmes, or schools with fewer than
four year 4 or year 8 children).
Of course, not all the children in the samples actually could be
assessed. Two student places in each sample were not filled because
insufficient students were available in those schools. Thirteen year
8 students and nine year 4 students left school at short notice and
could not be replaced. Three year 8 students and one year 4 student
withdrew or were withdrawn by their parents too late to be replaced.
One year 4 student was found to be misclassified (actually year 5).
Twenty-nine year 8 students and 15 year 4 students were absent from
school throughout the assessment week. Some other students were absent
from school for some of their assessment sessions, and a small percentage
of performances were lost because of malfunctions in the video recording
process. Some of the students ran out of time to complete the schedules
of tasks. Nevertheless, for almost all of the tasks over 90 percent
of the sampled students were assessed. Given the complexity of the
Project, this is a very acceptable level of participation.
Composition of the Sample
Because of the sampling approach used, regions were fairly represented
in the sample, in approximate proportion to the number of school
children in the regions.
|
96kb