Main
Samples, Assessed in English |
In
2005, 2879 children from 248 schools were in the main samples
to participate in national monitoring. Half were in year 4,
the other half in year 8. At each level, 120 schools were selected
randomly from national lists of state, integrated and private
schools teaching at that level, with their probability of selection
proportional to the number of students enrolled in the level.
The process used ensured that each region was fairly represented.
Schools with fewer than four students enrolled at the given
level were excluded from these main samples, as were special
schools and Mäori immersion schools (such as Kura Kaupapa
Mäori).
In May 2005, the Ministry of Education provided computer files
containing lists of eligible schools with year 4 and year 8 students,
organised by region and district, including year 4 and year 8
roll numbers drawn from school statistical returns based on enrolments
at 1 March 2005.
From these lists, we randomly selected 120 schools with year
4 students and 120 schools with year 8 students. Schools with
four students in year 4 or 8 had about a one percent chance of
being selected, while some of the largest intermediate (year
7 and 8) schools had a more than 90 percent chance of inclusion.
Mäori
Immersion Sample, Assessed Predominantly in Te Reo |
Details
of the sample for the Mäori immersion assessments will be
reported separately.
At the year
8 level, five of the 120 chosen schools in the main sample had
fewer than 12 year 8 students. For each of these schools, we
identified the nearest small school meeting our criteria to be
paired with the first school. Wherever possible, schools with
eight to 11 students were paired with schools with four to seven
students, and vice versa. However, the travelling distances between
the schools were also taken into account.
Similar pairing procedures were followed at the year 4 level. Three
pairs of very small schools were included in the sample of 120
schools.
In late May
and early June, we telephoned the principals or acting principals
of all schools in the year 8 sample. In these calls, we briefly
explained the purpose of national monitoring, the safeguards
for schools and students, and the practical demands that participation
would make on schools and students. We informed the principals
about the materials which would be arriving in the school (a
copy of a 20-minute NEMP videotape plus copies for all staff
and trustees of the general NEMP brochure and the information
booklet for sample schools). We asked the principals to consult
with their staff and Board of Trustees and confirm their participation
by the end of June.
A similar procedure was followed at the end of July with the principals
of the schools selected in the year 4 samples, and they were asked
to respond to the invitation by the end of August.
Of the 248
schools originally invited to participate, 247 agreed. A year
7 to 13 integrated high school in the year 8 sample declined
to participate because of heavy external demands in the previous
year. It was replaced by another integrated school. One very
small school in the year 4 sample that was willing to participate
was replaced by a similar school because the number of students
available in the original school declined to less than the number
required (eight).
Each
school sent a list of the names of all year 4 or year 8 students
on their roll. Using computer-generated random numbers, we randomly
selected the required number of students (12 or four plus eight
in a pair of small schools), at the same time clustering them
into random groups of four students. The schools were then sent
a list of their selected students and invited to inform us if
special care would be needed in assessing any of those children
(e.g. children with disabilities or limited skills in English).
For the year 8 sample, we received 103 comments about particular
students. In 43 cases, we randomly selected replacement students
because the children initially selected had left the school between
the time the roll was provided and the start of the assessment
programme in the school, or were expected to be away or involved
in special activities throughout the assessment week, or had been
included in the roll by mistake. Two more were replaced because
they were in Mäori immersion classes. The remaining 58 comments
concerned children with special needs. Each such child was discussed
with the school and a decision agreed. Eight students were replaced
because they were very recent immigrants or overseas students who
had extremely limited English-language skills. Twenty-nine students
were replaced because they had disabilities or other problems of
such seriousness that it was agreed that the students would be
placed at risk if they participated. Participation was agreed upon
for the remaining 21 students, but a special note was prepared
to give additional guidance to the teachers who would assess them.
For the year 4 sample, we received 128 comments about particular
students. Forty-seven students originally selected were replaced
because a student had left the school or was expected to be away
throughout the assessment week. Thirteen students were replaced
because of their NESB status and very limited English, and two
because they were in Mäori immersion classes. Twenty-five
students were replaced because they had disabilities or other problems
of such seriousness the students appeared to be at risk if they
participated. Special notes for the assessing teachers were made
about 41 children retained in the sample.
Communication
with Parents |
Following
these discussions with the school, Project staff prepared letters
to all of the parents, including a copy of the NEMP brochure,
and asked the schools to address the letters and mail them. Parents
were told they could obtain further information from Project
staff (using an 0800 number) or their school principal, and advised
that they had the right to ask that their child be excluded from
the assessment.
At the year 8 level, we received a number of phone calls including
several from students or parents wanting more information about
what would be involved. Six children were replaced because they
did not want to participate or their parents did not want them
to.
At the year 4 level we also received several phone calls from
parents. Some wanted details confirmed or explained (notably
about reasons for selection). Five children were replaced at
their parents’ request.
Practical
Arrangement with Schools |
On the basis
of preferences expressed by the schools, we then allocated each
school to one of the five assessment weeks available and gave
them contact information for the two teachers who would come
to the school for a week to conduct the assessments. We also
provided information about the assessment schedule and the space
and furniture requirements, offering to pay for hire of a nearby
facility if the school was too crowded to accommodate the assessment
programme. This proved necessary in several cases.
Results
of the Sampling Process |
As
a result of the considerable care taken, and the attractiveness
of the assessment arrangements to schools and children, the attrition
from the initial sample was quite low. Less than one percent
of selected schools in the main samples did not participate,
and less than three percent of the originally sampled children
had to be replaced for reasons other than their transfer to another
school or planned absence for the assessment week. The main samples
can be regarded as very representative of the populations from
which they were chosen (all children in New Zealand schools at
the two class levels apart from the one to two percent who were
in special schools, Mäori immersion programmes, or schools
with fewer than four year 4 or year 8 children).
Of course, not all the children in the samples actually could be
assessed. One student place in the year 4 sample was not filled
because insufficient students were available in that schools. Ten
year 8 students and 12 year 4 students left school at short notice
and could not be replaced. Five year 8 students were overseas or
on holiday for the week of the assessment. One year 8 and one year
4 student withdrew, or were withdrawn by their parents, too late
to be replaced. Fourteen year 8 students and 14 year 4 students
were absent from school throughout the assessment week. Some other
students were absent from school for some of their assessment sessions,
and a small percentage of performances were lost because of malfunctions
in the video-recording process. Some of the students ran out of
time to complete the schedules of tasks. Nevertheless, for almost
all of the tasks over 90 percent of the sampled students were assessed.
Given the complexity of the Project, this is a very acceptable
level of participation.
Composition
of the Sample |
Because of
the sampling approach used, regions were fairly represented in
the sample, in approximate proportion to the number of school
children in the regions.
REGION |
PERCENTAGES
OF STUDENTS FROM EACH REGION |
|
REGION |
%
year 4 sample |
%
year 8 sample |
|
Northland |
4.2 |
4.2 |
|
Auckland |
33.3 |
32.5 |
|
Waikato |
10.0 |
10.0 |
|
Bay
of Plenty/Poverty Bay |
8.3 |
8.3 |
|
Hawkes
Bay |
4.2 |
3.3 |
|
Taranaki |
2.5 |
3.3 |
|
Wanganui/Manawatu |
5.0 |
5.8 |
|
Wellington/Wairarapa |
10.8 |
10.0 |
|
Nelson/Marlborough/West
Coast |
4.2 |
4.2 |
|
Canterbury |
11.7 |
11.7 |
|
Otago |
4.2 |
4.2 |
|
Southland |
1.7 |
2.5 |
|
|
|
|
DEMOGRAPHY |
DEMOGRAPHIC
VARIABLES:
PERCENTAGES OF STUDENTS IN EACH CATEGORY |
|
VARIABLE |
CATEGORY |
%
year 4 sample |
%
year 8 sample |
|
Gender |
Male |
51 |
52 |
|
|
Female |
49 |
48 |
|
Ethnicity |
Pakeha |
70 |
74 |
|
|
Mäori |
21 |
18 |
|
|
Pasifika |
9 |
8 |
|
Geographic
Zone |
Greater
Auckland |
33 |
32 |
|
|
Other
North Island |
45 |
46 |
|
|
South
Island |
22 |
22 |
|
Community
Size |
<
10,000 |
14 |
16 |
|
|
10,000
– 100,000 |
25 |
25 |
|
|
>
100,000 |
61 |
59 |
|
School
SES Index |
Bottom
30 percent |
28 |
22 |
|
|
Middle
40 percent |
40 |
47 |
|
|
Top
30 percent |
32 |
31 |
|
Main
language |
English |
87 |
87 |
|
at
home |
Other |
13 |
13 |
|
Size
of School |
<
25 y4 students |
19 |
|
|
|
25
– 60 y4 students |
41 |
|
|
|
>
60 y4 students |
40 |
|
|
|
<35
y8 students |
|
18 |
|
|
35
– 150 y8 students |
|
37 |
|
|
>
150 y8 students |
|
45 |
|
Type
of School |
Full
Primary |
|
32 |
|
|
Intermediate
or Middle |
|
48 |
|
|
Year
7 to 13 High School |
|
14 |
|
|
Other (not
analysed) |
|
6 |
|