Although national monitoring has been designed primarily to present an overall national picture of student achievement, there is some provision for reporting on performance differences among subgroups of the sample. Eight demographic variables are available for creating subgroups, with students divided into subgroups on each variable, as detailed in Chapter 1.

Analyses of the relative performance of subgroups used the total score for each task, created as described in Chapter 1.

Five of the demographic variables related to the schools the students attended. For these five variables, statistical significance testing was used to explore differences in task performance among the subgroups. Where only two subgroups were compared (for School Type), differences in task performance between the two subgroups were checked for statistical significance using t-tests. Where three subgroups were compared, one-way analysis of variance was used to check for statistically significant differences among the three subgroups. Because the number of students included in each analysis was quite large (approximately 450), the statistical tests were quite sensitive to small differences. To reduce the likelihood of attention being drawn to unimportant differences, the critical level for statistical significance for tasks reporting results for individual students was set at p = .01 (so that differences this large or larger among the subgroups would not be expected by chance in more than one percent of cases). For tasks administered to teams or groups of students, p = .05 was used as the critical level to compensate for the smaller numbers of cases in the subgroups. For the first four of the five school variables, statistically significant differences among the subgroups were found for less than 11 percent of the tasks for both year 4 and year 8. For the remaining variable, statistically significant differences were found on more than half of the tasks at both levels. In the detailed report below, all “differences” mentioned are statistically significant (to save space, the words “statistically significant” are omitted).

Results were compared from students in large, medium-sized and small schools (exact definitions were given in Chapter 1).

For year 4 students, there was a difference among the three subgroups on one of the 30 tasks, with students from small schools scoring lowest on Link Task 19 (p45). There were no differences on questions of the Information Skills Survey (p47).

For year 8 students, there were no differences on any of the 46 tasks, or on questions of the Information Skills Survey.

Results were compared for students living in communities containing over 100,000 people (main centre), communities containing 10,000 to 100,000 people (provincial city) and communities containing less than 10,000 people (rural areas). For year 4 students, there were differences among the three subgroups on two of the 30 tasks. Students from the main centres scored highest on Breakdancing and lowest on Library Search. There were no differences on questions of the Information Skills Survey. For year 8 students, there were no differences on any of the 46 tasks, or on questions of the Information Skills Survey.

Results were compared for year 8 students attending full primary and intermediate (or middle) schools. There were no differences between these two subgroups on any of the 46 tasks, or on questions of the Information Skills Survey.

This year, for the first time, the NEMP samples included enough year 8 students attending year 7 to 13 high schools to permit comparisons between them and students attending intermediate schools. There were statistically significant differences (p < .01) on three of the 40 tasks attempted by individual students. Students from year 7 to 13 high schools scored higher on Hens, Atlas Y8 and Please, Mum!. There were no differences on questions of the Information Skills Survey.

Results achieved by students from Auckland, the rest of the North Island, and the South Island were compared.

For year 4 students, there were differences among the three sub-groups on one of the 30 tasks. Students from the rest of the North Island (excluding Auckland) scored highest on Library Search. There were no differences on questions of the Information Skills Survey.

For year 8 students, there were differences among the three subgroups on five of the 46 tasks: students from the South Island scored highest on Link Tasks 1, 2 and 3, City Mountains, and Please, Mum!. There were no differences on questions of the Information Skills Survey.

Schools are categorised by the Ministry of Education based on census data for the census mesh blocks where children attending the schools live. The resulting index takes into account household income levels and categories of employment. It uses 10 subdivisions, each containing 10 percent of schools (deciles 1 to 10). For our purposes, the bottom three deciles (1-3) formed the low decile group, the middle four deciles (4-7) formed the medium decile group, and the top three deciles (8-10) formed the high decile group. Results were compared for students attending schools in each of these three decile groups. For year 4 students, there were differences among the three subgroups on 17 of the 30 tasks, spread across the three task chapters. Because of the number of tasks showing differences, they are not listed here. Students in high decile schools performed better than students in low decile schools on all 17 tasks, usually with larger gaps between low and medium decile schools than between medium and high decile schools. There was also a difference on one question of the Information Skills Survey: students from low decile schools were most positive about hunting for information (question 5) For year 8 students, there were differences among the three subgroups on 25 of the 46 tasks, spread across the three task chapters but including 16 of the 21 year 8 tasks in Chapter 4. Because of the number of tasks showing differences, they are not listed here. Students in high decile schools performed better than students in low decile schools on all 25 tasks, usually with larger gaps between low and medium decile schools than between medium and high decile schools. There was also a difference on one question of the Information Skills Survey: students from low decile schools were most positive about writing down what they found out (question 8).

Three demographic variables related to the students themselves:

• Gender: boys and girls
• Ethnicity: Mäori, Pasifika and Pakeha (this term was used for all other students)
• Language used predominantly at home: English and other.

During the previous cycle of the Project (1999–2002), special supplementary samples of students from schools with at least 15 percent Pasifika students enrolled were included. These allowed the results of Pasifika students to be compared with those of Mäori and Pakeha students attending these schools. By 2002, with Pasifika enrolments having increased nationally, it was decided that from 2003 onwards a better approach would be to compare the results of Pasifika students in the main NEMP samples with the corresponding results for Mäori and Pakeha students. This gives a nationally representative picture, with the results more stable because the numbers of Mäori and Pakeha students in the main samples are much larger than their numbers previously in the special samples.

The analyses reported compare the performances of boys and girls, Pakeha and Mäori students, Pakeha and Pasifika students, and students from predominantly English-speaking and non-English-speaking homes.

For each of these three comparisons, differences in task performance between the two subgroups are described using “effect sizes” and statistical significance.

For each task and each year level, the analyses began with a t-test comparing the performance of the two selected subgroups and checking for statistical significance of the differences. Then the mean score obtained by students in one subgroup was subtracted from the mean score obtained by students in the other subgroup, and the difference in means was divided by the pooled standard deviation of the scores obtained by the two groups of students. This computed effect size describes the magnitude of the difference between the two subgroups in a way that indicates the strength of the difference and is not affected by the sample size. An effect size of +.30, for instance, indicates that students in the first subgroup scored, on average, three tenths of a standard deviation higher than students in the second subgroup.

For each pair of subgroups at each year level, the effect sizes of all available tasks were averaged to produce a mean-effect size for the curriculum area and year level, giving an overall indication of the typical performance difference between the two subgroups.

Results achieved by male and female students were compared using the effect-size procedures.

For year 4 students, the mean-effect size across the 24 tasks was 0.14 (girls averaged 0.14 standard deviations higher than boys). This difference is small. There were statistically significant (p < .01) differences favouring girls on 6 of the 24 tasks: Hens, Link Task 3, Bats, Link Task 8, Oh Pussy Cat, Pussy Cat! and Link Task 16. There was also a difference on one question of the Information Skills Survey: girls were more positive than boys about writing down what they found out (question 8).

For year 8 students, the mean-effect size across the 40 tasks was 0.27 (girls averaged 0.27 standard deviations higher than boys): a moderate difference. There were statistically significant differences on 24 of the 40 tasks, with girls performing better on all 24 tasks, spread across the three task chapters. Because of the number of tasks showing differences, they are not listed here. There were also differences on five of the eight questions of the Information Skills Survey. Girls reported that they more often had a really interesting study for which they had to find information (question 2) and more often voluntarily looked up information (question 3). Girls also were more positive about hunting for information (question 5), about how good they thought they were at hunting for information (question 6), and about how much they liked writing down what they found out (question 8).

Results achieved by Mäori, Pasifika and Pakeha (all other) students were compared using the effect-size procedures. First, the results for Pakeha students were compared to those for Mäori students. Second, the results for Pakeha students were compared to those for Pasifika students.

For year 4 students, the mean-effect size across the 24 tasks was 0.36 (Pakeha students averaged 0.36 standard deviations higher than Mäori students). This is a moderate difference. There were statistically significant differences (p <. 01) on 17 of the 24 tasks, spread across the three task chapters. Pakeha students scored higher than Mäori students on all 17 tasks. Because of the number of tasks showing differences, they are not listed here. There were no differences on questions of the Information Skills Survey.

For year 8 students, the picture was similar. The mean-effect size across the 40 tasks was 0.27 (Pakeha students averaged 0.27 standard deviations higher than Mäori students). This is a moderate difference. There were statistically significant differences on 18 of the 40 tasks, spread across the three task chapters. Pakeha students scored higher than Mäori students on all 18 tasks. Because of the number of tasks showing differences, they are not listed here. There were no differences on questions of the Information Skills Survey.

Readers should note that only 30 to 45 Pasifika students were included in the analysis for each task. This is lower than normally preferred for NEMP subgroup analyses, but has been judged adequate for giving a useful indication, through the overall pattern of results, of the Pasifika students’ performance. Because of the relatively small numbers of Pasifika students, p = .05 has been used here as the critical level for statistical significance.

For year 4 students, the mean-effect size across the 24 tasks was 0.37 (Pakeha students averaged 0.37 standard deviations higher than Pasifika students). This is a moderate difference. There were statistically significant differences on 14 of the 24 tasks, spread across the three task chapters. Pakeha students scored higher on all 14 tasks. Because of the number of tasks showing differences, they are not listed here. There was also a difference on one question of the Information Skills Survey: Pasifika students reported having to find information for a study more frequently (question 1).

For year 8 students, the mean-effect size across the 40 tasks was 0.46 (Pakeha students averaged 0.46 standard deviations higher than Pasifika students). This is a large difference. There were statistically significant differences on 29 of the 40 tasks, spread across the three task chapters, but including all tasks in Chapter 3. Pakeha students scored higher on all 29 tasks. Because of the number of tasks showing differences, they are not listed here. There were also differences on four questions of the Information Skills Survey. Pasifika students reported having to find information for a study more frequently (question 1) and more often voluntarily looking up information (question 3). They also were more positive about hunting for information (question 5), and about how much they liked writing down what they found out (question 8).

Results achieved by students who reported that English was the predominant language spoken at home were compared, using the effect-size procedures, with the results of students who reported predominant use of another language at home (most commonly an Asian or Pasifika language). Because of the relatively small numbers in the “other language” group, p = .05 has been used here as the critical level for statistical significance.

For year 4 students, the mean-effect size across the 24 tasks was 0.16 (students for whom English was the predominant language at home averaged 0.16 standard deviations higher than the other students). This is a small difference. There were statistically significant differences on three of the 24 tasks: students for whom English was the predominant language spoken at home scored higher on Link Task 1, Atlas Y4 and Oh Pussy Cat, Pussy Cat!. There was also a difference on one question of the Information Skills Survey. Students whose predominant language at home was not English reported that they more often had a really interesting study for which they had to find information (question 2).

For year 8 students, the mean-effect size across the 40 tasks was 0.18 (students for whom English was the predominant language at home averaged 0.18 standard deviations higher than the other students). This is a small difference. There were statistically significant differences on 16 of the 40 tasks: students for whom English was the predominant language spoken at home scored lower on Link Task 14, but higher on the other 15 tasks, spread across the three task chapters. Because of the number of tasks showing differences, they are not listed here. There were also differences on four questions of the Information Skills Survey. Students whose predominant language at home was not English reported that they more often voluntarily looked up information (question 3). They also were more positive about hunting for information (question 5), about how good they thought they were at hunting for information (question 6), and about how much they liked writing down what they found out (question 8)

School type (full primary, intermediate, or year 7 to 13 high school), school size, community size and geographic zone did not seem to be important factors predicting achievement on the information skills tasks. The same was true for the 2001 and 1997 assessments. However, there were statistically significant differences in the performance of students from low, medium and high decile schools on 57 percent of the tasks at year 4 level (compared to 43 percent in 2001 and 81 percent in 1997) and 54 percent of the tasks at year 8 level (compared to 71 percent in 2001 and 56 percent in 1997).

For the comparisons of boys with girls, Pakeha with Mäori, Pakeha with Pasifika students, and students for whom the predominant language at home was English with those for whom it was not, effect sizes were used. Effect size is the difference in mean (average) performance of the two groups, divided by the pooled standard deviation of the scores on the particular task. For this summary, these effect sizes were averaged across all tasks.

Year 4 girls averaged slightly higher than boys, with a mean effect size of 0.14 (compared to 0.06 in 2001). Year 8 girls averaged moderately higher than boys, with a mean effect size of 0.27 (compared to 0.15 in 2001). As was also true in 2001, the Information Skills Survey results at both year levels showed some evidence that girls were more positive than boys about information skills activities.

Pakeha students averaged moderately higher than Mäori students, with mean effect sizes of 0.36 for year 4 students and 0.27 for year 8 students (the corresponding figures in 2001 were 0.25 and 0.39).

Year 4 Pakeha students averaged moderately higher than Pasifika students, with a mean effect size of 0.37 (compared to 0.40 in 2001). Year 8 Pakeha students averaged substantially higher than Pasifika students, with a mean effect size of 0.48 (compared to 0.46 in 2001). The Information Skills Survey results showed that Pasifika students were more involved in and enthusiastic about some aspects of information skills.

Compared to students for whom the predominant language at home was English, students from homes where other languages predominated averaged slightly lower, with mean effect sizes of 0.16 for year 4 students and 0.18 for year 8 students. Comparative figures are not available for the assessments in 2001.