Kohia ngä taikäkä – Seek the heartwood:
Issues of validity in translating NEMP assessment tasks
 

TASK 1:  THE SANDWICH / TE HANAWITI

“The Sandwich” is a speaking task that was attempted by Year 4, Year 8 and Mäori Immersion students. The task was conducted in the one-to-one interview format. The students were presented with a picture book showing a story of a pig making a sandwich (the story was told without words, the only words were the title) and asked to make up their own story to go with the pictures. They were instructed to pretend they were telling the story to a young child, and to make the story as interesting as possible. They were then asked to take a few minutes to familiarise themselves with the story and then to tell the story in a “way that makes it fun to listen to” (English version of the task).

The results for this task are presented in Table 1. More than half the students in each of the subgroups reported in the table covered the main thread of the story well. Considerably fewer made the story detailed or interesting. When the results for Year 4 and Year 8 are compared, it can be seen that, for all five attributes examined, and overall, about 10 percent more Year 8 than Year 4 students scored highly. Students taught in the Mäori Immersion setting performed slightly better than Mäori students taught in general education, although these differences were not statistically significant (p ≤ 0.5%).

 Table 1: Results for “The Sandwich”
   
Assessment in English
Year 8 Mäori Students in General Education
Year 8 Mäori Students in Immersion Education
Year 4
Year 8
 ATTRIBUTE:  
Marks Awarded
% responses
 Covering the main thread
 of the story
 strong
2
60
73
62
83
 moderate
1
34
24
34
15
 weak
0
6
3
4
2
 
 Embellishing story
 strong
2
27
39
35
53
 moderate
1
53
50
47
21
 weak
0
20
11
18
26
 
 Clear oral expression of ideas
 strong
2
54
65
43
57
 moderate
1
34
30
57
30
 weak
0
8
5
10
13
 
 Making presentation  interesting
 strong
2
26
35
24
23
 moderate
1
54
52
57
41
 weak
0
20
13
19
36
 
 Overall effectiveness in  presenting story
 very high
3
8
15
9
34
 quite high
2
33
39
34
28
 moderate
1
47
40
50
23
 low
0
12
6
7
15
 
 % scoring 8 or above
 (out of a possible 11)
40
51
39
60

The task was selected for use as it was felt that it gave a good opportunity for students to tell a story. The book was chosen as the text for retelling as it appeared to contain elements of a story that most students could relate to. The illustrations are bright, cheerful and detailed, giving plenty of opportunity for description and elaboration. Students with a low level of confidence could attain some success by relying on a description of each page, rather than developing a narrative line.

This task is a link task, that is, it was designed and developed for use in 1996 and repeated in 2000 so trends over time could be observed. As students in Mäori Immersion education were not assessed in 1996, this was not a link task in that setting in 2000. At that time, the interests of, and suitability for, students being taught in the Mäori Immersion settings were not necessarily being considered. A different text could have been more suitable if that were the case.

This task would appear to be valid in terms of assessing an aspect of Mäori students’ skills and knowledge. It requires the construction of an oral monologue, responding to and interpreting visual cues in a self-structured account. Notwithstanding the particular text, there are issues in translation into a Mäori Immersion setting if the task is to be treated as equivalent in terms of interpretation of data.
For a second-language learner, the most accessible way to deal with this task is to simply ‘speak to the pictures’, recounting what is seen. Greater fluency is required to have internalised the structures concerned with shaping a narrative such as phrases dealing with time, or colloquial expressions which might be used in a humorous manner. Students may also be used to learning situations that provide scaffolding in terms of language extension and elaboration which is not provided here due to the open-ended nature of this task.

An examination of the requirements of the task also reveal some cultural cross-talk. Cultural constructs which may be taken as given in English can present some difficulties in translation. The concept of ‘fun’ or ‘funny’ is one such example. There is a certain idiom involved in these words which does not translate readily to Mäori as a word-for-word equivalent. In English, instructions are couched in an indirect manner which abstracts the task. For example, “Pretend you will tell the story to a young child”. The translation into Mäori became more direct. The construct of pretence, of imagining, indeed, of the mind and imagination, creates some tension for translators to present in Mäori in an unambiguous manner that is readily understood by students.


TASK 2:  

Task 2: Light the Lights/Whakakängia Ngä Rama
“Light the Lights” is a technology task that was attempted by Year 8 and Mäori Immersion students. The task was conducted in the stations format. The students were given a set of six wires (with alligator clips) and a board with two switches, a battery and a bulb attached. They were told that the board was to be wired up so two people could use it in a competition. They were asked to firstly wire up the board so one switch would make the light bulb glow, then draw the wires on the provided diagram. For part two of the task, they were asked to rewire the board so the light bulb would glow if either switch was pressed. This task was also a link task for the assessment in English.

The results for this task are presented in Table 2. In general, students struggled to complete this task, particularly wiring the circuit for two switches.

 Table 2: Results for “Light the Lights”
 
Year 8 Students Assessed in English
Year 8 Mäori Students in General Education
Year 8 Mäori Students in Immersion Education
 ATTRIBUTE:
 % correct
 Wired correctly for one switch
47
35
18
 Correctly wired for two switches
16
9
2

This task is an example of a task set in the technological area called “Electronics and Control” in Technology in the New Zealand Curriculum. While students may not have had direct experience wiring up this exact format, many students could have had opportunities to work with electrical circuits in this technological area, or to meet science objectives. Both Science in the New Zealand Curriculum (Ministry of Education, 1993b, p.76) and Pütaiao i roto i Te Matauranga o Aotearoa (Ministry of Education, 1996, p.40) state that “students should have had “learning experiences with . . . simple electrical circuits by the end of year 6”. However, NEMP does not collect data on school teaching programmes, so we cannot be certain that every student has had comparable learning experiences in this area. Proficiency with electrical circuits may also be being developed through students’ leisure activities.

This task, although translated and used in the NEMP assessments, may not reflect the teaching and learning situations that are occurring in Mäori Immersion settings. As in general education, Mäori Immersion schooling must deal with rationalising the “crowded curriculum”. In a context where all instruction will have at its core the learning of language, decisions are made about what language and contexts are most critical. More emphasis may be placed on aspects of learning which are seen to have a direct connection with specifically Mäori cultural experiences, therefore, an area which is not seen as traditionally Mäori may be considered to be less of a priority in the teaching setting.

Everyday vocabulary in English may be new curriculum jargon in Mäori that is not part of teachers’ language corpus, let alone that of their students. The word “battery” is a good example. It’s scientific equivalent, ‘electric cell’ may not be used by English-speaking students, but the word ‘battery’ could be expected to be part of the every day conversation in their home lives beyond school. In contrast, the Mäori word used in this context, ‘pühiko’, is comparatively recent. It is a composite word, crafted from two older words, pü and hiko. Pü has many different meanings which students could have experienced, but commonly means the base or the source. Hiko equates to ‘energy’, which immersion students could be familiar with as in the ‘flash’ of lightning, which in common usage means ‘electricity’. Students may also encounter the word “battery” rendered as a transliteration such as pateri, or peteri; or as a compound term such as päka hiko; or an alternative term such as unikä. These variations will juggle for exposure, comprehension, and recall by the student. Thus, the translated task becomes more complex due to the demands of language.


TASK 3:  KAI MOANA

  “Kai Moana” is a science task that was attempted by Year 4, Year 8 and Mäori Immersion students. The task was conducted in the stations format. Each student was provided with a set of ten stickers showing different types of seafoods and asked to complete a two part task. In part one, students were asked to stick each picture on the picture provided of a coastal scene, showing where that picture would usually be found, for example, in deep water, on the rocks and so on. In part two, students were asked to match the picture of the seafood to its name, the first one (paua) being done for them as a model. The ten seafoods (with Mäori names in brackets) were: paua (päua); kina; oyster (tio); crayfish (köura); crab (päpaka); snapper (tämure); mussel (kuku); eel (tuna); cockle (tuangi); and flounder (pätiki).

The results for this task are presented in Table 3. In both groups assessed for the assessment in English sample (Year 4 and Year 8), students showed a high level of competence in identifying the seafoods. Few students had difficulty in identifying the eel, crab or crayfish pictures. The oyster and the cockle pictures presented the most challenge. Year 8 students achieved better than the year 4 students when it came to identifying each seafood’s habitat. When the two groups of Mäori students are compared, on several occasions, Mäori Immersion students showed greater knowledge of where the seafood could be found, but less knowledge of the name of the seafood.

 Table 3: Results for “Kai Moana”
   
Assessment in English
Year 8 Mäori Students in General Education
Year 8 Mäori Students in Immersion Education
Year 4
Year 8
 SEAFOOD:
% correct
 Paua (Päua)  Location
21
41
55
69
   
 Kina  Identification
59
83
96
100
 Location
20
40
51
75
   
 Oyster (Tio)  Identification
57
94
98
71
 Location
19
44
43
65
   
 Crayfish (Köura)  Identification
94
98
98
100
 Location
32
53
60
62
   
 Crab (Päpaka)  Identification
97
98
98
82
 Location
64
74
65
86
   
 Snapper (Tämure)  Identification
66
87
94
75
 Location
48
70
67
76
   
 Mussel (Kuku)  Identification
76
89
98
60
 Location
21
39
41
63
   
 Eel (Tuna)  Identification
96
100
100
94
 Location
42
71
70
81
   
 Cockle (Tuangi)  Identification
50
71
84
56
 Location
54
60
56
73
   
 Flounder (Pätiki)  Identification
66
87
93
88
 Location
26
37
35
61


This task is an example of a task developed in English and transferred for use in the Mäori Immersion setting. One of the Level 3 Achievement Objectives in Science in the New Zealand Curriculum is “explain, using information from personal observation and library research, where and how a range of familiar New Zealand plants and animals live” (Ministry of Education, 1993b, p.58). Seafoods and their habitats were chosen as a context as it felt that this was important knowledge for the wider New Zealand community, and part of most students’ experience. ∫ This context also relies on knowledge and experience that goes beyond that of the school-based learning. It is likely that, for a number of students, it reflects ‘hands-on’ practice.

Students taught in the immersion setting generally performed better than their counterparts in the general education setting. However there were some anomalies here. Our feeling is that some students did not recognise the Mäori names for the kai moana such as tämure (snapper), kuku (mussel) and tuangi (cockle). English is the home language for many students who attend Mäori Immersion schooling, yet the use of English terms in their classrooms is discouraged. If students are gaining the knowledge acquisition required for this task from outside-of-school sources, this could have a negative effect in this testing situation. Either students are not familiar with the Mäori name provided on the tasksheet, or they could be reluctant to ‘code-switch’ by reverting to English words where they are unsure.

This is the only science task from the 1999 general education assessment where Mäori students performed statistically significantly better than Non-Mäori students, which implies this is an area of learning where Mäori students may have more experience. However, this task is at the knowledge end of the higher order thinking skills taxonomy. The challenge for NEMP lies in developing “cognitively demanding and context-embedded” (Cummins, 2000) assessment tasks.


TASK 4:  POWHIRI

"Powhiri” is a social studies task that was attempted by Year 4, Year 8 and Mäori Immersion students. The task was conducted in the interview format. For this activity, students were given a set of pictures showing people taking part in a welcoming ceremony on a marae. They were asked to put the pictures in an order and explain the ceremony to the interviewer. The marking schedule allowed for more than one correct answer, that is, there was not one particular order that was considered as the only correct ceremony. The results for this task are not presented here as the task will be reused in four years time as a link task. However, the results show that Year 8 students performed statistically significantly better than Year 4 students. Likewise, Mäori students in immersion settings performed statistically significantly better than Mäori students in the general education setting.

The idea for this task was proposed by a team of educators working in Mäori Immersion education and believed to be particularly appropriate for students being educated in this setting. It is closely based on a similar task “Marae” that was used in the 1997 assessment of social studies. While the understanding of the welcoming ceremony can be considered particularly appropriate for students educated in a Mäori Immersion setting, it is an important part of New Zealand’s cultural heritage, therefore it is appropriate to assess all students’ understandings of the process.

Mäori students achieved better results in this task than their non-Mäori counterparts. However, these results may not reflect the complexity of underlying factors. Students within mainstream may not have personal experience with this context, but have studied it as an ordered process of ritual with a recognisable start and finish. Mäori students are more likely to have been involved in the living concept of “marae” that is an essential part of their culture. Being used to playing at your family marae is a totally different experience to learning about it at school as a social studies topic. Therefore, students answering this question may come from different perspectives: being an observer of an unfamiliar process, or being a participant in an integral part of a culture. The results reported by NEMP do not reveal that two students who may achieve the same score may be operating within quite different schema.

next page

top of page    |    return to Probe Studies - INDEX   |    return to Probe Studies menu
  For further information and contact details for the Author    |    Contact USEE