Kohia
ngä taikäkä – Seek the heartwood:
Issues of validity in translating NEMP assessment tasks |
|
TASK
1: THE SANDWICH / TE HANAWITI
|
“The
Sandwich” is a speaking task that was attempted by Year 4, Year
8 and Mäori Immersion students. The task was conducted in the one-to-one
interview format. The students were presented with a picture book showing
a story of a pig making a sandwich (the story was told without words,
the only words were the title) and asked to make up their own story to
go with the pictures. They were instructed to pretend they were telling
the story to a young child, and to make the story as interesting as possible.
They were then asked to take a few minutes to familiarise themselves with
the story and then to tell the story in a “way that makes it fun
to listen to” (English version of the task).
The results for this task are presented in Table 1. More than half the
students in each of the subgroups reported in the table covered the main
thread of the story well. Considerably fewer made the story detailed or
interesting. When the results for Year 4 and Year 8 are compared, it can
be seen that, for all five attributes examined, and overall, about 10
percent more Year 8 than Year 4 students scored highly. Students taught
in the Mäori Immersion setting performed slightly better than Mäori
students taught in general education, although these differences were
not statistically significant (p ≤ 0.5%).
Table 1: Results for “The
Sandwich” |
|
|
|
Assessment
in English |
Year
8 Mäori Students in General Education |
Year
8 Mäori Students in Immersion Education |
Year
4 |
Year
8 |
ATTRIBUTE: |
|
Marks
Awarded |
%
responses |
Covering
the main thread
of the story
|
strong
|
2 |
60 |
73 |
62 |
83 |
moderate
|
1 |
34 |
24 |
34 |
15 |
weak
|
0 |
6 |
3 |
4 |
2 |
|
Embellishing story
|
strong
|
2 |
27 |
39 |
35 |
53 |
moderate
|
1 |
53 |
50 |
47 |
21 |
weak
|
0 |
20 |
11 |
18 |
26 |
|
Clear
oral expression of ideas
|
strong
|
2 |
54 |
65 |
43 |
57 |
moderate
|
1 |
34 |
30 |
57 |
30 |
weak
|
0 |
8 |
5 |
10 |
13 |
|
Making
presentation interesting
|
strong
|
2 |
26 |
35 |
24 |
23 |
moderate
|
1 |
54 |
52 |
57 |
41 |
weak
|
0 |
20 |
13 |
19 |
36 |
|
Overall
effectiveness in presenting story
|
very
high |
3 |
8 |
15 |
9 |
34 |
quite
high |
2 |
33 |
39 |
34 |
28 |
moderate |
1 |
47 |
40 |
50 |
23 |
low |
0 |
12 |
6 |
7 |
15 |
|
%
scoring 8 or above
(out of a possible 11)
|
|
40 |
51 |
39 |
60 |
|
The task was selected
for use as it was felt that it gave a good opportunity for students to
tell a story. The book was chosen as the text for retelling as it appeared
to contain elements of a story that most students could relate to. The
illustrations are bright, cheerful and detailed, giving plenty of opportunity
for description and elaboration. Students with a low level of confidence
could attain some success by relying on a description of each page, rather
than developing a narrative line.
This task is a link task, that is, it was designed and developed for use
in 1996 and repeated in 2000 so trends over time could be observed. As
students in Mäori Immersion education were not assessed in 1996,
this was not a link task in that setting in 2000. At that time, the interests
of, and suitability for, students being taught in the Mäori Immersion
settings were not necessarily being considered. A different text could
have been more suitable if that were the case.
This task would appear to be valid in terms of assessing an aspect of
Mäori students’ skills and knowledge. It requires the construction
of an oral monologue, responding to and interpreting visual cues in a
self-structured account. Notwithstanding the particular text, there are
issues in translation into a Mäori Immersion setting if the task
is to be treated as equivalent in terms of interpretation of data.
For a second-language learner, the most accessible way to deal with this
task is to simply ‘speak to the pictures’, recounting what
is seen. Greater fluency is required to have internalised the structures
concerned with shaping a narrative such as phrases dealing with time,
or colloquial expressions which might be used in a humorous manner. Students
may also be used to learning situations that provide scaffolding in terms
of language extension and elaboration which is not provided here due to
the open-ended nature of this task.
An examination of the requirements of the task also reveal some cultural
cross-talk. Cultural constructs which may be taken as given in English
can present some difficulties in translation. The concept of ‘fun’
or ‘funny’ is one such example. There is a certain idiom involved
in these words which does not translate readily to Mäori as a word-for-word
equivalent. In English, instructions are couched in an indirect manner
which abstracts the task. For example, “Pretend you will tell the
story to a young child”. The translation into Mäori became
more direct. The construct of pretence, of imagining, indeed, of the mind
and imagination, creates some tension for translators to present in Mäori
in an unambiguous manner that is readily understood by students.
|
TASK
2:
|
Task
2: Light the Lights/Whakakängia Ngä Rama
“Light the Lights” is a technology task that was attempted
by Year 8 and Mäori Immersion students. The task was conducted in
the stations format. The students were given a set of six wires (with
alligator clips) and a board with two switches, a battery and a bulb attached.
They were told that the board was to be wired up so two people could use
it in a competition. They were asked to firstly wire up the board so one
switch would make the light bulb glow, then draw the wires on the provided
diagram. For part two of the task, they were asked to rewire the board
so the light bulb would glow if either switch was pressed. This task was
also a link task for the assessment in English.
The results for this task are presented in Table 2. In general, students
struggled to complete this task, particularly wiring the circuit for two
switches.
Table
2: Results for “Light the Lights” |
|
Year
8 Students Assessed in English |
Year
8 Mäori Students in General Education |
Year
8 Mäori Students in Immersion Education |
ATTRIBUTE: |
%
correct |
Wired
correctly for one switch |
47 |
35 |
18 |
Correctly
wired for two switches |
16 |
9 |
2 |
|
This task is an example
of a task set in the technological area called “Electronics and
Control” in Technology in the New Zealand Curriculum. While students
may not have had direct experience wiring up this exact format, many students
could have had opportunities to work with electrical circuits in this
technological area, or to meet science objectives. Both Science in the
New Zealand Curriculum (Ministry of Education, 1993b, p.76) and Pütaiao
i roto i Te Matauranga o Aotearoa (Ministry of Education, 1996, p.40)
state that “students should have had “learning experiences
with . . . simple electrical circuits by the end of year 6”. However,
NEMP does not collect data on school teaching programmes, so we cannot
be certain that every student has had comparable learning experiences
in this area. Proficiency with electrical circuits may also be being developed
through students’ leisure activities.
This task, although translated and used in the NEMP assessments, may not
reflect the teaching and learning situations that are occurring in Mäori
Immersion settings. As in general education, Mäori Immersion schooling
must deal with rationalising the “crowded curriculum”. In
a context where all instruction will have at its core the learning of
language, decisions are made about what language and contexts are most
critical. More emphasis may be placed on aspects of learning which are
seen to have a direct connection with specifically Mäori cultural
experiences, therefore, an area which is not seen as traditionally Mäori
may be considered to be less of a priority in the teaching setting.
Everyday vocabulary in English may be new curriculum jargon in Mäori
that is not part of teachers’ language corpus, let alone that of
their students. The word “battery” is a good example. It’s
scientific equivalent, ‘electric cell’ may not be used by
English-speaking students, but the word ‘battery’ could be
expected to be part of the every day conversation in their home lives
beyond school. In contrast, the Mäori word used in this context,
‘pühiko’, is comparatively recent. It is a composite
word, crafted from two older words, pü and hiko. Pü has many
different meanings which students could have experienced, but commonly
means the base or the source. Hiko equates to ‘energy’, which
immersion students could be familiar with as in the ‘flash’
of lightning, which in common usage means ‘electricity’. Students
may also encounter the word “battery” rendered as a transliteration
such as pateri, or peteri; or as a compound term such as päka hiko;
or an alternative term such as unikä. These variations will juggle
for exposure, comprehension, and recall by the student. Thus, the translated
task becomes more complex due to the demands of language.
|
TASK
3: KAI MOANA
|
“Kai Moana” is a science task that was attempted by Year 4,
Year 8 and Mäori Immersion students. The task was conducted in the
stations format. Each student was provided with a set of ten stickers
showing different types of seafoods and asked to complete a two part task.
In part one, students were asked to stick each picture on the picture
provided of a coastal scene, showing where that picture would usually
be found, for example, in deep water, on the rocks and so on. In part
two, students were asked to match the picture of the seafood to its name,
the first one (paua) being done for them as a model. The ten seafoods
(with Mäori names in brackets) were: paua (päua); kina; oyster
(tio); crayfish (köura); crab (päpaka); snapper (tämure);
mussel (kuku); eel (tuna); cockle (tuangi); and flounder (pätiki).
The results for this task are presented in Table 3. In both groups assessed
for the assessment in English sample (Year 4 and Year 8), students showed
a high level of competence in identifying the seafoods. Few students had
difficulty in identifying the eel, crab or crayfish pictures. The oyster
and the cockle pictures presented the most challenge. Year 8 students
achieved better than the year 4 students when it came to identifying each
seafood’s habitat. When the two groups of Mäori students are
compared, on several occasions, Mäori Immersion students showed greater
knowledge of where the seafood could be found, but less knowledge of the
name of the seafood.
Table
3: Results for “Kai Moana” |
|
|
Assessment
in English |
Year
8 Mäori Students in General Education |
Year
8 Mäori Students in Immersion Education |
Year
4 |
Year
8 |
SEAFOOD: |
%
correct |
Paua
(Päua) |
Location |
21 |
41 |
55 |
69 |
|
|
|
|
|
Kina |
Identification |
59 |
83 |
96 |
100 |
Location |
20 |
40 |
51 |
75 |
|
|
|
|
|
Oyster
(Tio) |
Identification |
57 |
94 |
98 |
71 |
Location |
19 |
44 |
43 |
65 |
|
|
|
|
|
Crayfish
(Köura) |
Identification |
94 |
98 |
98 |
100 |
Location |
32 |
53 |
60 |
62 |
|
|
|
|
|
Crab
(Päpaka) |
Identification |
97 |
98 |
98 |
82 |
Location |
64 |
74 |
65 |
86 |
|
|
|
|
|
Snapper
(Tämure) |
Identification |
66 |
87 |
94 |
75 |
Location |
48 |
70 |
67 |
76 |
|
|
|
|
|
Mussel
(Kuku) |
Identification |
76 |
89 |
98 |
60 |
Location |
21 |
39 |
41 |
63 |
|
|
|
|
|
Eel
(Tuna) |
Identification |
96 |
100 |
100 |
94 |
Location |
42 |
71 |
70 |
81 |
|
|
|
|
|
Cockle
(Tuangi) |
Identification |
50 |
71 |
84 |
56 |
Location |
54 |
60 |
56 |
73 |
|
|
|
|
|
Flounder
(Pätiki) |
Identification |
66 |
87 |
93 |
88 |
Location |
26 |
37 |
35 |
61 |
|
This task is
an example of a task developed in English and transferred for use in the
Mäori Immersion setting. One of the Level 3 Achievement Objectives
in Science in the New Zealand Curriculum is “explain, using information
from personal observation and library research, where and how a range
of familiar New Zealand plants and animals live” (Ministry of Education,
1993b, p.58). Seafoods and their habitats were chosen as a context as
it felt that this was important knowledge for the wider New Zealand community,
and part of most students’ experience. ∫ This context also
relies on knowledge and experience that goes beyond that of the school-based
learning. It is likely that, for a number of students, it reflects ‘hands-on’
practice.
Students taught in the immersion setting generally performed better than
their counterparts in the general education setting. However there were
some anomalies here. Our feeling is that some students did not recognise
the Mäori names for the kai moana such as tämure (snapper),
kuku (mussel) and tuangi (cockle). English is the home language for many
students who attend Mäori Immersion schooling, yet the use of English
terms in their classrooms is discouraged. If students are gaining the
knowledge acquisition required for this task from outside-of-school sources,
this could have a negative effect in this testing situation. Either students
are not familiar with the Mäori name provided on the tasksheet, or
they could be reluctant to ‘code-switch’ by reverting to English
words where they are unsure.
This is the only science task from the 1999 general education assessment
where Mäori students performed statistically significantly better
than Non-Mäori students, which implies this is an area of learning
where Mäori students may have more experience. However, this task
is at the knowledge end of the higher order thinking skills taxonomy.
The challenge for NEMP lies in developing “cognitively demanding
and context-embedded” (Cummins, 2000) assessment tasks.
|
TASK
4: POWHIRI
|
"Powhiri”
is a social studies task that was attempted by Year 4, Year 8 and Mäori
Immersion students. The task was conducted in the interview format. For
this activity, students were given a set of pictures showing people taking
part in a welcoming ceremony on a marae. They were asked to put the pictures
in an order and explain the ceremony to the interviewer. The marking schedule
allowed for more than one correct answer, that is, there was not one particular
order that was considered as the only correct ceremony. The results for
this task are not presented here as the task will be reused in four years
time as a link task. However, the results show that Year 8 students performed
statistically significantly better than Year 4 students. Likewise, Mäori
students in immersion settings performed statistically significantly better
than Mäori students in the general education setting.
The idea for this task was proposed by a team of educators working in
Mäori Immersion education and believed to be particularly appropriate
for students being educated in this setting. It is closely based on a
similar task “Marae” that was used in the 1997 assessment
of social studies. While the understanding of the welcoming ceremony can
be considered particularly appropriate for students educated in a Mäori
Immersion setting, it is an important part of New Zealand’s cultural
heritage, therefore it is appropriate to assess all students’ understandings
of the process.
Mäori students achieved better results in this task than their non-Mäori
counterparts. However, these results may not reflect the complexity of
underlying factors. Students within mainstream may not have personal experience
with this context, but have studied it as an ordered process of ritual
with a recognisable start and finish. Mäori students are more likely
to have been involved in the living concept of “marae” that
is an essential part of their culture. Being used to playing at your family
marae is a totally different experience to learning about it at school
as a social studies topic. Therefore, students answering this question
may come from different perspectives: being an observer of an unfamiliar
process, or being a participant in an integral part of a culture. The
results reported by NEMP do not reveal that two students who may achieve
the same score may be operating within quite different schema.
next
page |
|
top
of page | return
to Probe Studies - INDEX | return
to Probe Studies menu |
|
For
further information and contact details for the Author
| Contact USEE |